Quotes on Software Testing: Load Testing, Unit Testing, Functional Testing

Helping Customers to Write Tests

Software Testing Magazine — Thu, 28 Nov 2019 09:46:59 +0000

Agile approaches aims to improve the collaboration between the development team and the end-users or the Product Owner in Scrum. As far as software testing is concerned, it is however deceptive to believe that this could happen without a strong contribution from software testing experts. In a test-driven approach, when developers pick up a user story to work on, the output of this conversation with the customer should include a set of tests that precisely specify what’s required. Customers are usually not software testers, so we must offer them guidance on this process and help them to identify the test scenarios we’ll need to consider (e.g., if they ask for new library members to choose a password when they join, we might ask the customer to consider what should happen if the password they choose is too weak, or what should happen if the password field is left blank, and so on.) Teams that expect customers to go away and write the tests themselves could be waiting a long time. This is a technical skill that takes a long time to master. If you have dedicated testers on your team, this is one area where they can prove very useful, helping the customer to articulate their needs as tests. Source : TDD, Jason Gorman, http://www.codemanship.co.uk/tdd.html

The post Helping Customers to Write Tests first appeared on Software Testing Magazine.

Testers and Product Owners in Agile Teams

Software Testing Magazine — Tue, 06 Mar 2018 15:00:00 +0000

When a software development team switches its approach to an Agile framework like Scrum, the place of software testing changes. In his book Scrum Product Ownership, Robert Galen describes the new role of software testers in Agile teams and their relationships with the product owners. It’s often the case that traditional testers struggle in their role transformation from Waterfall to agile methods. In traditional teams their role is very much at the end of the pipeline. Sure, they have some early planning and preparation to do but, in the end, developers throw software “over the wall” to them for testing. Typically, the development team is over schedule and there is always compression of testing plans and time. There is usually little cross-team collaboration and often they are blamed for overall application quality, even though they play only a part in that regard.$ In agile teams, the dynamic fundamentally changes. Instead of being at the end, testers need to move to the beginning of the process. They actually become your partner in defining and refining user stories. Not only in crafting the story definitions, but where testers can really shine, is in the areas of well-formed and comprehensive acceptance tests. Their primary and up-front role becomes, more or less, a Business Analyst on where they serve as a liaison between you and the development team; refining stories and ensuring their quality execution. They should be partners with development, often pairing with them, to assure that feature development is clear and aligned [...]

The post Testers and Product Owners in Agile Teams first appeared on Software Testing Magazine.

Crafting a Mobile Software Testing Strategy

Software Testing Magazine — Wed, 02 Mar 2016 10:05:31 +0000

In a perfect software development world, you have all the time and the resources available to test every aspect of your mobile apps. In reality, the time and resources for software testing are limited. In his book “Tap Into Mobile Application Testing“, Jonathan Kohl discusses how to define a strategy for mobile testing projects. Whenever you test, your job is to use your limited time and resources to generate ideas, execute tests, observe and document what happens to the application you are testing, and provide that information to the people who need it. While that sounds simple, software can be used in many ways, in many environments, for different purposes by different people with different handsets. There is a potentially limitless universe of options to test out. It is up to you to find an optimal path, and that’s just what you’ll do. You have two choices: 1. Choice One: Ignore thinking about strategy and therefore, implicitly create a strategy. That means you don’t really think about it and just dive in and test. If that’s your choice, you’ll likely miss important information and problems. You may do what you have always done before on testing projects, or ask someone else to tell you what to test. You can be passive and hope you stumble on the important problems, (and win that lottery) or you can be active and strategic about how you spend your time. 1. Choice Two: Think about and explicitly create a strategy to maximize your time [...]

The post Crafting a Mobile Software Testing Strategy first appeared on Software Testing Magazine.

Test Automation is Like Software Development

Software Testing Magazine — Wed, 06 Jan 2016 16:24:47 +0000

In many software development organizations, people make a difference between the software developers who actually write code and the software testers that are just there to perform some software quality activities. In the book “Experiences of Test Automation”, software testers from SAP share their view that test automation requires the same mindset than software development. One of the lessons we learned at SAP over the years was very simple: Test automation is nothing but software development. This statement, of course, sounds contradictory to the message some test tool vendors would like to send to the market: Test automation is easy, no programming skills are required, and it is best done by end users. At least for us, we can say that any test automation project that did not follow software development processes and standards failed or ended up with huge maintenance efforts. Proper test specifications, reviews, coding standards, and – probably most important – a strict reuse principle have been proven as key success factors. Source: Christoph Mecke, Melanie Reinwarth & Armin Gienger in “Experiences of Test Automation – Case Studies of Software Test Automation”, Dorothy Graham and Mark Fewster editors, Addison-Wesley Some software developers have considered software testers as people who weren’t good enough to write code. The reality is that creating and managing a software testing infrastructure requires the same skills than programming. Software testers should behave like programmers with their testing script and use the same tools and practices: coding and naming standards, configuration management, code review, [...]

The post Test Automation is Like Software Development first appeared on Software Testing Magazine.

Benefits and Limitations of Test Monkeys

Software Testing Magazine — Mon, 19 Oct 2015 14:00:49 +0000

A test monkey is an automated tool uses for random application testing. Unlike automated regression tests, test monkeys explore the system in a new way each time the test is run. In the book ” Experiences of Test Automation”, there is an interesting chapter that explains the benefits and limitations of test monkeys. Using the test monkeys revealed that it was possible to compensate for many of the limitations and gaps in the existing regression test automation. The main benefits gained were as follows: Early testing: Test monkeys do not require an established GUI or a highly stable application under test and can therefore be applied early in the applications lifecycle. * Valuable feedback: Test monkeys are used as a reliability indicator providing vital information to both developers and testers. Cost-effective automation: Test monkeys are cheap to implement and do not require considerable maintenance effort. Test monkeys can be put to use overnight and during weekends to find severe system breakdowns (nasty surprises we don’t want our customers to find). Long and complex test runs: Test monkeys can run for a long period of time covering wide areas of the application under test. Test monkeys do not need to synchronize with the application under test nor initialize the application to a known start state as in traditional test automation. The long test runs create compound situations that are very useful for detecting defects that only surface after a long execution time. Moreover, test monkeys excel in negative and stress testing, [...]

The post Benefits and Limitations of Test Monkeys first appeared on Software Testing Magazine.

Things to Remember When Writing Tests

Software Testing Magazine — Tue, 28 Apr 2015 14:18:24 +0000

Writing test should be part of the normal routine of every software developer. It is however not always the case. In his book ” Bad Tests, Good Tests”, Tomek Kaczanowski provides some interesting tips to improve your test writing activity. * Doing things the right way takes more time than doing them any old way. But this extra effort usually pays off in the long term. * Hard to write a test? Maybe the production code is of low quality? …or maybe you should consider writing tests before production code? * Writing code is teamwork. Don’t forget to discuss these (new) ways of writing tests with your colleagues. * It pays to (at least) scan the documentation of the tools you use. There’s a good chance you’ll find some hidden gems there. * Be pragmatic, and let experience be your guide. It doesn’t matter if someone – even the author of a book – advises you to employ some technique or other. If it doesn’t work for you, simply don’t do it! Reference: Bad Tests, Good Tests, Tomek Kaczanowski, http://practicalunittesting.com/btgt.php These are simple rules that you can apply in all your programming and unit testing moments. Maybe the most important tips for me is the fact that if you have difficulties to write test it might be due to the structure of your code. Writing tests easily is a very good indicator of the quality of your code. It forces you to view you code and your software design with [...]

The post Things to Remember When Writing Tests first appeared on Software Testing Magazine.

The Software Tester as a Designer

Software Testing Magazine — Tue, 03 Mar 2015 15:33:06 +0000

Software testing is an activity that has often been placed at the end of the software development life cycle, something that you did if there were some time left before the project deadline. In his book ” Scrum Shortcuts without Cutting Corners”, Ilan Goldstein explains that the software testers should be active since the beginning of the project. I believe that the core skill of a tester is actually that of design. Irrespective of who actually runs or implements a test, a seasoned professional tester will always be able to design the most effective test cases compared to anyone else on the team. Well-designed tests not only form the foundation for the eventual testing itself but can also provide vital input into the technical design that takes place during sprint planning. When a tester is involved in the design of a user story’s test cases prior to the sprint planning session, I can assure you that the meeting will be a great deal smoother and faster with fewer contentious debates. Reference: “Scrum Shortcuts without Cutting Corners: Agile Tactics, Tools, & Tips”, Ilan Goldstein, Addison-Wesley The main idea between this quote is that the software tester mindset will help the users and developers to see more easily the ambiguity in the requirements definition that become more apparent when you try to test them. This idea is similar to the “three amigos” and “power of three” concepts developed by other proponents of Agile testing that recommend to associated product owners, developers [...]

The post The Software Tester as a Designer first appeared on Software Testing Magazine.

The Limits of Code Coverage

Software Testing Magazine — Mon, 10 Nov 2014 16:13:12 +0000

Code coverage is a metric that gives the degree to which the source code of a program is tested by a particular test suite. This metric is provided by open source or commercial code coverage tools and displayed in quality dashboards like SonarQube. There are many discussions about the right level of code coverage. In his book Quality Code, Stephen Vance explains the limit of this metric. Additionally, code coverage can deceive. Coverage only shows you the code that you executed, not the code you verified. The usefulness of coverage is only as good as the tests that drive it. Even well-intentioned developers can become complacent in the face of a coverage report. Here are some anecdotes of innocent mistakes from teams I have lead in the past that let you begin to imagine the abuse that can be intentionally wrought. A developer wrote the setup and execution phases of a test, then got distracted before going home for the weekend. Losing his context, he ran his build Monday morning and committed the code after verifying that he had achieved full coverage. Later inspection of the code revealed that he had committed tests that fully exercised the code under test but contained no assertions. The test achieved code coverage and passed, but the code contained bugs. A developer wrote a web controller that acted as the switchyard between pages in the application. Not knowing the destination page for a particular condition, this developer used the empty string as a placeholder [...]

The post The Limits of Code Coverage first appeared on Software Testing Magazine.