Anti-Patterns in Software Testing

Since the start of software development, we have always had to test our software. And over the course of several decades, the discipline of software testing has seen many best practices and patterns developed. Unfortunately however, not all practices have been good and there are also anti-patterns in software testing.

Author: Jane Temov, Enov8, https://www.enov8.com/

In this article, we discuss some of the key Anti-Pattern found in “Software Testing”. An anti-pattern is a pattern of activities that tries to solve a certain problem but is actually “counter-productive”. It either doesn’t solve the problem, makes it worse, or creates new problems. In this article, I will sum up some common testing anti-patterns.

Anti-Patterns in Software Testing

Anti-Pattern #1 Involving Testing too Late 

Somewhat a legacy from the Pre-Agile “Waterfall” days.

The concept of having testing waiting in-line for the requirement analysts and developers to throw something over the wall. A cycle that invariably results in a raft of late bugs, “unnecessary” developer re-work, team quarrels, time-wasting and low morale.

Unfortunately, this approach to testing is all too common.

Good Pattern:

In the spirit of Agile, it is essential that all teams work together throughout the lifecycle and as early as possible. Continuous collaboration will promote “shared understanding”, healthy discussions and early “quality based” feedback.

Remember, catching a “defect” before it is built (or even authored) is much cheaper than catching a defect late in the development cycle.

Anti-Pattern #2 Manual Testing for Regression

Large applications will invariably have hundreds, more like thousands, of “testable” usage permutations. Unless you have unlimited staff, common-sense tells us that you can’t achieve efficient software delivery if you are re-testing (i.e. regression testing) all of this manually. It would simply take too long.

I guess one option is to not re-test.

“Hey, we tested it once right?”

Well yes, but not really! Products rarely stands still, and continual change will mean that an old functionality can behaves differently. It is only a matter of time before “neglect” will come back and bite you very badly (most likely in production).

Good Pattern:

Codify! A better alternative to manual testing is to automate as many repeating tests as you can*.

Note*: Not all tests are automatable.

Promote the importance of testing across your teams. Have everyone contribute to Test Automation methods across the relevant development and testing phases. From Unit Testing, System Testing, Integration Testing and End-to-End. There are plenty of tools out there to automate those “tedious” manual methods.

Anti-Pattern #3 Automation Over-Load

Yes, as a counterargument to my previous point, be wary of trying to automate too much.

Manual testing still has its place in the world of software testing and some things just don’t deserve to be automated. For example, tests that are too trivial (impact wise), test that are too hard to automate (due to complexity) or tests that are too expensive to automate (cost prohibitive).

Good Pattern:

Weigh up the benefits of automating a test before aimlessly doing do. Consider the benefit of automation versus cost, effort, risk and the likelihood of redundancy. And Use manual testing to fill the gaps.

Tip: Another thing you can’t expect to automate (or certainly not automate in its entirety) is exploratory testing. Exploratory Testing is a type of testing that benefits from tester experience, creativity and learning through “examination”.

Anti-Pattern #4 Test Environment Neglected

Falling somewhere between the realms of Development, Test and Operations, Test Environment Management (TEM) is often de-prioritized or worse still neglected. Key challenges through lack of accountability and/or capability include:

  • Environments not being “Fit for Purpose”
  • Environment unreadiness
  • Environment contention
  • Environment instability or outages
  • Low quality environment Test Data

After poor requirements, Test Environment Management anti-patterns are the top cause for Test Cycle delays. Don’t make TEM the reason for an inefficient Software Delivery Process.

Good Pattern:

Implement a Test Environment Management capability using frameworks like the EMMi (The Environment Management Maturity Index) for guidance. Promote concepts like Non-Production CMDB, Booking Management, Environment Calendars, Service Management, Orchestration and Metrics to drive continuous improvement.

Anti-Pattern #5 Using Uncleansed Data

We all want “Production Like” Data.

Good End-to-End data provides us with confidence that the testing we are doing in our test environments effectively simulates what happens in the real world. However, be careful, this does not entitle organizations to use “customer” data.

Note: The Privacy Regulation like GDPR & HIPAA are clear on this.

The risk of using customer data follows:

  • Loss of data can have significant impact on customers & employees.
  • Mismanagement of data can lead to “compliance” penalties i.e. fines & Sanctions.
  • Companies are exposed to significant financial risk due to brand damage & lawsuits.
  • Regulations are continually changing & evolving.
  • Information breaches are becoming more and more common.

Good Practice:

Understand what your data looks like (data literacy) and ensure that Customer Data or PII (Personally Identifiable Information) is obscure, for example masked or encrypted. It may seem like an “overhead” initially but cleansing your data will both protect your customer and provide you with a rich data set to improve your future quality engineering efforts.

Anti-Pattern #6 Not Training your Developers

Testing belongs to everyone, as ultimately, we are all working to the “same goal”. Despite that many organizations continue to promote a “Them & Us” (or Developers & Testers) mentality.
Don’t make the same mistake as identified in the anti-pattern #1, “Throwing it over the wall”, as it simply delays the finding and fixing of defects.

Good Practice:

Promote software testing inside development. Have your “software testers pair with developers” and teach them the relevant techniques so that they can test the features as they build them. When software testers and programmers are working together, you will build quality awareness across the group, promote collaboration across team members and ensure application testing shifts left.

Conclusion

Software Testing is no longer about a single “isolated” team banging on the keyboard. Ultimately testing is the responsibility of everyone on the project and that includes the analysts, developers, the environment guys, the data guys and everyone else. A team working together to find ways of improving quality & streamlining delivery through implementation of the correct software development methods and tools.

About the Author

Jane Temov is an experienced IT Environments Management & Resilience Evangelist. Areas of specialism include IT & Test Environment Management, Disaster Recovery, Release Management, Service Resilience, Configuration Management, DevOps &Infra/Cloud Migration.

Editorial Contribution: Article Ideas based on an earlier Enov8 post by Peter Morlion.

This article is written by Enov8 who specialize in Test Environments & Test Data Management. If Environments or Data are a problem for you, then please have a look at its tools:

Contact SoftwareTestingMagazine.com if you want to publish quality software testing content on this website.