In software development projects, the test cases are written based on the requirements. In this article, Sreerekha Bakaraju explains how software testers can use a test matrix approach to ensure the traceability from requirements to testing.
Author: Sreerekha Bakaraju
Requirement Analysis
Understanding the requirements is crucial in software testing. The requirement phase comes before the starting of the test phase in the software development life cycle. Product’s requirements are detailed in test plans and test cases. Most often, the requirements and walkthroughs are done by the Business team. Test cases are written based on the software requirements. If the requirements are not clear, the test cases become incomplete. It would be beneficial to spend sufficient time and effort to clearly understand the requirements.
Test Plans
A well-planned test plan is crucial for software testing. Every activity of the testing phase is listed in the test plan. First and foremost, understanding the requirements is very important before writing the test plan. Test plan initiation starts from the requirements. Some of the items in the test plan are: list of the resources required for testing, timelines, procedures that are going to be used, the components that are targeted for testing in specific releases, and the required hardware and software. Writing a test plan needs a lot of experience in the industry and it also requires foresight into the product release plans and the product roadmap.
To begin with, the test plan has to account for what is required to start the testing phase. Testing involves testing the system or application under a variety of configurations and circumstances. The test plan mentions various important items such as the release dates, the preparation of software builds, criteria for acceptance of the builds and testing, the sequence of things that will happen during the test phase, the entrance and exit criteria for various testing phases, and defect management. It also includes the definitions for priority, severity levels of the defects, and the methods of defect resolutions and methods of communications, etc. are documented.
Test Matrices
This section describes some general methods that are used in writing test matrices. Test matrices have to be written in conjunction with requirements. Test matrices give the overview and the summary of test cases. Test matrices ensure that it covers all the requirements. Based upon the test matrices, the test cases are written in detail.
A significant elimination of time and stress of the software tester is achieved through identifying and reusing the test cases, test libraries and common functions among multiple tests. The test scenarios that are in test matrices have one-to-one relationship with the requirements, i.e. for every scenario listed in test matrices, there is reference to the requirements. Test matrices include test scenarios. Every test scenario from the test matrices is taken and then expanded as a test case. Each test case tests the functionality of the application with required test data.
Below is a sample test matrix in generic format. This is also known as traceability matrix. This can be updated or modified depending on the product or application that you are working with.
Sample Traceability Matrix
| Serial Number | Test module | Description | Business Requirement | Functional or non-functional | Comment |
Methodologies
- Write test cases that are easy to maintain
- Identify reusable Functions, Procedures, Subroutines and framework functions, test cases and test data
- Identify the components and classify graphical user interface objects , database, functional, user acceptance, browser based, non-functional test cases and all various test scenarios
- Do not write machine specific tests, write generalized tests
- Make sure all the requirements are covered
- Document your work
- Use comments in the tests wherever required. For example, what its used for, what are inputs and expected results for the function or procedures wherever possible
- Start as early as possible writing test cases. Do not wait till the last minute. Since it is a huge task and adopts the waterfall approach
- Find the way to capture most of the basic defects in sanity tests
- Consider the product road maps in terms of the supported platforms, browsers versions etc
- Constantly maintain, and do enhancements to the test suites, test cases based on the requirements
- Be sure to check the latest software versions (in terms of tools)
Benefits, Enhancements, Maintenance
Test Matrices give the overview of the tests that would be covered and executed during the test phase. This gives an idea in terms of required resources, time, utilities, and the tools necessary for the test. This gives an overview of what is going to be tested and if any additional test cases are required. Walk-through of test matrices is conducted to the product, project and development teams. This gives insight to the developer about the tests that are going to be conducted during the test phase.
Test matrices and test cases have to be improved, updated, and maintained depending on the enhancements of the application or product and requirement specifications. Testers improve the quality of the test cases, which in turn improve the quality of the product or by adding tests as per the changing or enhancing the requirements of the Product. The ultimate goal is to uncover the defects from the application and to satisfy the customer’s needs. The test cases classification such as functional, non-functional, performance, industry standards, regression, sanity, smoke etc. can be mentioned in test matrices.
Below are some of the things that are to be considered for the maintenance of the test cases, test matrices and test framework.
- Need to add tests for change in requirements or enhancements
- Plan for regression test suites that support multiple releases
- Use generalized global functions, procedures and methods that can be used throughout the product life cycle
- Have test suites that can run independently and in the background
Regression tests, performance benchmarking across operating systems are required to be revisited based on the need and efficiency.
Conclusion
I have found these guidelines seemed to accomplish the goal of effectively writing and using test matrices in software testing. This article gives a general idea about the items that are to be considered when writing test matrices. Test matrices have to be developed and updated, depending on the enhancements of the application requirements.
References
[1] William E.Perry, 2nd ed., Effective Methods for Software Testing, 1999
[2] Sreerekha Bakaraju, 1st ed., Software Testing made Easy, 2020
About the Author
Sreerekha Bakaraju has over fifteen years of experience in the Information Technology industry. In addition to developing Application Software in Oracle, SQL, and PL/SQL in Windows and UNIX environments, and system development methodologies such as Agile, ITIL, she has also specialized in Software Testing, Quality Assurance fields. She has experience in the Financial, Banking, Telecommunications and Technological sectors. She is the author of the book “Software Testing Made Easy”.