Test cases and acceptance criteria are two important aspects of test management. In this article, Ovidiu Donciu discusses the similarities and differences between test cases and acceptance criteria and suggests that we could merge them.
Author: Ovidiu Donciu, CEO of QA Karma, https://www.qakarma.com/
Test cases
A test case is a set of conditions, steps and actions performed on a system to determine if it satisfies all requirements, acceptance criteria and design under the given conditions and based on user expectations — Ovidiu Donciu
A test case is usually a single step, or occasionally a sequence of steps, to test the correct behavior/functionality, features of an application. — Wikipedia
A test case is a singular set of actions or instructions for a tester to perform that validates a specific aspect of a product or application functionality. — Applause
Acceptance criteria
Acceptance criteria (AC) are the conditions that a software product must meet to be accepted by a user, a customer, or other systems. They are unique for each user story and define the feature behaviour from the end-user’s perspective. — Altexsoft
Acceptance Criteria are a set of statements, each with a clear pass/fail result, that specify both functional and non-functional requirements — LeadingAgile
Acceptance criteria are the predefined requirements that must be met, taking all possible scenarios into account, to consider a user story to be finished. — KnowledgeHut
Based on all the definitions above, there’s not a huge difference(if any) between test cases and acceptance criteria. They both seem to serve a common purpose, that of verifying that a team is building the right feature and that the feature is built the right way for the customer to be satisfied and to be revenue generating for the company.
Two possible (debatable) differences could be:
- The level of detail — test cases tend to cover scenarios and AC in a deeper way with more combinations and edge cases.
- Conditions, preconditions and data — AC tend to be more high level and general while test cases are move test data heavy and includes pre-conditions and post-conditions.
Characteristics of good test cases and acceptance criteria
Based on their characteristics, test cases and AC are also very similar in almost every aspect.
The stage at which test cases are written is also under debate. Many QA teams write them either after the feature is complete to have a better understanding on how things work and look (reactive approach), while some QA teams try their best at writing them prior to implementation (proactive approach).
Personally, I encourage every agile/engineering team to go with the proactive approach and write test cases before the implementation happens. If there are scope/functional/UI/UX changes during the implementation phase, the test cases can be updated accordingly. The main advantage of using this approach is that before any line of code is written, everybody (Business, PM, Dev, QA) is on the same page regarding the feature/change to be implemented. Also, having test cases prior to development helps developers know which cases to cover in their implementation, what automated tests to write and also what to cover in their functional testing giving back time to QA to focus more on exploratory and non-functional testing.
That being said, there is already a strong relationship between test cases and acceptance criteria. Writing good test cases strongly depends on understanding the system and the feature. To have that solid understanding, requirements and specially acceptance criteria need to be present and clear.
Test case and AC management tools?
In the case of acceptance criteria, they are usually documented using the project management tool the company uses without investing in any other external tool. Tools like: Jira, ClickUp, Trello, Monday and many other tools.
On the other side, though, almost everybody’s using external tools for keeping and managing test cases. While some of those tools do offer potentially useful features like test case execution, results tracking, reports, templates, etc., my experience with pretty much all of those hasn’t been great. The main pain points I encountered are:
- Buy-in from engineering and product teams — adding a layer of complexity to already complex processes and flows
- Very hard to keep them up-to-date — specially in big companies and complex projects the work of updating the test cases in such tools is huge and usually teams end up missing on that and after a while the whole test suite and the tool itself becomes obsolete and not useful
- Traceability — They boast on traceability, but to really have that in a seamless way needs so much setup and integrations with other things
- Cost —for small startups the costs might not be high, but for medium-large companies they really get expensive. Specially considering the fact that such a tool SHOULD NOT be used only by QA teams. There’s no value in such a tool if it is used in isolation, only by QA.
Personally, what I have seen working very well is a customized project management tools and workflow to incorporate test cases. Using this approach is also based on my recommendation to invest heavily in automated tests and use those tests code as documentation.
Based on this automation-based approach, where the test cases/AC are automated and the documentations lives inside the code, I come to propose a merging of the two: test cases & acceptance criteria.
Less process work, less friction, less work duplication, better and stronger collaboration between parties: business, product, QA, engineering.
What if … just what if … we merge them together?
Why should we still have and use both Acceptance Criteria and Test Cases, spending tons of time to maintain both, paying for tools to manage test cases that become outdated super quick and very expensive to maintain. Not to mention the layer of complexity a test case management adds to existing processes without offering much value in return. TCM process and tools are NOT to be used only by QA. Devs, Product and even business should use them a lot, too!
What if we would start to use AC as TC? ACs should be achievable, measurable, testable. So, if we already have testable ACs, why should we duplicate work in creating TCs for the same scenarios?
Edge cases!? If there are edge cases for a story that if not working correctly would block the story being released, then why are not those part of the acceptance criteria? And if there are “edge cases” that won’t block the release of the new story, why would we care about them once the implementation hits production?
Conclusions
A few ideas:
- Merge the two together
- Customize your project management tool and workflow to accommodate that
- QA works with Business and Product in defining super solid AC,
- AC refining and updating by Product, QA and Engineering based on scope, technical issues, etc.
- QA involved in design and prototype review
- QA and Product sets AC’s priority: P0, P1, P2 based on the story’s critical and happy path
- QA and Dev adds Automation_Level for every AC
- Create story subtasks OR use checklists for covering every AC with automated tests
- Include AC automation in the story’s estimation
- (at least) Every P0 and P1 AC should be covered by automated tests at different levels based on the test pyramid
- Use test code as documentation for future reference
About the Author
Ovidiu Donciu is a passionate software quality expert with 15 years of experience in the field. Recently, he also founded QA Karma, offering software quality services to tech companies around the world. Outside work he is passionate about the outdoors, traveling, soccer and reading.
This article was originally published on https://medium.com/@qakarma/test-case-and-acceptance-criteria-joining-forces-894cd5455002 and is reproduced with permission from Ovidiu Donciu.