Software testing software quality assurance tutorials and videos

A Release Review Checklist for Teams Where One Developer Does Most of the Work

Software Testing Magazine — Thu, 29 Oct 2026 14:45:48 +0000

There are many situations in software development where a single software developer is in charge of building or maintaining an application. How do you do code review in this case? In this article Niral Dhameliya suggests that a senior developer perform the release review process with a checklist can be finished in under an hour. Author: Niral Dhameliya, Partner and the CTO of Empiric Infotech LLP, https://empiricinfotech.com Many software products are built by very small teams. A startup hires one developer for its first app, a company extends its team with a single remote engineer, or a product moves into maintenance with one person doing most of the work. In each case, the usual safety net of a large team is missing: there is no second developer reading every change and no dedicated tester running every build. The fix is not to hire a full QA team. It is to put a second, senior pair of eyes on every release, and to make that review fast enough that nobody is tempted to skip it. This article describes a release review that a team lead can run in under an hour, the checklist behind it, and the habits that keep it practical. Why the developer’s own testing is not enough A developer who wrote a feature tests it the way they imagined it being used. That covers the happy path and the cases they thought of while coding. It rarely covers the cases they didn’t think of, because those are, by [...]

The post A Release Review Checklist for Teams Where One Developer Does Most of the Work first appeared on Software Testing Magazine.

Why AI Models Pass Benchmarks but Fail in Production

Software Testing Magazine — Mon, 19 Oct 2026 14:52:37 +0000

There are many cases where AI models that worked fine in development start to behave badly in production. In this article, Kanika Vatsyayan discusses the causes that lead to this situation. He recommends moving from static assessments to continuous quality engineering for AI models. Author: Kanika Vatsyayan, BugRaptors, https://www.bugraptors.com/ Academic evaluation suites celebrate clean milestones: an 88% score on MMLU, top-percentile GSM8K reasoning accuracy, and flawless Python generation on HumanEval. Engineering signs off, release candidates, clear staging checks, and the model goes live. Within weeks, runtime defects the surface. Support systems hallucinate unauthorized return policies; copilots emit malformed API payloads that freeze downstream services, and retrieval engines confidently serve outdated documentation. These models did not degrade; they operate as designed. The root cause is a fundamental mismatch: static AI benchmark testing measures isolated capabilities in a sterile environment, while live applications demand continuous, full-lifecycle AI model testing across distributed, stateful software. What Causes AI Models to Break in Real Environments? Traditional enterprise software breaks on explicit syntax errors, unhandled exceptions, or database query timeouts. Machine learning applications can fail silently while executing clean code, returning valid HTTP 200 responses and passing simple string-formatting checks. When probabilistic models interface with stateful enterprise architectures, runtime failures typically emerge across four distinct operational surfaces: Fabricated Business Logic: Large language models can generate ungrounded claims with great mathematical certainty, including false warranty exceptions, discounts, or transaction records. Orchestration and Tool Errors: An agent may pick the proper business action but fail downstream execution by [...]

The post Why AI Models Pass Benchmarks but Fail in Production first appeared on Software Testing Magazine.

Why Pull Requests Should Review Tests, Not Just Code

Software Testing Magazine — Mon, 12 Oct 2026 08:44:57 +0000

When you add new code to your applications, it is associated to the test scripts that should validate it. Do you give enough importance to these tests? In this article, Eli Lopian explains that when pull requests are opened, you should review the tests scripts as much as the code. Author: Eli Lopian, Typemock, https://www.typemock.com/ Modern software development has become remarkably disciplined about reviewing production code. A pull request is opened. Developers examine the implementation, question architectural decisions, look for security issues, consider maintainability, and discuss whether the change belongs in the codebase at all. Automated checks then add another layer of protection through compilation, static analysis, security scanning, and continuous integration. But there is an important gap in this process. The tests added by the pull request often receive far less scrutiny than the production code they are supposed to validate. A developer may submit hundreds of lines of implementation code alongside dozens of new unit tests. The production code receives detailed comments. The tests are compiled and executed. If everything turns green, the pull request moves toward approval. That process assumes something that should not be assumed: A passing test is not necessarily a good test. As AI-assisted development makes it dramatically easier to generate automated tests, this distinction is becoming more important. A Passing Test Can Still Be a Bad Test Continuous integration answers an essential question: Does the test currently pass? It does not necessarily answer: Does this test provide meaningful additional confidence in the software? [...]

The post Why Pull Requests Should Review Tests, Not Just Code first appeared on Software Testing Magazine.

Now Cleared for Failure: Tunnel Vision in Software Verification

Software Testing Magazine — Mon, 28 Sep 2026 09:19:16 +0000

Software testing efforts are mainly focused on the application code itself. However, recent major failures were due to configuration changes or system data. In this article, Sergei Cherednichenko discusses how quality assurance efforts should widen their scope from an application-centric perspective to a system-wide reliability verification. Author: Sergei Cherednichenko, https://linkedin.com/in/sergei-cherednichenko/ Several recent high-profile outages, including Cloudflare’s oversized bot-management feature file and GitHub’s policy access change, have resulted from configuration changes or system data rather than application code. Yet much of the industry’s testing investment remains focused on source code rather than system-level failures. Verification techniques such as property-based testing, executable reference models, and deterministic simulation have traditionally required significant time and resources, limiting adoption outside organizations such as Amazon Web Services (AWS) and Google. As artificial intelligence (AI) reduces some of those barriers, engineering teams have an opportunity to expand testing beyond code and place greater emphasis on system verification. Complexity requires system verification over code verification Complex systems have non-linear feedback relations between system elements, creating uncertain dynamics that static risk assessments do not readily capture. The more complex the system, the more difficult risk detection becomes. GitHub is a prime example of a complex distributed system. GitHub repositories and services host most modern software development tools, with an average of over 43 million pull requests each month globally in 2025. Most issues in these systems are not related to code but to deployment and dependency issues. Figure 1 demonstrates that software bugs, labeled as internal, are secondary to [...]

The post Now Cleared for Failure: Tunnel Vision in Software Verification first appeared on Software Testing Magazine.

Your AI Doesn’t Know What Is “Correct”

Software Testing Magazine — Mon, 14 Sep 2026 16:50:01 +0000

AI-assisted code generation has changed the capabilities of organizations to produce code. In this article, Hélder Ferreira discusses the impact on software quality created by this new situation. The main problem is that this new code is often created by tools that lack the knowledge of the global context of both the application infrastructure and the process being automated. Using functional tests as a baseline might help to limit the risks. Author: Hélder Ferreira, Director of Product Management, Sembi, https://www.sembi.com/ Everyone knows that AI “speeds” up development. Recent research found that developers using AI tools increased their coding activity, measured by commits, by as much as 180%. But producing more code doesn’t necessarily mean better software: AI-generated code often introduces errors that leave developers picking up the pieces. AI-generated code forces software developers to rework and debug the code that “only took a few seconds” to create. What’s missing and keeping AI from accurately producing and shipping code is a written, testable record of how the product should behave. Much of this context lives in disconnected tools or internal knowledge, leaving AI to guess at a developer’s intent. This is where test artifacts help AI-assisted development. Test context breaks down detailed user flows written in step by step detail. Unlike requirements or other dev artifacts that show a snapshot of how features were once built, tests capture your product’s truest expected behavior and your regression suite aggregates the cumulative and latest expected behavior of your product functionality including the multitude [...]

The post Your AI Doesn’t Know What Is “Correct” first appeared on Software Testing Magazine.

What Happens to QA When AI Writes 100% of the Code?

Software Testing Magazine — Thu, 27 Aug 2026 15:39:30 +0000

Today, you can find tools that generate code and AI-assisted test automation tools that will handle the quality assurance part. Is it the death of traditional manual testing? In this article, Rimpal Mistry reminds us that human validation is still needed to provide the final check of delivered software. Author: Rimpal Mistry, Co-Founder & VP Operations @ Testscenario Over the past few weeks, we have been watching something interesting happen on some of our projects. Let’s take one specific project that has reached a stage where 100% of the code is being written by AI, with no direct human intervention in development. Features are being built by AI. Bugs are being investigated and fixed by AI. Code reviews are handled through automation. Deployments are automated too. From development to deployment, there is now very little direct human involvement. That part is impressive. But the more interesting part for us is what happened to QA. The requirement for manual testing did not go down. It actually grew. It Still Starts With a Requirement The process still starts normally. Someone needs a feature, change, or fix. Earlier, a developer would read the requirement, understand the codebase, build the feature, test it, fix issues, and send the change for review. Now much of that can go directly to an AI coding system. The AI reads the requirement, looks at the existing code, identifies what needs to change, and starts building. Humans still decide what needs to be built. But once that direction is [...]

The post What Happens to QA When AI Writes 100% of the Code? first appeared on Software Testing Magazine.

Certainty Isn’t Correctness: The Real Cost of Trusting AI-Written Code

Software Testing Magazine — Mon, 10 Aug 2026 14:15:53 +0000

AI-written code can sail straight through your CI pipeline — lint clean, tests green, coverage above threshold — and still ship a bug nobody thought to test for. This article breaks down why AI code passes Continuous Integration (CI) checks built for a different generation of bugs, and lays out a short list of concrete, budget-free gates that catch what your CI pipeline can’t see today. Author: Suprajyotsna Dasari A few months ago, on a project piloting an AI-built pull-request reviewer, I watched a change clear our Continuous Integration (CI) pipeline in under four minutes. Lint was clean. Unit tests were green. Coverage sat above our threshold. The dashboard even threw us the little green confetti it saves for merges it’s proud of. It looked exactly like the hundred safe merges before it. Two days later, a downstream dashboard started showing physically impossible temperature values for one specific tag — its raw-to-engineering-unit scaling had silently flipped from Celsius to Fahrenheit under one code path — a case nobody, human or AI, had thought to write a test for. Nothing about that Continuous Integration pipeline lied to us. It answered every question we had told it to ask. We just hadn’t updated the questions for the kind of mistakes AI-generated code actually makes. That gap — between what a green build proves and what we assume it proves — is the real problem with AI-assisted development right now. It’s also a fixable one, and fixing it doesn’t require new tooling budget, [...]

The post Certainty Isn’t Correctness: The Real Cost of Trusting AI-Written Code first appeared on Software Testing Magazine.

Daniil Khudenko: U.S. Startups Need Not Bureaucracy, but Provable Software Quality

Software Testing Magazine — Wed, 05 Aug 2026 18:39:31 +0000

In 2026, the U.S. B2B technology market clearly illustrates a challenge faced by many growing startups: even a technically strong product may not be ready to meet the requirements of large enterprise customers. At an early stage, a company often succeeds through rapid development, informal communication, and close collaboration with its first users. Decisions are made quickly, knowledge is shared verbally, and emerging problems are resolved through the direct involvement of founders and engineers. As long as the product, team, and customer base remain relatively small, this model can indeed be effective. As the business grows, however, its limitations become increasingly apparent. When a startup begins working with large corporations, leading private companies, or organizations connected to the public sector, functioning software alone is no longer enough. Customers want to understand how the vendor manages defects, releases, product changes, and operational risks. As a result, quality assurance ceases to be merely an internal engineering function and becomes evidence of the company’s ability to scale responsibly. Such requirements are particularly high in the United States, which remains one of the world’s leading centers of startup activity. According to StartupBlink, the country ranks first globally in the development of its startup ecosystem. The United States is home to more than 93,000 startups, which have collectively raised more than $220 billion in funding. By comparison, fewer than 2,000 startups are registered in Russia, according to the same source. The difference is reflected not only in the number of companies or the volume of [...]

The post Daniil Khudenko: U.S. Startups Need Not Bureaucracy, but Provable Software Quality first appeared on Software Testing Magazine.