Software testing continuous integration tutorials and videos

Certainty Isn’t Correctness: The Real Cost of Trusting AI-Written Code

Software Testing Magazine — Mon, 10 Aug 2026 14:15:53 +0000

AI-written code can sail straight through your CI pipeline — lint clean, tests green, coverage above threshold — and still ship a bug nobody thought to test for. This article breaks down why AI code passes Continuous Integration (CI) checks built for a different generation of bugs, and lays out a short list of concrete, budget-free gates that catch what your CI pipeline can’t see today. Author: Suprajyotsna Dasari A few months ago, on a project piloting an AI-built pull-request reviewer, I watched a change clear our Continuous Integration (CI) pipeline in under four minutes. Lint was clean. Unit tests were green. Coverage sat above our threshold. The dashboard even threw us the little green confetti it saves for merges it’s proud of. It looked exactly like the hundred safe merges before it. Two days later, a downstream dashboard started showing physically impossible temperature values for one specific tag — its raw-to-engineering-unit scaling had silently flipped from Celsius to Fahrenheit under one code path — a case nobody, human or AI, had thought to write a test for. Nothing about that Continuous Integration pipeline lied to us. It answered every question we had told it to ask. We just hadn’t updated the questions for the kind of mistakes AI-generated code actually makes. That gap — between what a green build proves and what we assume it proves — is the real problem with AI-assisted development right now. It’s also a fixable one, and fixing it doesn’t require new tooling budget, [...]

The post Certainty Isn’t Correctness: The Real Cost of Trusting AI-Written Code first appeared on Software Testing Magazine.

Open Source Test Reporting Tools

Software Testing Magazine — Mon, 13 Oct 2025 04:00:00 +0000

Running automated tests is a good thing for software quality assurance. Test reporting tools help you understand the results of the current test run. You could also be interested to see how your tests results evolve over time. Here are some open source tools to help you do this, as they produce meaningful tests reports and dashboard. This article lists the main open source test reporting tools available today. Use our contact form to make us aware of some open source test reporting tools that we should add to this list. Here are some features that you could find in open source test reporting tools: Managing data of one or many languages and test automation frameworks Creation of test reports in different formats: HTML, PDF, … Report distribution via e-mail Textual and graphical representation of results of a single test run Timelines graphs for presenting the evolution of results for different test runs Integration in the continuous integration / continuous delivery (CI/CD) process Generic tools This section lists tools that works with multiple programming languages and test automation frameworks. * Updates October 13 2025: Liquid Test Reports, Playwright JSON Test Results Report, TestBeats December 7 2023: added Go-allure, HtmlTestRunner, Next Gen Report (nxgreport), Allure Report Allure Report is a flexible and lightweight open source multi-language test reporting tool. It provides clear graphical reports and allows everyone involved in the development process to extract a maximum of information from the everyday software testing process. Allure Reportsupport many languages (Java, JavaScript, .NET, [...]

The post Open Source Test Reporting Tools first appeared on Software Testing Magazine.

End to End (E2E) Testing Considered Harmful

Software Testing Magazine — Tue, 19 Sep 2023 16:37:30 +0000

This video presents the impact of End-To-End Testing upon a large scale agile project within the UK government, covering the historical motivation for End-To-End Testing so common to IT and how it has hindered team adoption of Continuous Delivery. End-To-End Testing – extensive end-to-end tests of a service against second/third party systems is the norm in IT, and in many companies it is mandated prior to production. Each end-to-end test will cover lots of scenarios, so test coverage will be high! Each end-to-end test will use the service itself as the test client, so maintenance costs will be low! However, End-To-End Testing invariably results in low test coverage and astronomical maintenance costs. Long setup times, slow response times, unreliable test data, unstable systems, and long resolution times combine to form an enormous barrier to Continuous Delivery that prevents an organization from reducing time to market and innovating in the marketplace. Video producer: https://leanagile.scot/

The post End to End (E2E) Testing Considered Harmful first appeared on Software Testing Magazine.

TestOps Infrastructure from Beginning to End: Theoretical and Technical Aspects

Software Testing Magazine — Mon, 27 Feb 2023 17:55:33 +0000

The creation of a TestOps infrastructure is a demanding task. TestOps (or test operations) refers to the discipline of managing the operational aspects of testing within the software delivery lifecycle. In this article, we focus on theoretical and technical aspects, and discuss some case studies to consider. Author: Maksim Sarychau, Test Automation Solution Architect and QA Engineering Manager at Solvd Theoretical aspects In the first part of this article, we discuss test maintainability based on Object Oriented Programming (OOP) best practices, how to keep automation results transparent for management, and other theoretical aspects of TestOps infrastructure. Reaching all platforms Increasingly, customers demand a cross-platform solution. From Web, Mobile, AI, to gaming consoles, a wider array of clients means considering all platforms. In order to make your tests truly integrated, you can use an all-in-one open-source solution called Carina. If you prefer, you can use any framework you want, though using a more complex solution will save time and money. Testing Maintainability When it comes to test maintainability, you should adhere to the main practices and good projecting patterns that are based on object-oriented programming. Using page objects for functional testing is a trending practice. However, these tests are difficult to maintain; look outside of the ‘proof-of-concept’ aspect for a clearer overview. Pro-tip: use an abstraction of UI for mobile automation with iOS and Android. In this way, you can reuse source code without having to make drastic changes to it. Using page objects and UI components should be done as much as possible [...]

The post TestOps Infrastructure from Beginning to End: Theoretical and Technical Aspects first appeared on Software Testing Magazine.

Are Your CI/CD Pipelines Secure?

Software Testing Magazine — Tue, 21 Sep 2021 19:25:56 +0000

Angel Rivera of CircleCI discusses common pain points in properly securing applications, CI/CD pipelines and protecting sensitive access gates to integration targets. You will learn strategies to secure their applications, sensitive data and DevOps pipeline integration points. Most CI/CD pipelines used by DevOps teams utilize integrations with services such as APIs, databases and other critical systems to complete their workflows. These integrations usually required the use of extremely sensitive secrets such as passwords, tokens or certificates and must be securely protected at all times. Unauthorized access of these pipeline secrets open these systems to threats from bad actors and illegal access of data. Video producer: https://www.jfokus.com

The post Are Your CI/CD Pipelines Secure? first appeared on Software Testing Magazine.

Continuous Integration: Improving Software Quality and Reducing Risk

Software Testing Magazine — Mon, 09 Aug 2021 14:39:00 +0000

As a software developer, you know that one of the critical period in a project is when you try to make integrate your code in the overall application and push it towards the final user. It is sometimes a long process that you would like to accelerate so that you could obtain a quicker feedback on the quality of your code. “Continuous Integration: Improving Software Quality and Reducing Risk” is a book written by Paul Duvall, with Steve Matyas and Andrew Glover that will help you improve the way you build and deliver software. After an initial presentation of the continuous integration (CI) concepts and objectives, the content of the book goes far beyond the simple “continuous build” aspect to cover all disciplines concerned by CI: risk management, configuration management, database evolution, software testing, inspections, deployment. It is clear that CI is just not installing a suite of tools, but is mainly changing software development practices and process. Each chapter is well-structured with practical examples related to real life situations. The book reach also nicely the objective of maintaining a balance between a somewhat tools- and language-neutral position, but still giving enough practical advice so that you could quickly adapt the advice to your own software development environment. Final appendixes give valuable information on CI resources and evaluating available CI tools. Finally, you can get more and updated information on continuous integration from the website associated to the book: http://www.integratebutton.com Reference: Continuous Integration: Improving Software Quality and Reducing Risk; Andrew [...]

The post Continuous Integration: Improving Software Quality and Reducing Risk first appeared on Software Testing Magazine.

10 Steps to Continuous Integration, Testing and Delivery

Software Testing Magazine — Mon, 19 Feb 2018 14:48:47 +0000

DevOps is based on continuous delivery and anything that breaks the continuity is a bottleneck. While Agile and DevOps have become common terms in Development and Testing organizations, manual build and deployment processes are still causing problems along with integration and testing. This presentation, Tanya Kravtsov shares her experiences along with tools and methods that facilitate the bottleneck discovery process while encouraging innovative thinking among team members. You will explore ways you can use various techniques to identify, prioritize, and resolve bottlenecks. Learn ways to deal with the most common bottlenecks that cripple development progress—data generation, test environment setup, test execution, and results analysis. This talk discusses various automation tools and techniques that can help to address process bottlenecks and achieve a true Continuous Integration, Testing and Delivery state. Video producer: http://oredev.org/

The post 10 Steps to Continuous Integration, Testing and Delivery first appeared on Software Testing Magazine.

DevOps Benefits for Software Testing

Software Testing Magazine — Mon, 29 Aug 2016 13:53:19 +0000

One of the main trends in software development is to deliver software more quickly. DevOps, continuous delivery or continuous integration are some of the approaches that have been promoted recently to achieve this goal. In their article “DevOps Advantages for Testing”, Gene Gotimer and Thomas Stiehm discusses the advantages that these approaches could provide to software testing and software quality. The article starts with a presentation of the different aspects of DevOps, continuous delivery and continuous integration. It then discusses the topics of code coverage, mutation testing and static analysis, explaining how they can be integrated in a continuous integration process. The article then presents the concept of delivery pipeline, defined as the process of taking a code change from a developer and getting it delivered to the customer or deployed into production. For each topic discussed, the article proposes some open source tools that can be used for this activity.The end of the article discusses how these concepts are applied in a case study. The conclusion of Gene Gotimer and Thomas Stiehm is that “The journey towards a continuous delivery practice relies heavily on quality tests to show if the software is (or is not) a viable candidate for production. But along with the increased reliance on testing, there are many opportunities for performing additional tests and additional types of tests to help build confidence in the software. By taking advantage of the automated tests and automated deployments, the quality of the software can be evaluated and verified more [...]

The post DevOps Benefits for Software Testing first appeared on Software Testing Magazine.