How do you perform software testing in a continuous deployment environment where you have more than 10 releases a day? How can you be confident that the code running in production is still reliable? In this blog post, Andy Hume explains the software quality process the Guardian uses for continuous deployment mode.
As they don’t have the time for full regression tests, the Guardian team only run a full set of unit tests on the Scala and JavaScript codebase, as well as check the output of some key application endpoints in a headless browser. If these tests passes, the code is automatically deployed to a continuous integration environment. Developers can sanity check their changes in this environment and them can immediately deploy to production. And it is where the software quality assurance starts…
The Guardian team monitor in real-time two important metrics: pageviews and page load times. They have a large number of automated browser tests using Selenium WebDriver that are run in production, immediately after a code deployment. They also monitor the rate and details of run-time JavaScript errors occurring on the web site.