Deep Oracles: Multiplying the Value of Automated Tests

One of the most widely touted drawbacks of the automated tests is that they work in strictly bounded context. They can only detect problems for which they are specifically programmed. The standard automated test has a bunch of assertions in the last step. By definition, an automated test cannot detect an ‘unknown’ problem. Because of their narrow focus, the automated tests are occasionally compared to dumb robots. It takes a lot of time and effort to write and support them however their return on investment is still marginal.

This mantra has been heard so many times that people just starting in the testing field can easily accept it as a truism. Using the six techniques (flaky behavior, random test data, attack proxy, logs insights, data quality and application metrics), any automated tests can be transformed into a sensitive and highly advanced suite. New, unseen or unanticipated problems are now immediately highlighted. And the best part is that you don’t need to modify the existing tests.

Video producer: