In this article, Cem Kaner discusses the usage of oracles when teaching software testing. In software testing, an oracle is the expected result of the test. It is an heuristic that should help you decide if the program passed your test. He ask the question: “If you don’t have authoritative oracles (“authoritative” = an oracle that is always correct), then how can you test? How can you specify a test in a way that a junior tester or a computer can run the test and correctly tell you whether the program passed it?”
The issue of Cem Kaner about oracles is that if your rely on them you could have either “misses” when you think the program but it did something wrong. For example, a program that adds 2+2 might get 4, but it is clearly broken in some way if it takes 10 hours to get that result of 4. You can also have “false alarms” when the test is considered as a failure even if it behaves correctly. This might happen when we test the performance of some code and at the same time somebody is putting additonial load on the server. He thinks that the theoretical approach to create oracles have some issues: they helpful to evaluate test results, but not to design tests, particularly automated tests. The paper is completed with an interesting presentation of consistency heuristics used in software testing.