Your car, your fridge, your thermostat might share one thing in common: an Internet connection. This new phenomenon is called the Internet of Things or IoT. All these connected hardware are also driven by software that should be subjected to software testing. In this article, Pavel Novik discusses the main challenges of testing IoT devices.
Author: Pavel Novik, A1QA, http://www.a1qa.com/
Today, we face the rise of interrelated computing devices operating in any kind of environment – commonly known as Internet of Things (IoT). Promising to make our everyday lives easier, there are many pitfalls in this relatively new connected life. In a study published in the recently published CapGemini World Quality Report, 85 percent of the participants admitted that IoT products are part of their business. Consequently, there is a whole quickly growing world of connectivity and functionality that must be validated and verified.
However, it is worrisome that 68 percent of the organizations in which IoT products or solutions are embedded do not have a software testing strategy for this specific aspect of their IT infrastructure. It is likely that many of them rely solely on the manufacturers to assure the high quality of their devices. There is however hope that the situation will change soon, as 30 percent of the respondents to this survey said they were planning to implement an IoT test strategy in the near future. This will lead to both an emergence of new tools and their design to handle the exponential increase in test requirements and data.
Essentially, IoT technologies are expanding the machine-to-machine and machine-to-human communication and data exchange. As users increasingly rely on connected, self-steering and self-learning devices, the key quality aspects that should be considered are security, performance, functionality and usability.
Choosing devices for testing the IoT
There are today thousands of various inter-connected products on the market, all of which are collecting and sending information. Some companies develop and sell their own devices. For them, it is not a problem to have them all at hand. But there are also the companies that develop applications for different IoT devices. There are high probabilities that they can’t afford to buy each one for testing.
Companies that belong to the second category must choose the most suitable set of devices to provide maximum test coverage. Once the device is out, it may be beneficial to get usage information (e.g., Google Analytics) to find out what devices are most popular among the target audience. With the actual information from meaningful customers, the company can make an educated choice about which devices they should test and compose a list of supported devices.
In addition to device selection, there is also another point that QA and software testing companies should consider. They should develop an extensive test strategy to deal with the customers’ growing needs and stay competitive in the SQA market.
What are the peculiarities of testing when it comes to the IoT environment?
In the new context of immersion, QA engineers have to test not only the functionality of devices, but also their interconnectivity. It is necessary to be sure that if one device fails, the others can work normally. Moreover, connectivity between the devices and communication infrastructure should work smoothly.
According to experience and current trends, we suggest the following IoT main testing areas:
1. Security
Security testing should cover privacy, autonomy, control and spying. Appropriate security and penetration testing is essential, because weak security measures can lead to sensitive personal information losses. When it comes to IoT devices, cybercriminals can not only steal private information, but also attack a home security system or a car mechanical system to cause an accident.
2. Performance
Performance testing covers both the network communication and the internal computation capabilities. During performance testing, QA engineers should verify whether all data is transmitted and stored correctly, even when an unexpected disruption in service takes place.
3. Functionality
Functionality testing is aimed at Web/UI, embedded and back-end computing. The goal is to verify that the application functions meet all the functional requirements. During functional testing, QA engineers can perform negative and positive test cases. In positive testing, an application is checked against valid input data. Negative testing is used to prove that an application doesn’t work when invalid input data is provided. Besides functional testing, you need to perform exploratory testing to test devices from the user point of view. QA engineers should strive to go beyond general test scenarios, and consider body movements, voice commands and sensor utilization when writing test cases.
4. Compatibility
As IoT devices usually require a variety of configurations, compatibility testing is highly recommended. It is used to check that all features function correctly in different configurations, combinations of device versions, protocol versions, mobile devices and mobile OS versions. There is also a broad range of technical level quality attributes: compatibility, installation and resource utilization. These should be checked to provide the customer with objective test results.
Conclusion
These software testing methods and techniques are quickly developing alongside the IoT. To deliver a high-quality app, it is vital to ensure testing is conducted in the target environment with a thorough check of all relevant aspects. If you take care of each key areas of IoT testing, the connected experience will be a better one for all.
References
CapGemini World Quality Report 2016-17:
https://www.capgemini.com/thought-leadership/world-quality-report-2016-17
About the Author
Pavel Novik manages the mobile testing department and the Center of Excellence at A1QA. With more than six years of experience in the SQA industry, Pavel focuses on technologies and approaches that enable seamless and efficient testing of mobile applications, wearable technologies and IoT objects.