Mobile devices are now one of the major market for software. This article discusses the four main challenges of software testing concerning mobile applications: the variety of devices and operating systems; the performance of networks, the user interfaces; the limited amount of software testing tools targeting this area.
Author: Mush Honda, KMS Technology, http://www.kms-technology.com/
Recent software advancements have truly unlocked the potential of mobile devices. According to ABI Research (), the mobile app market expanded to $27 billion in 2013—a figure projected to grow even further in years to come. Such a meteoric rise inevitably creates problems for developers under pressure to produce apps quickly for diverse devices.
Many QA departments are also under the gun—playing catch-up as they try to test apps thoroughly before the target go-to-market dates that marketing people too often schedule too early. Four key challenges in particular differentiate the mobile-app testing process. Developers and QA teams must learn how to overcome them to ensure success:
1. Fragmentation: Mobile apps run on a wide variety of devices, different OS versions, and manufacturer user-interfaces.
2. Networks: Network performance, which varies from carrier-to-carrier, impacts user experiences significantly.
3. Usability: Clever user-interface design is essential for creating apps people can use intuitively.
4. Tools: Mobile app testing tools are limited compared to traditional app testing (but you can find them if you know where to look).
Develop a Pragmatic Testing Strategy
To overcome these challenges, first create a testing strategy that includes an optimal mix of devices for maximum coverage—real devices, emulators, and remote-access devices delivered via the cloud. Also consider real-world conditions such as these two examples:
- How will different locations and fluctuations in connectivity impact the app?
- How will the app handle interruptions such as incoming calls, text messages, and social media notifications?
The test strategy should cover four main areas—functional, performance, security and usability—and automating the process plays a key factor in making the project more manageable. Knowing where to focus your resources is also a prerequisite for getting the best out of your staff.
You can’t cover an exhaustive combination of all devices and OS versions, so consider the product and its intended use and then target the most important devices. If you are developing a mass-market app, for example, analyze which device has the biggest market share and focus on the OS version with the greatest penetration.
Also investigate what is popular within the target demographic, which helps extract more value from the testing. Another key consideration is different screen sizes. For this, it helps to produce a requirement checklist for interactions that occur through screen interactions as well as hardware buttons.
Stay Flexible to React to Emerging Requirements and Changing User Conditions
It’s impractical to create a complete testing plan and then expect to repeat it on actual hardware until the app ships. As additional requirements emerge, you need to plan in stages. Start by using emulators to identify bugs and functional problems. You can then move to real devices and remote-access devices when testing in real-world conditions and when considering the impact of security and the network.
To validate the quality of the product, primarily rely on functional and usability testing—but remember that new features may also be thrown into the mix during development. So validate everything before the live product release while also considering compatibility and the impact of software upgrades.
When emulating end users, it’s best to go beyond the optimal conditions and test the performance impact of other conditions end users often run into:
- Low signal strength, 3G and 4G networks, and different Wi-Fi speeds
- Network drops when switching to or from Wi-Fi
- Offline app usage
- Constraints created by specific chipsets
- Unexpected impacts on memory
- Low battery power
- Other degraded phone functionality
Also think about low-level resource usage, and watch for buffering problems or garbage data generation. Security considerations include data encryption and multi-user support handling.
Of course we don’t have the space here to cover all testing scenarios, but hopefully this will give you some things to consider in setting up a mobile testing plan for your company.
About the Author
Mush Honda is QA Director at KMS Technology, a provider of IT services across the software development lifecycle with offices in Atlanta and Vietnam. He was previously a tester at Ernst & Young, Nexidia, Colibrium Partners and Connecture. KMS services include application management, testing, support, professional services and staff augmentation.