March 16, 2016
Learn more about a dozen mobile app-testing challenges that can sabotage your company’s mobile strategy.
Mobile apps are a daily way of life for today’s consumers. As the number of apps has soared into the millions, consumers have upped the ante on their expectations for quality and usability. Complicating matters further for app developers is the diversity in device types, operating systems, form factors, and new capabilities. Ensuring product quality has never been more demanding, complex, time consuming, and expensive.
Putting your mobile apps through the paces is paramount to meeting consumers’ expectations. In fact, I would argue that establishing an effective mobility testing strategy is as important as quality app design. Catching issues earlier on in the cycle leads to significant cost savings. For example, assuming an average of $700 to fix one defect, and an average of 20 bugs per release, this translates to cost savings of $14,000 per release. When you do the math, the importance of implementing an effective mobile app testing strategy is obvious.
But, mobile application testing raises unique challenges:
- Diversity in platforms, operating systems, and devices.
- Automation challenges: sophisticated user experiences involve touch, gestures, GPS location, audio, sensors, physical actions (NFC), may require customized testing on real devices.
- Network performance: cellular networks exhibit wide variations in bandwidth latency, and reliability.
- Application complexity and sophistication: mobile devices and apps are becoming more sophisticated, requiring more complex testing.
- New operating system versions often break applications: developers have no control over when new operating system versions will appear, and when or whether users will upgrade. This requires not only support for new OS version but ensuring backward compatibility.
- Bug-fix latency: some app stores have a submission latency of one to two weeks, making quality more important.
- New technology risks: mobile is at the leading edge of new technologies, such as “code once and deploy everywhere” platforms, that are not yet well-understood in terms of testing.
- Performance variations: performance differences between top-end devices and low-performance handsets.
- Operator intervention: operators may modify mobile web content to optimize network performance so desk-based testing may not show what real-world handset user experiences are like.
- Contextual issues: gauging whether on-screen text is readable by users of all ages, whether screen buttons and hyperlinks far enough apart, etc.
- Peripherals: add-on devices such as Bluetooth peripherals are not generally accessible to testing tools.
- Security is an important aspect often overlooked or considered very late in mobile software development life cycle (SDLC), but critical to the success of mobile applications considering the variety of devices.
It is often difficult to understand the big picture of mobile testing and chalking out a test strategy. Holistic knowledge of mobile application types, their architecture and development platforms, and market adoption data will help prepare a far better and optimal test strategy. It is only after learning about all these aspects that one truly appreciates how closely the development and testing sides are intertwined.
As applications become more complex, testing on real devices is far more important than emulators or remote devices. Small bugs and edge cases are missed without testing on the actual devices.
Organizations with a variety of apps need to support large set of devices. By identifying the necessary test cases, devices to be tested on, and any other custom requirements, automated and exploratory testing can be simultaneously executed on multiple devices in a lab like PwC’s Mobility Lab. This allows for short testing cycles with speed to market. A detailed and actionable report can be delivered back to the client outlining how the application performed and related weaknesses.