The REAL Challenge of Testing Mobile Apps

The amazing growth of mobile technology over the last few years is, to put it simply, mind-blowing. Of course, there are literally hundreds of devices to choose from today, but it’s the software that really does the blowing of the mind for consumers, largely thanks to the huge rise in mobile applications.

But for developers and QA departments it’s a different kettle of fish, as they struggle to create and test new apps before they’re rushed to market. The testing of mobile apps is a fairly new challenge that is constantly developing and changing, in tandem with the ever-changing mobile world it supports.

Our own testing and automation experts have come across a number of challenges when working on mobile app projects, including the following four main challenges:

• Mobile devices are much more restricted when compared to desktops: the underlying OS typically sandboxes each application and allows very limited inter process access, unless a phone is ‘rooted’ (a typical, if not fully legal, requirement for phone development and testing, which gives full administrative control over the phone, including third party apps).
• Controlling the UI navigation of a mobile device is harder: mobile UI response times are harder to predict than their desktop equivalents, and hence makes screenshot-based verifications of actions and responses that much more difficult (although it is possible to use logcat or custom services) – in comparison to desktop applications, which have many ways of monitoring and verifying UI responses.
• Mobile devices, by definition, are not statically located entities: depending on where the device is, the user’s carrier network may introduce many challenges that completely override a prior tested use-case.
• The proliferation of different screen sizes and form factors make UI-based testing an ever complex task.

The challenges above should certainly be taken into consideration when planning your testing. Of course, a successful testing strategy really should have a wide range of devices to test for maximum coverage, with some real devices and emulators/simulators too.

But if you thought that emulators and simulators were the ultimate answer to your testing woes…think again. Emulators and simulators really are an excellent option to add to your testing strategy, but bear in mind that even though they may be cheaper, more readily available, and are good at catching layout issues, they are also generally slower than real devices, do not fully compare with real-life user scenarios, and also do not have the same range of performance data available.

From our own experience, the challenges we’ve mentioned can be overcome. However, we’d also recommend a pragmatic approach, and the acceptance that the testing of ALL possible scenarios is extremely unlikely. As a minimum, you should consider the app itself and its intended use and market, and target the most important devices.

Feel free to contact us if you need further advice or assistance regarding mobile testing and overcoming any mobile testing challenges.