Mobile testing can make or break small screen experience

the proof that considerable advances have been made in mobile testing is evident when one compares the user experience a decade ago to present time. From frustratingly slow and difficult to navigate, mobile interfaces are now coming into their own, providing millions of seamless and happy encounters.

Does this mean mobile testing has finally found its groove?

Well, yes and no. Yes, because the small screen platform is finally being understood as a discrete medium that requires specific testing skills and expertise. Mobile testing is now an area of specialisation. And no, because the diversity of content, continuous updates and the plethora of devices on the market at any given time – with their associated operating and application software – create a constant challenge to mobile testers, and their clients, to provide a fast, error-free and fluid user experience.

With what do we test it, dear Liza?

“One of the biggest tribulations for companies wishing to test their software on mobile devices is access. The number of makes and models available can make it a cumbersome, expensive and ongoing process,” says Jacques Fouché, CEO of software testing firm Inspired Testing.

Cloud-based public device farms provide a range of mobile devices and offer a useful rental service, but they have some drawbacks. Firstly, it can get very expensive, especially for larger corporates that have a wide range of services they need to cover. Larger organisations often require 24/7 access to the devices for a prolonged period of time. “Public device farms are more suitable for smaller companies that have a couple of apps they want to test, but this also depends on their risk profile,” explains Fouché.

Another option is cloud-based private device farms, where you strategically procure your own devices and either house them in your own data centre within the business, or with an outsourced provider. Either way, you have 100% access to the devices at any time, without having to pay a rental fee. ‘Strategically procure’ means you buy certain models within a brand, as testing on one device will give you similar screen real estate on four or five other devices. “This option is good for high-volume device users requiring stringent levels of security, such as companies in the financial technology space,” says Fouché. Companies that strive to achieve a high frequency of continuous integration and deployment would also do well to consider a private device farm, as this will give them unrestricted access to the devices.

Houston, we have lift-off… or not

Once the challenge of how to access devices for testing purposes has been solved, the process of testing begins. This is where companies can really come unstuck, says Lloyd Dignan, Director: New Business, Inspired Testing. “Whether it’s a new release or an update, a complete retest is the only solution. There is a perception that once you have released your app or launched your Web site, any subsequent updates will not be as critical. Many big players have discovered the hard way that this is not the case. No company is immune to crashes and errors. To avoid this, testing should be viewed not as a ‘quick fix’, but an integral part of the DevOps process,” he explains.

Understanding which devices your market uses is also critical to the testing process. Tapping into the information streams available, such as market research, Google Analytics and your marketing engine will provide insights into the make and model of devices accessing your site or app, from which you can extrapolate technical data. Also bear in mind that device usage differs significantly from country to country. For example, in the UK, the majority of 18-35-year-olds have the latest iPhone. In Romania, the Samsung Galaxy J5 is the most popular phone, while Chinese brands Tecno and Infinix are big brands in Nigeria. “Keeping current with devices and operating system updates is one of the main challenges facing companies,” says Dignan.

One thing mobile users across markets, countries and continents do have in common is intolerance to poor app and Web site performance. “There is absolutely no loyalty,” says Dignan. “If an app crashes or takes too long to load (between four and eight seconds on average), the user will move on – and even delete the app before they go.” Because there is such fierce competition, there is significant pressure on companies to go live as soon as possible. “The opportunity for mobile is massive, but organisations have to make sure their systems and applications are fully tested – and the errors fixed – before they release. Errors become compounded with each new update and it gets harder to fix each time,” explains Dignan.

A changing landscape

There are clear statistics that show a massive transition from user access via desktop and laptop to mobile devices (including tablet), while the consumption of data has moved significantly from laptop to mobile. Mobile devices are not only used to access apps; they are becoming the platform of choice to access Web sites. This makes Web site testing as important as app testing, as an estimated 63% of users worldwide now use their mobile devices to access Web sites. There is much more likelihood of someone completing their cart on a phone than on a laptop or desktop. In addition, says Dignan, the emergence of responsive Web sites – Web sites that shrink down and respond to the size of the screen – made mobile browsing much easier, while many companies are now releasing apps to augment their Web sites, to provide an interactive user service. Progressive Web applications that look and load like ‘normal’ Web pages, but offer the functionality of an app, are also becoming standard practice.

These shifts are creating massive opportunities in the mobile space, and consequently, a significant explosion in mobile testing, says Fouché, particularly in industries such as fintech, logistics and retail. Automated mobile testing has become an area of specialisation in itself, and mobile testing is no longer a sub-set of testing, but a category in its own right.