For the last several years, companies of all shapes & sizes have been building and maturing mobile development teams. Whether it’s native app or responsive web development, ramping up teams for mobile programming has become a priority for every modern organization. Until now, the efforts have been focused on development while testing has been relegated to an afterthought. However, the testing approaches must also change in order to support these new mobile implementations! A focus on proper testing tools and best practices is essential to maintaining quality & customer experience. In other words, the testing approach must scale along with the development efforts in order to deliver high quality applications and websites to a growing mobile audience.
For most organizations, manual testing is the primary option. This means testers have a library of devices to use in-hand for testing mobile websites and apps. As the device landscape has grown, the time and effort required to maintain a device library has grown as well, making this an unscalable approach. Additionally, these device libraries are not available (or must be replicated) to testers working remotely (e.g. offshore, work from home, etc.). In other words, manual testing has significant limitations particularly if teams are distributed across regions or countries.
Quality Assurance (QA) testers aren’t the only ones impacted by the limits of the manual testing approach. Designers, web developers, and native app developers all feel the pains of validating their designs and code on the variety of supported devices. Production support engineers have unique challenges because they must support not only the devices & platforms but also potentially more than one version of your applications. And of course, the business will inevitably want to see and deliver demos of the applications on one or more devices before the official release of a site or app. For each of these scenarios, a physical device library has been the only option, and it is clearly an unrealistic solution.
Enter Mobile Labs. Mobile Labs deviceConnect is “an on-premise, private cloud mobile device testing platform.” In other words, it’s a cabinet that houses a set of physical devices (up to 48!), and provides network users with remote access into physical devices. These are not simulators or emulators, but the real devices! And because they are accessible (via remote control) only on the internal network, the process of remoting into the devices benefits from the network security already established in your organization.
In addition to providing individuals with access to devices, this tool includes support for automated testing and deployments as well as remote debugging on devices. The automated testing integration hooks are provided through Mobile Labs Trust and support tools like HP’s QuickTest Professional and Unified Functional Testing (UFT). You can use Selenium scripts to automate web testing for responsive and mobile-targeted web applications. As of the deviceConnect 6.0 release, Appium and Calabash are now also supported for native application testing. There are also scripts for automating deployments to deviceConnect, which integrate well in most continuous integration environments. On the developer front, deviceBridge is a new product that allows developers to connect their machines to devices in deviceConnect, allowing them to run code on those devices. Currently, only iOS is supported, but Android is coming soon. This feature is a great addition in supporting the QA testing cycle and alleviating the “it works on my machine” fallacy.
In the Wild
By now, you’re probably thinking, “this tool sounds pretty good on digital paper, but what’s the real story?” I’m glad you asked! We helped a Fortune 100 financial services client setup their 48-slot cart and implement processes, procedures, and device management around its usage. Their primary goals were to provide onshore & offshore resources with physical devices to test native & web apps while reducing the number of device libraries required across the organization.
While setting up the cart was fairly straightforward, developing the processes, governance, and training took more effort. You can buy a tool, but until you build procedures around it, you won’t realize its true value. We started with creating processes around supporting & maintaining the cart. In order to manage the potentially large volume of users, we first determined how we would grant access. Ultimately, we decided to open access to anyone who got manager approval. Limiting access would put up barriers to adoption, and since this was a new concept in the organization, we determined allowing anyone to request access would increase adoption rates. To support the anticipated adoption rates, we also developed a wiki-style introduction page with information about the tool as well as frequently asked questions and tips for using deviceConnect.
In the same vein as supporting new users, we also created a process for handling upgrades. Knowing that this cart would be utilized by people at all hours of the day, we came up with a strategy to coordinate outages with the QA group, and communicated those outages in a timely manner. Mobile Labs does a great job of releasing cart updates to support the latest operating system versions very quickly after they are released to the public. In the absence of a test-environment cart, we had to be prepared to move quickly & efficiently on deviceConnect upgrades in order to support the demand for new OS versions in the cart. All of this production support required close collaboration between the internal cart support team and the QA team.
Device management was the final piece needed to ensure success. Developing a strategy to manage the devices and OS versions supported by the organization was key to the cart’s success over the long term. Two major components to device management were naming someone as The Device Manager (aka a governance leader) and access to reliable analytics of mobile traffic on which to base decisions. The Device Manager was responsible for building a team of stakeholders from around the organization including QA, development, the business, and production support at a minimum. This provided a breadth of knowledge & input from across the various departments and user touch points. As an added bonus to supporting the cart’s usage, a good device management strategy also supported the overall organizational strategy around mobile device support.
Overall, people loved being able to test on a variety of devices from wherever they were located. We even had several cases where business owners were able to demo new, unreleased features to stakeholders without having to figure out how to project from an iPhone or Android phone. Managers were relieved at not having to manage a large device library. The number of tests executed increased and the cost per test decreased by supporting offshore testing teams. The business continues to see the value in the decreased overhead of device library management and an increased number of test cases being executed.