You’ve written your app, finished testing, and now it’s time for placing your app (or updated version) on iTunes or Google Play.
But wait… Have you really tested enough? Even with the best development, QA and testing practices, you still want your app to have a “final test” with hundreds, or even thousand of individuals if possible. And, if you have a high-profile app that is critical to your company’s success, this “final check point” can mean the difference between great reviews — or a potential disaster if you’ve missed something.
If you work at a larger company, you’re in luck. Many employees are willing and able to work “after 5 o’clock” to help you with mobile app testing
, especially for important apps that project the company brand, generate revenue or provide customer service. You’d like their help and want to harness the enthusiasm. However, you still need a way to distribute your app to this large audience, in a controlled manner, and ensure that you get feedback, can quickly update the test version, and gain insight.
At Apperian, we have developed a full-fledged “mobile application lifecycle
” system that is used for enterprise distribution of apps within an organization to employees and contractors. For many years Apperian’s solution has been used by companies to secure, manage and distribute “in-house” apps.
But, a funny thing happened on the way to the App Store. We found our customers using the platform to test “business-to-consumer” apps, i.e., apps not just for in-house use but for their customers! It turns out that the features we had built in mapped perfectly to the needs of large-scale “ad hoc” testing.
Historically, companies have tried to solve the “large ad-hoc distribution” problem with tools such as TestFlight or HockeyApp. However, these products are not meant for large-scale use, and require significant manual effort on the user and developer side. In addition, TestFlight was recently acquired by Apple, and now is being folded into iTunes — and has had it’s Android support removed. Customers began turning to us as a TestFlight alternative
The Apperian solution provides everything you need to feel confident that your app gets the test coverage it needs. For example, the solution allows your final app binary to have “wrapping
” applied before you distribute it. This “wrapping” adds cool features which I’ll talk about later - but does not require any SDK or changes to your code, which lowers risk.
A Real World Example
Here’s an example of how Apperian’s solution can be used in “real world” testing.
Let’s say you want to distribute your final test version to 1000 employees who have already indicated they’re interested in helping. The “test team” has a native private “enterprise app store
” installed (through a one-click install), and when you’re ready to go, you upload your app version. Because you want to know the “take up rate”, you apply a policy
of “app usage” which tracks app analytics. You want crash reports, so you also apply the “crash report” policy. Finally, you want to be able to view (and control) the user’s app while it’s running, so you select “remote control” as a policy.
Now you publish your app. Each user gets a “push notification” badge on their private App Store. But you want to make sure your app is tested soon (within the next 24 hours). Using the system, you send a notification message to all users that asks them to install and use the app, and explain why. You even provide a “bounty” for the first 10 users to report in! (Apperian includes app feedback, both via a rating and text message).
After a few hours, you realize that the version you sent actually was not the right one! Relax. Simply publish the new version “over the top” of the current version, and require “immediate update”. This will force the new version to be installed instantly by all users with no fuss. (If needed, you can also publish multiple versions to different groups.)
The next day, you can see exactly how many users ran the app, see the feedback and ratings, and view a few of the crash reports (oops!). You see from a comment that a particular user had difficulty with the app, so you contact them to run the app and allow you to “remote control” into their iPhone to “take over” the app to see the problem. Now you can see the problem “through their eyes”.
After another iteration to a smaller group (you can restrict to any group or individual), you’re ready to go! But before you finish, you want to make sure the “beta version” of the app you sent is shut down. Disable or remove the app from the test group, and voila, the app will no longer operate. You can even wipe the app data or send a message saying “Testing is over - thanks” to all the users.
In summary, large-scale ad-hoc testing of mobile apps is a best practice and can result in significant app quality improvement, leading to better app ratings and success. For mission-critical apps that do e-commerce or provide customer support, this can translate into significant ROI. And with Apperian, the ability to make testing easy — and fun — by getting help from your company’s employees is within reach.
this webinar to learn how to optimize mobile app performance testing to increase mobile app quality and ratings. Take away best practices for managing mobile app testing for hundreds or thousands of users, achieving "faster time to market" by iterating development cycles more quickly, and improving overall app quality through continuous innovation and user feedback.
Read the press release
announcing our large-scale solution for user acceptance testing for mobile apps.