Mobile App Interested in A/B Testing? Here Are Your Options

Published February 24, 2014 16:15 | By Ayo Omojola

hipmob

Editor's note: Ayo Omojola is Co-founder & CEO at Hipmob.


Web developers have long had tools for A/B testing, including standalone tools like Optimizely, Unbounce and Visual Website Optimizer, and marketing software with testing built in such as Hubspot and Adobe. For app developers it's only recently that similar tools have cropped up. As a heavy user of Optimizely on the web, I appreciate the value of a) making changes without having to write code (because often the marketer or person doing the A/B testing is not the same person developing the app) and b) making changes without having to resubmit to the appstore.

Given that every app update for iOS requires approval (and a few days), this decouples testing and results from the AppStore approval cycle. So if you're in the market for an A/B Testing tool, here are your options, most of which accomplish both (a) and (b).


1. Apptimize Available for iOS and Android, starts at $50/month for 200,000 monthly active users. Enables updates/changes without resubmitting to the App Store.Apptimize

2. Leanplum Available for iOS, Android, PhoneGap, and Unity. Starts at $79/month for 1,000 daily active users. Also enables visual (in browser) editing of your app.Leanplum

3. Taplytics Currently available only on iOS, and starts at $20/month for 10,000 daily active users. Also enables visual (in browser) editing of your app.Taplytics

4. Artisan Available for iOS and Android, and starts at $1,000/month for 25,000 monthly active users. Artisan is one of a couple of A/B testing frameworks which comes as part of a larger analytics package. Also enables visual (in browser) editing of your app.Artisan

5. Vessel Available for iOS, Android, PhoneGap/HTML5, and Windows phone. Vessel.io starts at $150/month and also comes as part of a larger analytics package, which enables visual (in browser) app editing.Vessel

6. Swrve Swrve is very clear about being an enterprise solution. They don't have a lot of documentation or feature descriptions publicly available so it's hard to tell what their service actually does. That being said, they do list EA, Gameloft and Activision as customers.Swrve

7. Arise Available for iOS and Android, Arise.io starts at 99 Euros for 50,000 monthly active users.Arise

8. Splitforce Splitforce is available for iOS and Unity, and starts at $299 for 5,000 daily active users.Splitforce

9. Optimimo Optimimo is available on Android only and starts at $49 per month per 10,000 daily active users.Optimimo

10. Amazon A/B Testing Built on AWS, and available for iOS, Android and Android Kindle Fire. It's completely free (no limits on apps, team members or monthly active users). Amazon A/B testing is still in beta though, so this pricing might change.Amazon A/B Testing

11. Appiterate Appiterate touts itself as "A WYSIWYG A/B Testing Platform for Native Mobile Apps." They have a free one month trial then pricing starts at $49/month for up to 10K MAUs. Appiterate

Open Source A/B testing tools for mobile. (note - these repos look like they haven't been active in about a year)
1. Abba by Stripe: https://github.com/maccman/abba (Javascript)
2. Switchboard by Keepsafe: https://github.com/KeepSafe/Switchboard (iOS and Android)
3. Clutch.io by Twitter: https://github.com/clutchio/clutch

Appiterate

If there's anything I missed, please share! A full list with comparisons of all the services is up here. and here And (shameless plug) if you're looking for other conversion and communication tools for your app, check us (Hipmob) out for in-app messaging and in-app helpdesk.