AWS Device Farm vs Loader.io: What are the differences?
What is AWS Device Farm? Test your app on real devices in the AWS Cloud. Run tests across a large selection of physical devices in parallel from various manufacturers with varying hardware, OS versions and form factors.
What is Loader.io? Simple Cloud-based Load Testing. Loader.io is a free load testing service that allows you to stress test your web-apps/apis with thousands of concurrent connections.
AWS Device Farm and Loader.io can be categorized as "Load and Performance Testing" tools.
According to the StackShare community, Loader.io has a broader approval, being mentioned in 14 company stacks & 8 developers stacks; compared to AWS Device Farm, which is listed in 5 company stacks and 3 developer stacks.
What is AWS Device Farm?
What is Loader.io?
Need advice about which tool to choose?Ask the StackShare community!
Why do developers choose AWS Device Farm?
What are the cons of using AWS Device Farm?
What are the cons of using Loader.io?
Sign up to get full access to all the companiesMake informed product decisions
I don’t remember exactly how I heard about Loader.io. I think I was adding load testing services to Leanstack. I saw it was a SendGrid Labs project, so there would be competent people behind it. And since they had a Heroku Add-On it was easy to get started. Loader.io is cool because it’s super simple to set up.
When executing tests, you can see error rate and average response times. But we also check the Heroku logs to see if they are real errors.
My biggest complaint: figuring out what load to set for your tests is difficult. We don’t understand the language they use and no one we’ve spoken to that has used Loader.io understands it either. We’ve been testing at 250 clients (maintain client load) for all of our tests on 2 dynos. That means a constant load of 250 people using the site over a minute, or so I thought. The number of requests at the end of the test suggests it’s more like 250 additional clients hitting the site every second for a minute. But I guess accommodating a higher load is better anyways? 250 concurrent users seems to be our average HN traffic spike so that’s why we went with that load.
I use loader.io for distributed load testing.
That especially matters when you have a cross-region load-balancer and you want to make sure how your app behaves on a global scale.