Need advice about which tool to choose?Ask the StackShare community!

AWS Device Farm

75
178
+ 1
5
Sauce Labs

312
433
+ 1
438
Add tool

AWS Device Farm vs Sauce Labs: What are the differences?

Introduction

In this article, we will discuss the key differences between AWS Device Farm and Sauce Labs. Both tools offer cloud-based testing services and are widely used by developers and testers to improve the quality of their mobile and web applications.

  1. Deployment Options: AWS Device Farm primarily focuses on native and hybrid mobile applications, offering extensive support for iOS and Android devices. It provides its own set of real devices in the cloud for testing, allowing users to run tests on a wide range of device configurations. On the other hand, Sauce Labs supports both web and mobile application testing. In addition to real devices, Sauce Labs also provides virtual machines for web application testing on different operating systems and browser versions.

  2. Integration with CI/CD Pipelines: AWS Device Farm seamlessly integrates with popular continuous integration and continuous deployment (CI/CD) tools like Jenkins, with plugins available for easy integration. It allows developers to automate testing as part of their CI/CD pipelines. Sauce Labs also offers similar integrations with industry-standard CI/CD tools, making it possible to run tests automatically by triggering them from the CI/CD pipeline stages.

  3. Testing Scalability: AWS Device Farm provides the ability to run tests on multiple devices concurrently, allowing for quick parallel execution of tests. It offers flexible scheduling options, allowing users to run tests in parallel on different devices or sequentially to ensure comprehensive test coverage. On the other hand, Sauce Labs offers even higher scalability with the capability to run tests in parallel across hundreds of virtual machines and real devices, enabling faster execution and shorter test cycles.

  4. Test Automation Frameworks: AWS Device Farm supports popular test automation frameworks such as Appium, Calabash, Espresso, and XCTest for mobile app testing. It also provides a built-in web-based test automation framework called AWS Device Farm TestNG Selenium. Sauce Labs supports a wide range of test automation frameworks including Selenium WebDriver, Appium, Protractor, and Espresso, making it highly versatile for different types of testing scenarios.

  5. Reporting and Analysis: AWS Device Farm provides detailed test execution reports, including logs, screenshots, and videos, to help diagnose and troubleshoot issues. It also integrates with AWS CloudWatch to collect real-time metrics for monitoring and analysis. Sauce Labs offers rich reporting capabilities with detailed information about test runs, screenshots, videos, and performance data. It also provides real-time analytics dashboards for monitoring and tracking test results.

  6. Integration with Third-Party Tools: AWS Device Farm can be seamlessly integrated with other AWS services such as AWS CodePipeline and AWS CloudFormation for streamlined application development and deployment. It also offers APIs for integrating with custom tools and frameworks. Sauce Labs provides integration with various third-party tools including popular issue tracking systems, collaboration platforms, and project management tools, enabling easy integration into existing development workflows.

In summary, AWS Device Farm and Sauce Labs offer powerful cloud-based testing services, each with its own set of unique features and strengths. While AWS Device Farm is particularly suited for native and hybrid mobile app testing with extensive device support, Sauce Labs provides broader coverage for both web and mobile app testing and offers advanced scalability and reporting capabilities. The choice between the two ultimately depends on the specific requirements of the project and the testing needs of the organization.

Advice on AWS Device Farm and Sauce Labs

I am looking to purchase one of these tools for Mobile testing for my team. It should support Native, hybrid, and responsive app testing. It should also feature debugging, parallel execution, automation testing/easy integration with automation testing tools like Selenium, and the capability to provide availability of devices specifically for us to use at any time with good speed of performing all these activities.

I have already used Perfecto mobile, and Sauce Labs in my other projects before. I want to know how different or better is AWS Device farm in usage and how advantageous it would be for us to use it over other mentioned tools

See more
Replies (3)
Aaron Evans
Testing Strategist at One Shore · | 3 upvotes · 5.8K views

A SaaS offering like Sauce Labs (or BrowserStack or LambdaTest, etc) will provide a remote Selenium/Appium Grid including the ability to run test automation in parallel (up to the amount based your subscription level) an a wide array of browsers and mobile devices.

These tools can be expensive, but if you can afford them, the expertise and effort of maintaining the grid, browser updates, etc. is worth it.

AWS Device Farm can be significantly cheaper, but is much more work to setup and run. It will not give you as many devices, or the reporting and screen/video capture you get with the the services. And there is no support for AWS Device Farm, and very poor documentation. I have used it, but do not recommend it. Running your own grid and physical device lab is better, but I'd stick with a service like Sauce Labs or Perfecto which will save you time and give you better services despite the higher price tag.

See more
Recommends
on
BrowserStackBrowserStack

Stability - Just works. Availability - More than 15 datacenters. Enterprise features like SSO, local testing and SOC2/GDPR compliant.

See more
Jaymie Falconi
Recommends
on
BitBarBitBar

BitBar's Dedicated Devices would be a great option for you. It allows you to dedicate (reserve) devices for your use only which also having access to all of the devices in the shared cloud. BitBar has the features and integrations that you are looking for as well.

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of AWS Device Farm
Pros of Sauce Labs
  • 3
    1000 free minutes
  • 2
    Pay as you go pricing
  • 60
    Selenium-compatible
  • 46
    Webdriver compatible
  • 35
    Video recordings of every test
  • 31
    Qa
  • 29
    Mobile support
  • 26
    Any programming language
  • 23
    Developer tools
  • 21
    Test local and firewalled servers
  • 20
    Jenkins integration
  • 18
    Pristine VMs
  • 17
    CI Compatible
  • 11
    Appium support
  • 9
    Parallel testing
  • 8
    Rapid environment preparation
  • 8
    Mobile device support
  • 7
    Easy testing on almost any device
  • 7
    Allows me to Focus more test automation rather than IT
  • 6
    Secure testing and easy setup
  • 5
    Easy setup with CI and fast automated tests
  • 5
    Quick support response
  • 4
    Fast and reliable to host the automated tests
  • 4
    Easy to setup and understand,
  • 3
    Easy setup and integration with Travis Ci
  • 3
    Maintained browser matrix
  • 3
    Easy onboarding, do not need to manager VMs/OS/Browsers
  • 2
    Efficient tool to verify product quality
  • 2
    Teamcity Integration and mobile testing win
  • 2
    Hany for platform testing
  • 2
    Great documentation
  • 2
    Generous free trial
  • 2
    Easy. Straightforward. Scalable
  • 2
    Great way to integrate test suite on cloud
  • 2
    Simplicity of Sauce-connect
  • 1
    Very Good, Quick, flexible Infrastructure Support
  • 1
    It's great for my QA work
  • 1
    Awesome tech support
  • 1
    Having this available for CI servers is fantastic
  • 1
    Amazing service to do cloud cross browser testing
  • 1
    Depth of integrations
  • 1
    Because of its cloud based support for appium
  • 1
    Easy setup, Works great with selenium.
  • 1
    QE support
  • 1
    Manuals are not very well versed for beginners
  • 1
    Secure testing
  • 1
    Cheaper than browserstack
  • 1
    Stable
  • 0
    Simple to set up and integrate so many browser configs

Sign up to add or upvote prosMake informed product decisions

Cons of AWS Device Farm
Cons of Sauce Labs
  • 1
    Records all sessions, blocks on processing when done
  • 1
    You need to remember to turn airplane mode off
  • 2
    Relatively slow
  • 2
    Expensive

Sign up to add or upvote consMake informed product decisions

What is AWS Device Farm?

Run tests across a large selection of physical devices in parallel from various manufacturers with varying hardware, OS versions and form factors.

What is Sauce Labs?

Cloud-based automated testing platform enables developers and QEs to perform functional, JavaScript unit, and manual tests with Selenium or Appium on web and mobile apps. Videos and screenshots for easy debugging. Secure and CI-ready.

Need advice about which tool to choose?Ask the StackShare community!

What companies use AWS Device Farm?
What companies use Sauce Labs?
See which teams inside your own company are using AWS Device Farm or Sauce Labs.
Sign up for StackShare EnterpriseLearn More

Sign up to get full access to all the companiesMake informed product decisions

What tools integrate with AWS Device Farm?
What tools integrate with Sauce Labs?

Sign up to get full access to all the tool integrationsMake informed product decisions

What are some alternatives to AWS Device Farm and Sauce Labs?
Xamarin Test Cloud
Run your app on our huge (and growing) collection of real devices from around the world. Select devices based on form factor, manufacturer, operating system, or even popularity in your target market. We’re adding over 100 devices every month, and if there’s a specific device you need, we’re taking requests.
Firebase
Firebase is a cloud service designed to power real-time, collaborative applications. Simply add the Firebase library to your application to gain access to a shared data structure; any changes you make to that data are automatically synchronized with the Firebase cloud and with other clients within milliseconds.
BrowserStack
BrowserStack is the leading test platform built for developers & QAs to expand test coverage, scale & optimize testing with cross-browser, real device cloud, accessibility, visual testing, test management, and test observability.
Xamarin
Xamarin’s Mono-based products enable .NET developers to use their existing code, libraries and tools (including Visual Studio*), as well as skills in .NET and the C# programming language, to create mobile applications for the industry’s most widely-used mobile devices, including Android-based smartphones and tablets, iPhone, iPad and iPod Touch.
Kobiton
It enables developers and testers to perform automated and manual testing of mobile apps and websites on real devices. Modern DevOps and Quality environments require apps to be tested on hundreds of device/OS/browser combinations. Managing an in-house device-lab is expensive, resource intensive, restrictive and very manual. Kobiton allows for instant provisioning of real devices for testing with automated or manual scripts, and also allows current on-premise devices to be plugged in to form a holistic testing cloud.
See all alternatives