Karma vs Sauce Labs: What are the differences?
Karma and Sauce Labs can be categorized as "Browser Testing" tools.
Some of the features offered by Karma are:
- Test on Real Devices
- Remote Control
- Testing Framework Agnostic
On the other hand, Sauce Labs provides the following key features:
- 700+ browser/OS/device combinations for cross-browser and platform testing to improve web and mobile app quality and eliminate the overhead of internal infrastructure
- Highly reliable, on-demand cloud for enterprise-grade scalability and industry standard security
- Optimized for popular testing frameworks, CI systems, and surrounding tools and services
"Test Runner" is the primary reason why developers consider Karma over the competitors, whereas "Selenium-compatible" was stated as the key factor in picking Sauce Labs.
Karma is an open source tool with 10.7K GitHub stars and 1.61K GitHub forks. Here's a link to Karma's open source repository on GitHub.
According to the StackShare community, Karma has a broader approval, being mentioned in 119 company stacks & 57 developers stacks; compared to Sauce Labs, which is listed in 66 company stacks and 11 developer stacks.
What is Karma?
What is Sauce Labs?
Need advice about which tool to choose?Ask the StackShare community!
Sign up to add, upvote and see more prosMake informed product decisions
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions
I am working on #OpenSource file uploader. The uploader is the widget that other developers embed in their apps. It should work well in different browsers and on different devices. BrowserStack and Sauce Labs help to achieve that. I can test the uploader in many varieties of browsers+OS only used my browser without virtual machines.
Sauce Labs is a fantastic testing tool that I am using for my QA internship.
I am particularly happy with the speed of which any virtual machine loads - that saves me heaps of time and allows me to concentrate on executing more tests instead of blankly staring at the screen (and I do appreciate funny quotes displayed during the minimal wait) . I constantly have to switch between different OS and browsers and so far I had nothing but speedy and reliable experience. Moreover, getting Sauce Labs to work with local links could not have been easier!
Sauce Labs tool is in general very simple to use for either manual or automated testing, with a lot of great resources available that are easily accessible. I additionally feel that Sauce Labs goes beyond a testing tool - it is also a community of testers and for testers, which I find really appealing. I have been using BrowserStack previously and although it was also a great testing tool, I feel that it is lacking the support Sauce Labs offers.
All in all - I only have positive things to say about Sauce Labs, the only thing that could possibly be improved (and I'm just being picky) is the speed of which iOS cloud loads - it appears to be slightly slower than other virtual machines.
SauceLabs is widely used by QAs for both types of testing manual and automation. It provides possibility of compatibility testing on different OS and browsers + mobile platforms for those who don't have big test stand with all real devices. I started to use SauceLabs only because of not having real devices to test website for compatibility and responsive design. After this useful experience I started to learn about automation with Selenium Webdriver and again SauceLabs helped me with tests execution on different platforms. It's easy to setup and to integrate in existing solutions and saved us a lot of time. I love reporting provided by SauceLabs, you can check video and screenshots after or view running test on the fly. Now I recommend SauceLabs to all QAs in my and other companies, even developers are using it to reproduce bugs and check their code on different devices. SauceLabs has very good documentation and big community where any appeared question can be quickly answered. I receive their emails about news and tips how to use it efficiently, sometimes there are useful webinars at SauceLabs. Thank you for this great everyday job!
We chose to use Sauce Labs as our Selenium Grid in the cloud b/c we didn't want to support testing on Windows/IE in-house due to security concerns.
Thus choosing a service like Sauce Labs was the only solution we had to execute our Watir-WebDriver (Selenium) acceptance tests on IE and other browser/OS combos.
Pros: It took me a few hours to get setup and immediately start running tests. Our main test stack consists of: Cucumber/Watir-WebDriver (Selenium-WebDriver) and getting it up and running was not that hard. It was very fast to spin up a VM and run tests and view the screencasts and screenshots. Easier to use this as opposed to setting up an in-house Selenium Grid setup.
Cons: I didn't find ANY documentation on how to setup Watir-WebDriver on Sauce Labs, I had to figure it out via Googling. The Watir tests do run a tad bit slow on SL. The UI is a bit dated, tried out the new beta UI and it looks nicer but doesn't seem to be ready for production use, the screencast view in beta mode was way too big.
Overall I like it as it's easy to use and gets the job done. I have yet to setup w/Jenkins, that's the next stop.
We worked on a 3-year engagement with a major pharmaceutical company. Our builds had to go through a complex review and deployment process with different many players making code and configuration changes across various build profiles and environments. We wanted to ensure that nothing was broken in our application code in this process, and that any breakages would be immediately identified.
We used Sauce Labs to create a testing suite for our app, and added tests with each new release. This helped us identify several critical issues before they ever surfaced as problems on the public-facing sites. We found the Sauce Labs platform very easy to use. We were already experienced with Selenium, but had a few questions about how to run and manage our tests on Sauce. We found the documentation to be quite helpful. On the rare occasions that we had to reach out to Sauce Labs support, we received prompt, thorough replies which kept our development process moving forward at full speed.
The Sauce Labs tools have become a vital part of our development infrastructure.
Like: Lot's of OS/Browsers for tests, supports all major frameworks. Works great with Jenkins(their Sauce OnDemand plug-in is pretty awesome). Can be used in many ways and very versatile.
Lots of business problem solved....just having a quick ability to create test environments. That's a big one and Sauce Labs solves that. A realized benefit is having the ability to have videos and screenshots saved of tests.
Dislike: I would like to see improved would be the performance. Tests run faster locally than they do on Sauce, but if you run your tests in parallel and can have a number of VM's running at the same time then this usually doesn't create much of a problem.
Other Feedback: You can look at other cloud options like BrowserStack, but they don't give you the same ease of use and support for the price. You can create test agents in your own network, but they you have to constantly maintain each vm or system and that eats up valuable time. It's much easier and price effective to go with Sauce plus you get great support with them when you run into issues.
Browser testing grid and test automation so we don't need the servers to do it. Integrates with CI.
All services, directives and controllers from angular are tested using Karma.