Need advice about which tool to choose?Ask the StackShare community!
Amazon EC2 vs Amazon EMR: What are the differences?
Amazon EC2: Scalable, pay-as-you-go compute capacity in the cloud. Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers; Amazon EMR: Distribute your data and processing across a Amazon EC2 instances using Hadoop. Amazon EMR is used in a variety of applications, including log analysis, web indexing, data warehousing, machine learning, financial analysis, scientific simulation, and bioinformatics. Customers launch millions of Amazon EMR clusters every year.
Amazon EC2 can be classified as a tool in the "Cloud Hosting" category, while Amazon EMR is grouped under "Big Data as a Service".
Some of the features offered by Amazon EC2 are:
- Elastic – Amazon EC2 enables you to increase or decrease capacity within minutes, not hours or days. You can commission one, hundreds or even thousands of server instances simultaneously.
- Completely Controlled – You have complete control of your instances. You have root access to each one, and you can interact with them as you would any machine.
- Flexible – You have the choice of multiple instance types, operating systems, and software packages. Amazon EC2 allows you to select a configuration of memory, CPU, instance storage, and the boot partition size that is optimal for your choice of operating system and application.
On the other hand, Amazon EMR provides the following key features:
- Elastic- Amazon EMR enables you to quickly and easily provision as much capacity as you need and add or remove capacity at any time. Deploy multiple clusters or resize a running cluster
- Low Cost- Amazon EMR is designed to reduce the cost of processing large amounts of data. Some of the features that make it low cost include low hourly pricing, Amazon EC2 Spot integration, Amazon EC2 Reserved Instance integration, elasticity, and Amazon S3 integration.
- Flexible Data Stores- With Amazon EMR, you can leverage multiple data stores, including Amazon S3, the Hadoop Distributed File System (HDFS), and Amazon DynamoDB.
"Quick and reliable cloud servers" is the primary reason why developers consider Amazon EC2 over the competitors, whereas "On demand processing power" was stated as the key factor in picking Amazon EMR.
Airbnb, Uber Technologies, and Netflix are some of the popular companies that use Amazon EC2, whereas Amazon EMR is used by Netflix, Medium, and Yelp. Amazon EC2 has a broader approval, being mentioned in 3607 company stacks & 1618 developers stacks; compared to Amazon EMR, which is listed in 95 company stacks and 18 developer stacks.
DigitalOcean was where I began; its USD5/month is extremely competitive and the overall experience as highly user-friendly.
However, their offerings were lacking and integrating with other resources I had on AWS was getting more costly (due to transfer costs on AWS). Eventually I moved the entire project off DO's Droplets and onto AWS's EC2.
One may initially find the cost (w/o free tier) and interface of AWS daunting however with good planning you can achieve highly cost-efficient systems with savings plans, spot instances, etcetera.
Do not dive into AWS head-first! Seriously, don't. Stand back and read pricing documentation thoroughly. You can, not to the fault of AWS, easily go way overbudget. Your first action upon getting your AWS account should be to set up billing alarms for estimated and current bill totals.
We first selected Google Cloud Platform about five years ago, because HIPAA compliance was significantly cheaper and easier on Google compared to AWS. We have stayed with Google Cloud because it provides an excellent command line tool for managing resources, and every resource has a well-designed, well-documented API. SDKs for most of these APIs are available for many popular languages. I have never worked with a cloud platform that's so amenable to automation. Google is also ahead of its competitors in Kubernetes support.
GCE is much more user friendly than EC2, though Amazon has come a very long way since the early days (pre-2010's). This can be seen in how easy it is to edit the storage attached to an instance in GCE: it's under the instance details and is edited inline. In AWS you have to click the instance > click the storage block device (new screen) > click the edit option (new modal) > resize the volume > confirm (new model) then wait a very long time. Google's is nearly instant.
- In both cases, the instance much be shut down.
There also the preference between "user burden-of-security" and automatic security: AWS goes for the former, GCE the latter.
Most bioinformatics shops nowadays are hosting on AWS or Azure, since they have HIPAA tiers and offer enterprise SLA contracts. Meanwhile Heroku hasn't historically supported HIPAA. Rackspace and Google Cloud would be other hosting providers we would consider, but we just don't get requests for them. So, we mostly focus on AWS and Azure support.
I chose DigitalOcean because their pricing is very fair. Their tech support is very quick to respond to any inquiries you may have. They also have a community of developer who are more then happy to help you with any non-account issues you may have.
The drawbacks of this decision are their managed services can be quite pricey at $15/mo extra for a MySQL database.