What is AWS Elastic Beanstalk?
Who uses AWS Elastic Beanstalk?
AWS Elastic Beanstalk Integrations
Why developers like AWS Elastic Beanstalk?
Here are some stack decisions, common use cases and reviews by companies and developers who chose AWS Elastic Beanstalk in their tech stack.
Back in 2014, I was given an opportunity to re-architect SmartZip Analytics platform, and flagship product: SmartTargeting. This is a SaaS software helping real estate professionals keeping up with their prospects and leads in a given neighborhood/territory, finding out (thanks to predictive analytics) who's the most likely to list/sell their home, and running cross-channel marketing automation against them: direct mail, online ads, email... The company also does provide Data APIs to Enterprise customers.
I had inherited years and years of technical debt and I knew things had to change radically. The first enabler to this was to make use of the cloud and go with AWS, so we would stop re-inventing the wheel, and build around managed/scalable services.
For the SaaS product, we kept on working with Rails as this was what my team had the most knowledge in. We've however broken up the monolith and decoupled the front-end application from the backend thanks to the use of Rails API so we'd get independently scalable micro-services from now on.
Our various applications could now be deployed using AWS Elastic Beanstalk so we wouldn't waste any more efforts writing time-consuming Capistrano deployment scripts for instance. Combined with Docker so our application would run within its own container, independently from the underlying host configuration.
Storage-wise, we went with Amazon S3 and ditched any pre-existing local or network storage people used to deal with in our legacy systems. On the database side: Amazon RDS / MySQL initially. Ultimately migrated to Amazon RDS for Aurora / MySQL when it got released. Once again, here you need a managed service your cloud provider handles for you.
Future improvements / technology decisions included:
Caching: Amazon ElastiCache / Memcached CDN: Amazon CloudFront Systems Integration: Segment / Zapier Data-warehousing: Amazon Redshift BI: Amazon Quicksight / Superset Search: Elasticsearch / Amazon Elasticsearch Service / Algolia Monitoring: New Relic
As our usage grows, patterns changed, and/or our business needs evolved, my role as Engineering Manager then Director of Engineering was also to ensure my team kept on learning and innovating, while delivering on business value.
One of these innovations was to get ourselves into Serverless : Adopting AWS Lambda was a big step forward. At the time, only available for Node.js (Not Ruby ) but a great way to handle cost efficiency, unpredictable traffic, sudden bursts of traffic... Ultimately you want the whole chain of services involved in a call to be serverless, and that's when we've started leveraging Amazon DynamoDB on these projects so they'd be fully scalable.
We initially started out with Heroku as our PaaS provider due to a desire to use it by our original developer for our Ruby on Rails application/website at the time. We were finding response times slow, it was painfully slow, sometimes taking 10 seconds to start loading the main page. Moving up to the next "compute" level was going to be very expensive.
We moved our site over to AWS Elastic Beanstalk , not only did response times on the site practically become instant, our cloud bill for the application was cut in half.
In database world we are currently using Amazon RDS for PostgreSQL also, we have both MariaDB and Microsoft SQL Server both hosted on Amazon RDS. The plan is to migrate to AWS Aurora Serverless for all 3 of those database systems.
Additional services we use for our public applications: AWS Lambda, Python, Redis, Memcached, AWS Elastic Load Balancing (ELB), Amazon Elasticsearch Service, Amazon ElastiCache
When creating the web infrastructure for our start-up, I wanted to host our app on a PaaS to get started quickly.
A very popular one for Rails is Heroku, which I love for free hobby side projects, but never used professionally. On the other hand, I was very familiar with the AWS ecosystem, and since I was going to use some of its services anyways, I thought: why not go all in on it?
It turns out that Amazon offers a PaaS called AWS Elastic Beanstalk, which is basically like an “AWS Heroku”. It even comes with a similar command-line utility, called "eb”. While edge-case Rails problems are not as well documented as with Heroku, it was very satisfying to manage all our cloud services under the same AWS account. There are auto-scaling options for web and worker instances, which is a nice touch. Overall, it was reliable, and I would recommend it to anyone planning on heavily using AWS.
As Mixmax began to scale super quickly, with more and more customers joining the platform, we started to see that the Meteor app was still having a lot of trouble scaling due to how it tried to provide its reactivity layer. To be honest, this led to a brutal summer of playing Galaxy container whack-a-mole as containers would saturate their CPU and become unresponsive. I’ll never forget hacking away at building a new microservice to relieve the load on the system so that we’d stop getting paged every 30-40 minutes. Luckily, we’ve never had to do that again! After stabilizing the system, we had to build out two more microservices to provide the necessary reactivity and authentication layers as we rebuilt our Meteor app from the ground up in Node.js. This also had the added benefit of being able to deploy the entire application in the same AWS VPCs. Thankfully, AWS had also released their ALB product so that we didn’t have to build and maintain our own websocket layer in Amazon EC2. All of our microservices, except for one special Go one, are now in Node with an nginx frontend on each instance, all behind AWS Elastic Load Balancing (ELB) or ALBs running in AWS Elastic Beanstalk.
delayed_job is a great Rails background job library for new projects, as it only uses what you already have: a relational database. We happily used it during the company’s first two years.
But it started to falter as our web and database transactions significantly grew. Our app interacted with users via SMS texts sent inside background jobs. Because the delayed_job daemon ran every couple seconds, this meant that users often waited several long seconds before getting text replies, which was not acceptable. Moreover, job processing was done inside AWS Elastic Beanstalk web instances, which were already under stress and not meant to handle jobs.
We needed a fast background job system that could process jobs in near real-time and integrate well with AWS. Sidekiq is a fast and popular Ruby background job library, but it does not leverage the Elastic Beanstalk worker architecture, and you have to maintain a Redis instance.
We ended up choosing active-elastic-job, which seamlessly integrates with worker instances and Amazon SQS. SQS is a fast queue and you don’t need to worry about infrastructure or scaling, as AWS handles it for you.
We noticed significant performance gains immediately after making the switch.
AWS Elastic Beanstalk is Amazon's excellent PaaS offering that saves me from a bunch of time and effort building and maintaining my own platform.
Rather than me having to DIY configuration for Amazon EC2 , Amazon CloudWatch , Amazon VPC , AWS Elastic Load Balancing (ELB) and more, AWS Elastic Beanstalk handles that for me, so I can focus on my app/service.
AWS Elastic Beanstalk's Features
- Elastic Beanstalk is built using familiar software stacks such as the Apache HTTP Server for Node.js, PHP and Python, Passenger for Ruby, IIS 7.5 for .NET, and Apache Tomcat for Java
- There is no additional charge for Elastic Beanstalk - you pay only for the AWS resources needed to store and run your applications.
- Easy to begin – Elastic Beanstalk is a quick and simple way to deploy your application to AWS. You simply use the AWS Management Console, Git deployment, or an integrated development environment (IDE) such as Eclipse or Visual Studio to upload your application
- Impossible to outgrow – Elastic Beanstalk automatically scales your application up and down based on default Auto Scaling settings
- Complete control – Elastic Beanstalk lets you "open the hood" and retain full control over the AWS resources powering your application
- Flexible – You have the freedom to select the Amazon EC2 instance type that is optimal for your application based on CPU and memory requirements, and can choose from several available database options
- Reliable – Elastic Beanstalk runs within Amazon's proven network infrastructure and datacenters, and provides an environment where developers can run applications requiring high durability and availability.