Google Kubernetes Engine

We began our hosting journey, as many do, on Heroku because they make it easy to deploy your application and automate some of the routine tasks associated with deployments, etc. However, as our team grew and our product matured, our needs have outgrown Heroku. I will dive into the history and reasons for this in a future blog post.

We decided to migrate our infrastructure to Kubernetes running on Amazon EKS. Although Google Kubernetes Engine has a slightly more mature Kubernetes offering and is more user-friendly; we decided to go with EKS because we already using other AWS services (including a previous migration from Heroku Postgres to AWS RDS). We are still in the process of moving our main website workloads to EKS, however we have successfully migrate all our staging and testing PR apps to run in a staging cluster. We developed a Slack chatops application (also running in the cluster) which automates all the common tasks of spinning up and managing a production-like cluster for a pull request. This allows our engineering team to iterate quickly and safely test code in a full production environment. Helm plays a central role when deploying our staging apps into the cluster. We use CircleCI to build docker containers for each PR push, which are then published to Amazon EC2 Container Service (ECR). An upgrade-operator process watches the ECR repository for new containers and then uses Helm to rollout updates to the staging environments. All this happens automatically and makes it really easy for developers to get code onto servers quickly. The immutable and isolated nature of our staging environments means that we can do anything we want in that environment and quickly re-create or restore the environment to start over.

The next step in our journey is to migrate our production workloads to an EKS cluster and build out the CD workflows to get our containers promoted to that cluster after our QA testing is complete in our staging environments.

I was thinking what could be the best option for deploying Daily's #frontend applications. On one side there is Netlify , promoting and encouraging the JAM stack, very easy to deploy and manage your deployments. And on the other side, Kubernetes , specifically Google Kubernetes Engine as I don't like to manage my own clusters. Kubernetes provides much more options, in terms of deployment strategy, networking, etc but requires far more configurations. As I don't have any SSR on my applications, I decided that the ease of use of Netlify is the number one priority for the project.

Google Kubernetes Engine no-ops. Maintaining reliable Kubernetes setup is not priority for the product driven engineering team. GKE makes it incredibly simple and cost-effective to run our container applications

We use Kubernetes because it is the best and easiest way to orchestrate your klusters. Intergrated with Amazon EC2 Container Service or Google Kubernetes Engine it works wonderfully.

We recently moved our main applications from Heroku to Kubernetes . The 3 main driving factors behind the switch were scalability (database size limits), security (the inability to set up PostgreSQL instances in private networks), and costs (GCP is cheaper for raw computing resources).

We prefer using managed services, so we are using Google Kubernetes Engine with Google Cloud SQL for PostgreSQL for our PostgreSQL databases and Google Cloud Memorystore for Redis . For our CI/CD pipeline, we are using CircleCI and Google Cloud Build to deploy applications managed with Helm . The new infrastructure is managed with Terraform .

Read the blog post to go more in depth.

We are hardcore Kubernetes users and contributors. We loved the automation it provides. However, as our team grew and added more clusters and microservices, capacity and resources management becomes a massive pain to us. We started suffering from a lot of outages and unexpected behavior as we promote our code from dev to production environments. Luckily we were working on our AI-powered tools to understand different dependencies, predict usage, and calculate the right resources and configurations that should be applied to our infrastructure and microservices. We dogfooded our agent (http://github.com/magalixcorp/magalix-agent) and were able to stabilize as the #autopilot continuously recovered any miscalculations we made or because of unexpected changes in workloads. We are open sourcing our agent in a few days. Check it out and let us know what you think! We run workloads on Microsoft Azure Google Kubernetes Engine and Amazon EC2 and we're all about Go and Python!

So, the shift from Amazon EC2 to Google App Engine and generally #AWS to #GCP was a long decision and in the end, it's one that we've taken with eyes open and that we reserve the right to modify at any time. And to be clear, we continue to do a lot of stuff with AWS. But, by default, the content of the decision was, for our consumer-facing products, we're going to use GCP first. And if there's some reason why we don't think that's going to work out great, then we'll happily use AWS. In practice, that hasn't really happened. We've been able to meet almost 100% of our needs in GCP.

So it's basically mostly Google Kubernetes Engine , we're mostly running stuff on Kubernetes right now.

#AWStoGCPmigration #cloudmigration #migration

At Shopify, over the years, we moved from shards to the concept of "pods". A pod is a fully isolated instance of Shopify with its own datastores like MySQL, Redis, Memcached. A pod can be spawned in any region. This approach has helped us eliminate global outages. As of today, we have more than a hundred pods, and since moving to this architecture we haven't had any major outages that affected all of Shopify. An outage today only affects a single pod or region.

As we grew into hundreds of shards and pods, it became clear that we needed a solution to orchestrate those deployments. Today, we use Docker, Kubernetes, and Google Kubernetes Engine to make it easy to bootstrap resources for new Shopify Pods.

used this when setting up the kubernetes infrastructure for a client. Google Kubernetes Engine