Shopify

Shopify

799 Followers
Shopify powers tens of thousands of online retailers including General Electric, Amnesty International, CrossFit, Tesla Motors ... more
Tools by Shopify

Decisions 8

Kir Shatrov

Production Engineer at Shopify

The core Shopify app has remained a Rails monolith, but we also have hundreds of other Rails apps across the organization. These are not microservices, but domain-specific apps: Shipping (talks with various shipping providers), Identity (single sign on across all Shopify stores), and App Store to name a few. Managing a hundred apps and keeping them up to date with security updates can be tough, so we've developed ServicesDB, an internal app that keeps track of all production services and helps developers to make sure that they don't miss anything important.

ServicesDB keeps a checklist for each app: ownership, uptime, logs, on-call rotation, exception reporting, and gem security updates. If there are problems with any of those, ServicesDB opens a GitHub issue and pings owners of the app to ask them to address it. ServicesDB also makes it easy to query the infrastructure and answer questions like, “How many apps are on Rails 4.2? How many apps are using an outdated version of gem X? Which apps are calling this service?”.

20 89.9K

Kir Shatrov

Production Engineer at Shopify

jQueryjQueryJavaScriptJavaScript
#FrameworksFullStack

The client-side stack of Shopify Admin has been a long journey. It started with HTML templates, jQuery and Prototype. We moved to Batman.js, our in-house Single-Page-Application framework (SPA), in 2013. Then, we re-evaluated our approach and moved back to statically rendered HTML and vanilla JavaScript. As the front-end ecosystem matured, we felt that it was time to rethink our approach again. Last year, we started working on moving Shopify Admin to React and TypeScript.

Many things have changed since the days of jQuery and Batman. JavaScript execution is much faster. We can easily render our apps on the server to do less work on the client, and the resources and tooling for developers are substantially better with React than we ever had with Batman.

#FrameworksFullStack #Languages

18 225.4K

Kir Shatrov

Production Engineer at Shopify

At Shopify, over the years, we moved from shards to the concept of "pods". A pod is a fully isolated instance of Shopify with its own datastores like MySQL, Redis, Memcached. A pod can be spawned in any region. This approach has helped us eliminate global outages. As of today, we have more than a hundred pods, and since moving to this architecture we haven't had any major outages that affected all of Shopify. An outage today only affects a single pod or region.

As we grew into hundreds of shards and pods, it became clear that we needed a solution to orchestrate those deployments. Today, we use Docker, Kubernetes, and Google Kubernetes Engine to make it easy to bootstrap resources for new Shopify Pods.

13 253K

Kir Shatrov

Production Engineer at Shopify

kubernetes-deploykubernetes-deployShipitShipit
#BuildTestDeploy

Shipit, our deployment tool, is at the heart of Continuous Delivery at Shopify. Shipit is an orchestrator that runs and tracks progress of any deploy script that you provide for a project. It supports deploying to Rubygems, Pip, Heroku and Capistrano out of the box. For us, it's mostly kubernetes-deploy or Capistrano for legacy projects.

We use a slightly tweaked GitHub flow, with feature development going in branches and the master branch being the source of truth for the state of things in production. When your PR is ready, you add it to the Merge Queue in ShipIt. The idea behind the Merge Queue is to control the rate of code that is being merged to master branch. In the busy hours, we have many developers who want to merge the PRs, but at the same time we don't want to introduce too many changes to the system at the same time. Merge Queue limits deploys to 5-10 commits at a time, which makes it easier to identify issues and roll back in case we notice any unexpected behaviour after the deploy.

We use a browser extension to make Merge Queue play nicely with the Merge button on GitHub:

Both Shipit and kubernetes-deploy are open source, and we've heard quite a few success stories from companies who have adopted our flow.

#BuildTestDeploy #ContainerTools #ApplicationHosting #PlatformAsAService

13 65.6K

Followers 799

Vinod Verma
Steve Meisner
Jonathan Jurgens
prem singh
Karim Kawambwa
Nur Kordi - Inveon
Pal Fish
Olivier Alcouffe
Edgar Gustavo Gamboa Sánchez
Justin Foster
Jean-Marc INIKO
Emmanuel Jaftha
Luis Fernando Villanueva Pérez
Tom Tolleson
Thiago Bruno
Naveen Bonthu
Emmanuel Kwame Amoako Wilson
Taylor Page
Santiago Escobar
Zack_ Prod
Xavier Maximin
Allan Jiang
ISmoke Fresh
Javed Khan
simonbalazic5233
Daniel Tsionit
Jophin Joseph