Shopify

How to Introduce Composite Primary Keys in Rails

One line of code can 5x your Rails application performance. What if you could make a small change to your database design that would unlock massively more efficient data access? At Shopify, we dusted off some old database principles and did exactly that with the primary Rails application.

Building Mental Models of Ideas That Don’t Change

Developers constantly face information overload. There's always a new programming language, methodology, framework, or platform. How do you make sense of it all? You use mental models to guide you.

How to Do an In-depth Liquid Render Analysis with Theme Inspector

Shopify’s Online Store provides greater flexibility to build themes representing the brand of a merchant’s online store. However, like with any programming language, one often writes code without being aware of all the performance impact it creates. Whether it be performance impact on Shopify’s servers or observable pe

How to Build an Experiment Pipeline from Scratch

A guide to building an email experimentation pipeline from the ground up. You'll learn how to implement a similar pipeline with a relatively simple setup from scratch.

Images as Code: Representing Localized and Evolving Products on Marketing Pages

A deep dive into a JavaScript-based module to scale HTML content like an image would inside of its container and a look at why it was developed.

How to Use Quasi-experiments and Counterfactuals to Build Great Products

A/B tests are not the only tool to understand causality: quasi-experiments and counterfactuals are powerful tools for causal inference if used right.

Enforcing Modularity in Rails Apps with Packwerk

Packwerk is an open source static analysis tool used to enforce boundaries between groups of Ruby files we call packages.

Under Deconstruction: The State of Shopify’s Monolith

It’s been 18 months since we talked about our Rails modular monolith. We lay out the current state of the work, and things we’d do differently if we started again.

Tophatting in React Native

To provide a more convenient and painless experience, we built a tool enabling React Native developers to quickly load their peer’s work within seconds.

5 Ways to Improve Your React Native Styling Workflow

In April, we announced Shop, our digital shopping assistant that brings together the best features of Arrive and Shop Pay. The Shop app started from our React Native codebase for our previous package tracking app Arrive, with every screen receiving a complete visual overhaul to fit the new branding. While our product d

How to Track State with Type 2 Dimensional Models

In this post, I’ll show how you can create Type 2 dimensional models using modern ETL tooling like PySpark and dbt (data build tool).

ShipIt! Presents: A Look at Shopify's API Health Report

Shuting Chang, Robert Saunders, Karen Xie, and Vrishti Dutta join us to talk about Shopify’s API Health Report, the tool built to surface breaking changes affecting Shopify Partner apps.

How Shopify Reduced Storefront Response Times with a Rewrite

We rewrote critical software that powers all online storefronts on Shopify, offering the fastest online shopping experience, entirely from scratch and without downtime.

Building Reliable Mobile Applications

Merchants worldwide rely on Shopify's Point Of Sale (POS) app to operate their brick and mortar stores. Unlike many mobile apps, the POS app is mission-critical. Any downtime leads to long lineups, unhappy customers, and lost sales. The POS app must be exceptionally reliable, and any outages resolved quickly. Reliabili

Using DNS Traffic Management to Add Resiliency to Shopify’s Services

If you are lacking understanding of what is DNS, traffic management, or why we would even use it, read Part 1: Introduction to DNS traffic management. Distributed systems are only as resilient as we build them to be. Domain Name System (DNS) traffic management (TM) is a well-used approach to do so. In this second part

An Introduction to DNS Traffic Management

Distributed systems are only as resilient as we build them to be. Domain Name System (DNS) traffic management is a well-used approach to do so. In this first part of a two-part series, we aim to give a broad overview of DNS and how it’s used for traffic management, as well as the different reasons why we want to use DN

Migrating Large TypeScript Codebases To Project References

Using TypeScript enabled our ever-growing Admin teams to leverage TypeScript’s compiler to catch potential bugs and errors well before they ship.

ShipIt! Presents: AR/VR at Shopify

On June 20, 2020, ShipIt!, our monthly event series, presented AR/VR at Shopify. Daniel Beauchamp, Head of AR/VR at Shopify talked about how we’re using this increasingly ubiquitous technology at Shopify.

How We’re Solving Data Discovery Challenges at Shopify

Humans generate a lot of data. Every two days we create as much data as we did from the beginning of time until 2003! The International Data Corporation estimates the global datasphere totaled 33 zettabytes (one trillion gigabytes) in 2018. The estimate for 2025 is 175 ZBs, an increase of 430%. This growth is challengi

How We Built Size.link

Join Daniel on July 20, 2020 at 1 pm EST for ShipIt! Presents: AR/VR at Shopify. Daniel will be talking about what we’re doing with AR/VR at Shopify. Please Register Size.link is a free tool built by Shopify’s Augmented Reality (AR) team that lets anyone view the size of a product in the space around them using their s

Media at Scale: Callbacks vs pipelines

Shopify recently launched the ability to add videos and 3d models to products as a native feature within the Admin section. Learn how we used pipelines to manage the long-term maintenance and understandability of the code.

A First Look at Reanimated 2

With Reanimated 2, the team at Software Mansion asked: “How would it look if there were no constraints when writing gestures and animations in React Native?” and started from there.

Writing Better, Type-safe Code with Sorbet

I want to talk about one contribution that has takeaways for any Ruby developer. Here are 4 reasons for why we adopted Sorbet for static type checking.

Shopify's Data Science & Engineering Foundations

Shopify’s Data team uses these foundational approaches to data warehousing and analysis empowering us to deliver the best results for our ecosystem.

Spark Joy by Running Fewer Tests

Developers write tests to ensure correctness and allow future changes to be made safely. However, as the number of features grows, so does the number of tests. Tests are a double-edged sword. On one hand, well-written ones catch bugs and maintain a program’s stability, but as the code base grows, a high number of tests

Understanding Programs Using Graphs

I want to show you a little of what this sea-of-nodes graph data structure looks like and how it works, and I think it’s worth doing this for this for a couple of reasons.

ShipIt! Presents: How Shopify Uses Nix

On May 25, 2020,  ShipIt!, our monthly event series, presented How Shopify Uses Nix. Building upon on my What is Nix post, I show how we rebuilt our developer tooling using Nix, and show off some of the tooling we actually use at Shopify on a day-to-day basis. I wasn't able to answer all the questions during the event,

7 Ways to Make Your SQL Workshop Beginner-friendly

Here are some of the approaches we use to help data workshop participants walk away equipped and excited to start using data to power their work.

What Is Nix

This article is going to be a crash course in what Nix is, how to think about it, and why it's such a valuable and paradigm-shifting piece of technology.

A Brief History of TLS Certificates at Shopify

In this post, we’ll cover Shopify’s journey from manually provisioning TLS certificates to the fully automated system that supports over 1M businesses.

Dev Degree: Behind the Scenes

On April 24th, we proudly celebrated the graduation of our first Dev Degree program cohort. This milestone holds a special place in Shopify history because it’s a day born out of trial and error, experimentation, iteration, and hustle. The 2016 cohort had the honor and challenge of being our first class, lived through

How to Fix Slow Code in Ruby

How do we effectively find out why our application is slow? Even if we have a fix for the slow code, how can we prove that our new code is faster?

Categorizing Products at Scale

By: Jeet Mehta and Kathy Ge With over 1M business owners now on Shopify, there are billions of products being created and sold across the platform. Just like those business owners, the products that they sell are extremely diverse! Even when selling similar products, they tend to describe products very differently. One

Software Release Culture at Shopify

We try to find the path to those goals that our developers want to take. We want to create experiences through tooling that can make our developers feel productive, and we want to do our best to make shipping feel like a celebration and not a chore.

Building Arrive's Confetti in React Native with Reanimated

Shopify is investing in React Native as our primary choice of mobile technology moving forward. As a part of this we’ve rewritten our package tracking app Arrive with React Native and launched it on Android—an app that previously only had an iOS version. One of the most cherished features by the users of the Arrive iOS

Optimizing Ruby Lazy Initialization in TruffleRuby with Deoptimization

Shopify's involvement with TruffleRuby began half a year ago, with the goal of furthering the success of the project and Ruby community. TruffleRuby is an alternative implementation of the Ruby language (where the reference implementation is CRuby, or MRI) developed by Oracle Labs. TruffleRuby has high potential in spe

Refactoring Legacy Code with the Strangler Fig Pattern

There are strategies for reducing the size and responsibilities of large objects. Here’s one that worked for us at Shopify, an all-in-one commerce platform supporting over 1M merchants across the globe.

The Evolution of Kit: Automating Marketing Using Machine Learning

I’ll talk about the engineering decision my team made to transform Kit from a rule based system to an artificially-intelligent assistant.

Creating Native Components That Accept React Native Subviews

This guide will show you how to make a native component that provides bottom sheet functionality to React Native and renders React views inside.

Your Circuit Breaker is Misconfigured

Learn to predict how your application will behave in times of failure and how to configure every parameter for your Semian circuit breaker.

Great Code Reviews—The Superpower Your Team Needs

In this article, I have distilled a series of very practical techniques we use at Shopify to ship valuable code that can stand the test of time.

Bug Bounty Year in Review 2019

We’re reflecting on our Bug Bounty program for 2019 and introducing new program changes for 2020.

React Native is the Future of Mobile at Shopify

After years of native mobile development, we’ve decided to build all of our new mobile apps using React Native. As I’ll explain, that decision doesn’t come lightly.

Scaling Mobile Development by Treating Apps as Services

Scaling development without slowing down the delivery speed of new features is a problem that companies face when they grow. Speed can be achieved through better tooling, but the bigger the teams and projects, the more tooling they need. When projects and teams use different tools to solve similar problems, it gets har

How to Implement a Secure Central Authentication Service in Six Steps

We created a centralized authentication service, called Identity, by building upon functionality from the OpenID Connect (OIDC) specification.

How Shopify Manages API Versioning and Breaking Changes

This year, we released the 1st official version of our API. Our API versioning scheme allows us to providing apps with stable behavior and predictable timelines.

Successfully Merging the Work of 1000+ Developers

Collaboration with a large team is challenging, and even more so if it’s on a single codebase, like the Shopify monolith. Shopify changes 40 times a day. We follow a trunk-based development workflow and merge around 400 commits to master daily. There are three rules that govern how we deploy safely, but they were hard

Four Steps to Creating Effective Game Day Tests

Four steps to creating effective Game Day tests. Game Day testing is deliberately triggering failure modes within production systems, and analyzing whether the systems handle these problems in the ways we expect.

Sam Saffron AMA: Performance and Monitoring with Ruby

Sam Saffron AMA: The co-founder of Discourse and the creator of the mini_profiler, memory_profiler, mini_mime and mini_racer gems answers our questions about Ruby performance and monitoring.

Make Great Decisions Quickly with TOMASP

We're naturally bad at making complex decisions. I’ll show you how you can consistently make great decisions using a simple framework called TOMASP.

Five Common Data Stores and When to Use Them

An important part of any technical design is choosing where to store your data. This post describes five common data stores and their attributes.

How to Write Fast Code in Ruby on Rails

Part of Shopify’s success with Ruby on Rails is an emphasis on writing fast code. But, how do you really write fast code? Let’s talk about a few ways to start writing faster code in Active Record, Rails, and Ruby.

How Shopify Manages Petabyte Scale MySQL Backup and Restore

At Shopify, we need a robust backup and restore solution for our MySQL servers. We used disk-based snapshots to reduced our RTO to under 30 minutes.

How Shopify Scales Up Its Development Teams

Have you clicked on this article because you’re interested in how Shopify scales its development teams and the lessons we’re learning along the way? Well, cool, you’ve come to the right place. But first, a question. Are you sure you need to scale your team?

Want to Improve UI Performance? Start by Understanding Your User

Ask yourself what the user is trying to accomplish when they visit a page. Once you know this, it helps identify the actions to improve UI performance.

Building Resilient GraphQL APIs Using Idempotency

Shopify’s new Payment Service, which centralizes payment processing for certain payment methods, uses API idempotency to prevent payment issues.

Living on the Edge of Rails

Shopify's Core monolith has been running the latest revision of the Rails framework in production since Feb 2019.

Pagination with Relative Cursors

Using incremental page numbers for pagination scales poorly, so to solve this, Shopify uses relative cursor pagination to deliver faster queries.

Lessons from Leading a Remote Engineering Team

Shopify's Director of Production Engineering talks about the benefits of remote teams, building a remote team, and setting your people up to succeed.

Componentizing Shopify’s Tax Engine

How Shopify built for the long term by moving our tax engine to a componentized architecture within Shopify’s codebase with zero downtime.

Implementing Android POS Receipt Printing on Shopify

How Shopify developed a pragmatic, adaptive and highly testable solution to Android POS receipt printing for Shopify POS.

Mobile Release Engineering at Scale with Shipit Mobile

Shopify minimized the challenges of mobile release engineering at scale with Shipit Mobile: a platform to create, view, and manage app releases.

Pair Programming Explained

The Shopify Pair Programming primer. Learn all about pair programming, it's benefits , it's misconceptions and the techniques you can use to improve.

A New Kubectl Plugin for Kubernetes Ingress Controller ingress-nginx

Shopify introduces a new kubectl plugin for Kubernetes Ingress Controller ingress-nginx making it easier to upgrade and debug.

Building Shopify’s Application Security Program

How Shopify built an application security program that encourages safety at high speed, removes complexities, and fosters an environment for creative problem solving so that everyone can focus on delivering amazing products to our merchants.

One Million Dollars in Bug Bounties

Today, we’re excited to announce that we’ve awarded over $1M USD in bounties through our bounty programs. At Shopify, bounty programs complement our security strategy and allow us to leverage a community of researchers who help secure our platform. They each bring their perspective and specialties and are can evaluate