Scaling Zapier to Automate Billions of Tasks

20,534
Zapier
Zapier is for busy people who know their time is better spent selling, marketing, or coding. Instead of wasting valuable time coming up with complicated systems - you can use Zapier to automate the web services you and your team are already using on a daily basis.

Editor's note: By Bryan Helmig, ‎Co-founder & CTO at Zapier



Zapier is a web service that automates data flow between over 500 web apps, including MailChimp, Salesforce, GitHub, Trello and many more.

Imagine building a workflow (or a "Zap" as we call it) that triggers when a user fills out your Typeform form, then automatically creates an event on your Google Calendar, sends a Slack notification and finishes up by adding a row to a Google Sheets spreadsheet. That's Zapier. Building Zaps like this is very easy, even for non-technical users, and is infinitely customizable.

As CTO and co-founder, I built much of the original core system, and today lead the engineering team. I'd like to take you on a journey through our stack, how we built it and how we're still improving it today!

The Teams Behind the Curtains

It takes a lot to make Zapier tick, so we have four distinct teams in engineering:

  • The frontend team, which works on the very powerful workflow editor.
  • The full stack team, which is cross-functional but focuses on the workflow engine.
  • The devops team, which keeps the engine humming.
  • The platform team, which helps with QA, and onboards partners to our developer platform.

All told, this involves about 15 engineers (and is growing!).

The Architecture

Our stack isn't going to win any novelty awards — we're using some pretty standard (but awesome) tools to power Zapier. More interesting are the ways we're using them to solve our particular brand of problems, but let's get the basics out of the way:

The Frontend

We're smack in the middle of transitioning from Backbone to React. We use Babel for ES6 and Webpack + Gulp to compile the frontend. We rely heavily on CodeMirror to do some of the more complex input widgets we need, and use React + Redux to do much of the heavy lifting for the uber-powerful Zap editor.

The Backend

Python powers a large majority of our backend. Django is the framework of choice for the HTTP side of things. Celery is a massive part of our distributed workflow engine. Most of the routine API work is done with the epic requests library (with a bunch of custom adapters and abstractions).

The Data

MySQL is our primary relational data store — you'll find our users, Zaps and more inside MySQL. Memcached and McRouter make an appearance as the ubiquitous caching layer. Other types of data go in other data stores that make more sense. For example, in-flight task counts for billing and throttling find themselves in Redis, and Elasticsearch stores historical activity feeds for Zaps. For data analysis we love us some AWS Redshift.

The Platform

Most of our platform is nestled inside our fairly monolithic core Python codebase, but there are lots of interesting offshoots that offer specialized functionality. The best example might be how we utilize AWS Lambda to run partner/user provided code to customize app behavior and API communication.

The Infrastructure

Since Zapier runs on AWS, we have quite a bit of power at our fingertips. EC2 and VPC are the centerpiece there, though we do use RDS where possible along with copious numbers of autoscaling groups to ensure the pool of servers are in tip-top shape. Jenkins, Terraform, Puppet and Ansible are all daily tools for the devops team. For monitoring, we can't rave enough about Statsd, Graylog, and Sentry (they're so good).

Some Rough Numbers

These numbers represent a rough minimum to help the reader guage the general size and dimensions of Zapier's architecture:

  • over ~8m tasks automated daily
  • over ~60m API calls daily
  • over ~10m inbound webhooks daily
  • ~12 c3.2xlarge boxes running HTTP behind ELB
  • ~100 m3.2xlarge background workers running Celery (split amongst polling, hooks, email, misc)
  • ~3 m3.medium RabbitMQ nodes in a cluster
  • ~4 r3.2xlarge Redis instances - one hot, two failover, one backup/imaging
  • ~12 m2.xlarge Memcached instances behind ~6 c3.xlarge McRouter instances
  • ~10 m3.xlarge ElasticSearch instances behind ~3 m3.xlarge no-data ElasticSearch instances
  • ~6 m3.xlarge ElasticSearch instances behind ~1 c3.2xlarge Graylog server
  • ~10 dc1.large Redshift nodes in a cluster
  • 1 master db.m2.2xlarge RDS MySQL instance w/ ~2 more replicas for both production reads and analysis
  • a handful of supporting RDS MySQL instance (more details below)
  • ...and tons of microservices and miscellaneous specialty services

Improving the Architecture

While the broad strokes of the architecture remain the same - we've only performed a few massive migrations - a lot of work has been done to grow the product in two categories:

  1. Supporting big new product features
  2. Scaling the application for more users

Let's dive into some examples of each, with as many nitty-gritty details as possible without getting bogged down!

Big Features like Multi-Step Zaps

When we started Zapier (fun fact: it was first called Snapier!) at a Startup weekend, we laid out the basic architecture in less than 54 hours (fueled by lots of coffee and more than a few beers). Overall, it was decent. We kept the design really, really simple, which was the right call at the time.

Except where it was too simple. Specifically, we made Zaps two-stepped: a trigger paired with an action, full stop.

It didn't take long for us to realize the missed opportunity, but the transition was going to be pretty complex. We had to implement a directed rooted tree with support for an arbitrary numbers of steps (nodes), but maintain 1-to-1 support for existing Zaps (of which there were already hundreds of thousands). And we had to do that while preserving support for hundreds of independent partner APIs.

Starting at the data model, we built a very simple directed rooted tree implementation in MySQL. Just imagine a table where every row has a self-referencing parent_id foreign key, plus an extra root_id foreign key to simplify queries, and you pretty much got it. We discussed switching to a proper graph database (like neo4j) but decided against it because the sorts of queries we make are simple and over isolated graphs of smaller sizes (roughly ~2-50 nodes).

A key aspect to make this work is inter-step independence. Every step has to consume some data (which folder to read from or which list ID to add to, for example), do some API magic, and return some data (say, the new file created or the new card added to a list), but otherwise be ignorant of its placement in the workflow. Each independent step is as dumb as a rock.

In the middle exists the omniscient workflow engine which coordinates independent Celery tasks by stringing together steps as tasks — one step feeding into the next as defined by the Zap's directed rooted tree. This omniscient engine also houses all the other goodies like error & retry handling, reporting, logging, throttling and more.

Even after we nailed the backend support, we had another huge problem: how do you build a UI for this thing?

First, you make sure you have some amazing designers and Javascript engineers on the team. Then you wrestle with nested Backbone views for a while before moving onto React. :-) In all seriousness: React is a godsend for the sorts of complex interfaces we are building.

One of the unique things about React is the performance characteristics are developer friendly, but only as long as you have your data figured out. If you aren't using immutable data structures, you should use some structural sharing library to do all mutations, along with deep Object.freeze() in development to catch spots where you attempt mutation directly.

There are tons of challenges in building such a complex UI, much of it around testing and feedback, but a huge amount of time was spent just getting the long-tailed data from different APIs to fit elegantly into the same places. Just about every weird shape of data has to be accounted for.

Finally, we were tasked with getting the new editor in front of users for alpha and beta testing. To do this we shipped both versions of the editor simultaneously and used feature switches to opt users in. We did months and months of testing and tweaking before we were happy with the result - you can check out the Multi-Step Zap launch page to get an idea of where it ended up.



Scaling the Application

It would all be for nought if the service isn't up and running reliably. As such, much of our attention is focused jointly on application design to support horizontal scalability and redundant infrastructure to ensure availability.

Some of the wiser decisions we've made so far is to double down on tech we're comfortable with and spin out isolated functionality when we hit a bottleneck. The key is to reuse the exact same solution and move it to another box where it is free to roam fresh pastures of CPU and RAM.

For example, over the last year or so we noticed session data had eaten up a ton of our primary database's IO and storage. Since session data is effectively a key/value arrangement with softer consistency requirements, we heavily debated options like Cassandra or Riak (or even Redis!), but ultimately decided to stand up a dedicated MySQL instance with a single sessions table.

Our instinct as engineers was to find the tool best suited to the job, but as a practical matter, the job didn't warrant additional operational complexity. We know MySQL, it can do simple key/value storage and our application already supports it. Talk about a no-brainer.

Further, careful design of the application can make horizontal scaling equally simple. Long running background tasks (like our Multi-Step Zaps) aren't bound by strict consistency requirements due to their light write pattern, so it is trivial (and safe!) to use MySQL read-only replicas as the primary touch point. Even if we occasionally get horrible replica lag measured in the minutes, 99.9% of Zaps aren't changing — and certainly aren't changing soon — so they continue to hum along.

Another good practice is to assume the worst. Design for failure from the beginning. While usually this is easier said than done, nowadays it is actually surprisingly easy to do. For starters: use auto scaling groups with auto-replacement. A common misconception is that ASGs are only for scaling to accommodate fluctuating load. Wrong! The ASG + ELB combo can be your backbone of reliability, one that enables you to randomly kill instances without worry since they get replaced in quick order.

Somehow we keep re-learning that the simpler the system is, the better you'll sleep.

The Day-To-Day

Locally, our engineers enjoy a fully functioning environment courtesy of Docker. docker-machine and docker-compose together stand up proper versions of MySQL, Memcached, Redis, Elasticsearch as well as all the web and background workers. We generally recommend running npm and even runserver locally, as file watching is kind of broken with VirtualBox.

The canonical GitHub "pull request model" drives most of our projects that are engineering focused, where day-to-day work is logged and final code review happens before merging. Hackpad houses the majority of our documentation, including copious onboarding documentation.

A big thing at Zapier is all hands support. Every four or five weeks, every engineer spends a full week in support helping debug and fix difficult customer issues. This is hugely important to us as it provides a baseline for understanding customers' pain (plus, you might have to deal with the bug you shipped!).

For CI and deployment, we use Jenkins to run tests on every commit in every PR as well as to provide a "one-click deploy" that anyone at the company can press. It's not uncommon for a new engineer to click the deploy button the first week on the job!

We have a full staging environment in a standalone VPC, as well as a handful of standalone web boxes perfect for testing long lived pull requests. Canary deploys to production are common — complete with full logs of any errors courtesy of Graylog.

You Can Zapier, Too!

Developers can use Zapier to do some pretty awesome stuff.

In addition to Multi-Step Zaps, we've also launched the ability to write Python and Javascript as Code steps in your workflow. No need to host and run scripts yourself — we take care of all of that. We also provide bindings to call out to the web (requests and fetch) and even store a bit of state between runs!

Our users are employing Code steps to build Slack bots (and games!), to replace one-off scripts and lots more. I personally use Code steps to write bots and tools to track code & bug metrics to a spreadsheet, transform oddly formatted data and replace a ton of crontabs.

Or, if you have an API that you want non-developers to be able to consume, we have a pretty epic Developer Platform, too. Simply define your triggers, searches and actions, and any user can mix your API into their workflows and integrate your app with over 500 apps like GitHub, Salesforce, Google Docs, and more.

And, we are often hiring, so keep an eye on our jobs page if you'd like to help us help people work faster and automate their most tedious tasks!


Zapier
Zapier is for busy people who know their time is better spent selling, marketing, or coding. Instead of wasting valuable time coming up with complicated systems - you can use Zapier to automate the web services you and your team are already using on a daily basis.
Tools mentioned in article
Open jobs at Zapier
Data Warehouse Engineer

DATA WAREHOUSE ENGINEER-REMOTE

Hi there!

We're looking for an experienced Data Warehouse Engineer to join the Data Insights team at Zapier. Interested in helping grow a product that helps the world automate their work so they can get back to living? Then read on…

We know applying for and taking on a new job at any company requires a leap of faith. We want you to feel comfortable and excited to apply at Zapier. To help share a bit more about life at Zapier, here are a few resources in addition to the job description that can give you an inside look at what life is like at Zapier. We hope you'll take the leap and apply.

Zapier is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse workforce.

 

About You

You are a skilled written communicator. Zapier is a 100% remote team and writing is our primary means of communication. You can communicate about technical topics without unnecessary jargon. You translate unfamiliar data modeling concepts into approachable ones to team mates and stakeholders with less experience working with data.

You have experience in SQL. You’ve used written complex SQL queries that join across data from multiple systems, matching them up even when there was not a straightforward way to join the tables. You've designed tables with an eye towards ease of use, high performance and with focus on the consumer of the data. You've documented schemas and created data dictionaries.

You appreciate our team’s values of eagerness to collaborate with diverse teammates from any function of the organization or with any level of data knowledge, iterating over your deliverables, and being curious.

You strive to learn about business and product goals so you can design data models that empower analysis and reporting. You value partnering with your Data Science colleagues to ensure your efforts align with data objectives.

You have experience with data modeling: you understand data warehouse modeling and have a nuanced appreciation for when and where traditional approaches work and when they don’t. You have experience developing ETL pipelines to support business data needs.

You understand that the perfect is the enemy of the good, and default to action by shipping MVP code and iterating as needed to get towards better solutions.

 

Things You’ll Do

  • Develop ETL/ELT pipelines to ingest and transform data from upstream databases and APIs into a data warehouse. Our stack includes AWS Redshift, Matillion ETL, Astronomer/Airflow, Kafka, NiFi and custom Python. 
  • Data modeling, including designing and expanding slowly changing dimension and fact tables.
  • Build reports and dashboards in data visualization tools. We use Looker.
  • Work with product teams and help them surface interesting data back in the product for our users and partners to see.
  • Influence design of our product’s data model.
  • As a part of Zapier's all-hands philosophy, help customers via support (or other mechanisms) to ensure they have the best experience possible.

About Data Insights at Zapier

Data Insights exists to champion evidence-based decision making at Zapier through scalable, discoverable and impactful data insights. We are a collaborative team of Data Scientists and Data Warehouse Engineers influencing Zapier to deliver the best possible experience to users, customers and partners. 

On the Data Science side, our responsibilities include proactive and request-driven analysis; ideation, design and analysis of experiments; quantitative research (i.e. data mining & machine learning); metrics definition, reporting and goal setting for teams; insights discovery (i.e. hosting a searchable repository); instrumentation (i.e. event design); and administering data literacy and data self-serve platforms (we use Looker) to Zapier.

On the Data Warehouse Engineering side, our responsibilities include building scalable data models and ELT / ETL pipelines, insights tooling (i.e. developing tools to accelerate or improve the quality of analysis / data structures); work with our stakeholders to design data that feeds back into Zapier products; and influencing the data architecture of product teams.

We collaborate with Data Foundations, which builds and maintains a platform of capabilities for tooling, frameworks and infrastructure.

Curious about our stack? We use AWS Redshift (and the related AWS products AWS Glue, Redshift Spectrum, AWS S3), Looker, Airflow, Matillion ETL, Kafka, Python, NiFi and dbt.

 

The Whole Package

Our flexible, distributed environment lets us work with the best people from around the world. Zapiens live in 40+ countries, including the United Kingdom, Thailand, India, Nigeria, Taiwan, Guatemala, New Zealand, Australia, and more!

Zapier offers:

  • Competitive salary and bonus program
  • Equity for All: Stock options (or equivalent) for every Zapien
  • Healthcare + dental + vision coverage*
  • Retirement plan with 4% company match*
  • $2,000 annual learning stipend for use on courses, conferences, and more—your choice
  • Two annual all-company retreats
  • 14 weeks paid leave for new parents of biological or adopted children
  • Customized Zapiversary rewards on your 1, 3, 5, 7 and 10 year work anniversaries
  • Leading-edge equipment. We set you up with an Apple laptop and provide an additional budget for you to choose other home office accessories and software you may need.
  • Time to renew. We encourage Zapiens to take at least 2 weeks off each year. Most of us take 4-5 weeks, in addition to locally recognized holidays

*While we take care of Zapiens around the world the best we can, healthcare and retirement plans are currently available specifically in the UK, Canada, New Zealand, Australia, and United States. 

  

How to Apply

We have a non-standard application process designed to promote inclusion and equity. We first ask a few questions in our application form that would typically be asked at the start of an initial interview. This helps speed up the process and lets us get to know you a bit better right out of the gate. Please be sure to answer each question; the resume and CV fields are optional. 

After you apply, you are going to hear back from us—even if we don’t see an immediate fit with our team. In fact, throughout the process, we strive to make sure you never go more than seven days without hearing from us.

Zapier is an equal opportunity employer. We're excited to work with talented and empathetic people, and do not discriminate based on race, color, sex, gender identity or expression, sexual orientation, religion, national origin, physical or mental disability, military or veteran status, genetic information, pregnancy, age, or any other status protected by local law. Our code of conduct provides a beacon for the kind of company we strive to be, and we celebrate our differences because those differences are what allow us to make a product that serves a global user base.

Zapier is is committed to inclusion. As part of this commitment, Zapier will ensure that people with disabilities are provided reasonable accommodations. If reasonable accommodations are needed to participate in the job application or interview process, please contact jobs@zapier.com.

Director, Engineering - Growth Zone
NAMER

Director, Engineering - North America

 

Hi there!

We're looking for an Engineering Director to join the Engineering team at Zapier. Zapier’s on a mission to make everyone more productive at work. Over 3 million professionals already use Zapier to save more time, but there are millions more to reach.

In this role, you will be responsible for leading the engineering organization developing and owning the systems that drive Zapier’s growth. You will work closely with your managers, peers, and executives to staff teams and execute on a multifaceted product and tooling strategy. 

If you’re interested in advancing your career at a fast-growing, profitable, impact-driven company, then read on…

 

Our Commitment to Applicants

Culture and Values at Zapier

Zapier Guide to Remote Work

Zapier Code of Conduct

Diversity and Inclusivity at Zapier

About You

  • You’ve managed and scaled engineering organizations. You’re a skilled and experienced engineering leader who has built teams working on SaaS-based products with modern tech stacks. You’ve led teams of managers, managers of managers, and technical leaders and helped each team be successful.
  • You understand product development and growth. You are experienced in understanding the complexities and nuances of delivering and growing products, including the business and engineering implications of product decisions. You’ve driven the complex and challenging tradeoffs between speed, cost, and functionality when building products in one or more fast-moving companies. You have deep experience in managing complex projects with a variety of agile development and industry best practices at your disposal.
  • You are customer-focused. You have a track record of deeply understanding customers and markets. You have a passion for building products that delight internal and external customers and make their jobs easier and more productive.
  • You can develop and sell a strategy to your team. You have a good sense of vision and strategy, you’re able to communicate it to your teams and stakeholders, and you can translate that strategy into execution. You partner closely with product management, design, marketing, and the support organizations to drive and achieve business outcomes.
  • You are a skilled mentor, coach, and engineering leader. You have a passion and a track record of mentoring engineers and leaders while at the same time growing your own skills. You've introduced and broadened engineering practices and enhanced engineering cultures. You’ve led engineering teams through complex technical and architectural decisions by providing strategic guidance.
  • You're a skilled written communicator. Zapier is a 100% remote team and writing is our primary means of communication. You'll use written summaries and reports to communicate vision, strategy, plans, findings, and results to the engineering and full Zapier team.
  • You're a champion for doing things efficiently. At Zapier, the work you do will have a disproportionate impact on the business. We believe in systems and processes that let us scale our impact to be larger than ourselves.

Things You’ll Do

  • Hire and retain engineers and engineering leaders for your teams.
  • Coach, mentor, and grow senior engineering leadership.
  • Collaborate closely with your product and design leadership peers to set the product direction and vision to drive the growth of Zapier’s product, users, and revenue.
  • Own the execution of Zapier’s growth vision.
  • Collaborate with engineering and product leaders to influence changes to the Zapier platform, primitives, and product to support the growth of Zapier’s product, users, and revenue.
  • Ensure engineering excellence throughout your teams including following agile development practices,  engineering and architecture best practices, and the Zapier system design principles.
  • Communicate regularly and effectively throughout our distributed organization.
  • As a part of our All Hands Support initiative, help customers have the best possible experience with Zapier

 

The Whole Package

Location: North America

Our flexible, distributed environment lets us work with the best people from around the world. Zapiens live in 40+ countries, including the United Kingdom, Thailand, India, Nigeria, Taiwan, Guatemala, New Zealand, Australia, and more!

Zapier offers:

  • Competitive salary and profit-sharing program
  • Equity for All: Stock options (or equivalent) for every Zapien
  • Healthcare + dental + vision coverage*
  • Retirement plan with 4% company match*
  • $2,000 annual learning stipend for use on courses, conferences, and more—your choice
  • Two annual all-company retreats
  • 14 weeks paid leave for new parents of biological or adopted children
  • Customized Zapiversary rewards on your 1, 3, 5, 7 and 10 year work anniversaries
  • Leading-edge equipment. We set you up with an Apple laptop and provide an additional budget for you to choose other home office accessories and software you may need.
  • Time to renew. We encourage Zapiens to take at least 2 weeks off each year. Most of us take 4-5 weeks, in addition to locally recognized holidays.
  • Opportunity to work with Zapier’s amazing partners network 

*While we take care of Zapiens around the world the best we can, healthcare and retirement plans are currently available specifically in the UK, Canada, New Zealand, Australia, and United States.

How to Apply

We have a non-standard application process designed to promote inclusion and equity. We first ask a few questions in our application form that would typically be asked at the start of an initial interview. This helps speed up the process and lets us get to know you a bit better right out of the gate. Please be sure to answer each question; the resume and CV fields are optional.

After you apply, you are going to hear back from us—even if we don’t see an immediate fit with our team. In fact, throughout the process, we strive to make sure you never go more than seven days without hearing from us.

Zapier is an equal opportunity employer. We're excited to work with talented and empathetic people, and do not discriminate based on race, color, sex, gender identity or expression, sexual orientation, religion, national origin, physical or mental disability, military or veteran status, genetic information, pregnancy, age, or any other status protected by local law. Our code of conduct provides a beacon for the kind of company we strive to be, and we celebrate our differences because those differences are what allow us to make a product that serves a global user base.

Zapier is is committed to inclusion. As part of this commitment, Zapier will ensure that people with disabilities are provided reasonable accommodations. If reasonable accommodations are needed to participate in the job application or interview process, please contact jobs@zapier.com.

#LI-REMOTE 

Engineer, Backend

As Zapier continues to scale our mission to democratize automation, we’re hiring Backend Engineers across the organization. Our organization works in the EPD model, meaning that Engineering, Product, and Design work together in Product Engineering squads to build and ship features & products our customers love. 

Zapier is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse workforce. 

About You 

You love building. You have at least three years of experience as a Software Engineer building distributed, scaleable web-applications. You're familiar with Python and backend frameworks such as Django. You enjoy a balance of application development and reliability tuning. Zapier is built on Python, Django, React, Node.js, and AWS. We use modern tools, which means you’ll have the opportunity to work with software like Memcached, RabbitMQ, Docker, GitLab and Redis. 

You excel working on a team. You enjoy being part of a team that solves problems together, collaborating on ideas and working to meet team goals. You like pairing up with teammates when someone gets blocked. You give and accept feedback professionally and don’t let ego get in the way of delivering the best solution for the customer. You can summarize technical concepts effectively as you work with people from disciplines outside of engineering.

You embrace service ownership. You appreciate the speed and autonomy a team has when it owns its services. You also know ownership comes with the responsibility to monitor your team’s services and make sure they are reliable and performant.

You love shipping to customers. Nothing we build matters if it isn’t meeting customers’ needs. You’ll be on a team that collaborates with internal customers’ to provide interfaces to our vast collection of integrations. You’ll also learn to empathize with Zapier’s end users and add capabilities to the Platform that enable Zapier to offer new products. You'll have the opportunity to ship code daily that will be used by millions of people.

You love doing things efficiently. At Zapier, we believe in “Don't be a Robot, Build the Robot.” We build systems and processes that let us scale our impact to be larger than ourselves.

You love learning. Engineering is an ever-evolving world. You enjoy exploring areas that you might not have experience with yet. You have a balanced approach to adopting new tech vs using well-understood patterns.

Things You Might Do

  • Directly impact how our customers can be more productive by building and supporting the foundational building blocks of Zapier.
  • Redesign and build services that execute API calls to thousands of 3rd party APIs. REST, GraphQL, XML-RPC; we handle them all!
  • Work with Amazon services like Lambda and MSK to build scalable solutions that process millions of requests.
  • Operate and scale the systems you and your team build.
  • Ship code to millions of users every week.
  • Build and scale web products that directly impact our customers or partners.
  • Create tooling that enables automation that leverage other microservices in our architecture. 
  • Develop solutions based on containerized microservices in a cloud based infrastructure
  • Monitor reliability and performance through dashboards and tools to ensure availability
  • Dig into bugs and feature requests 
  • Work and collaborate with other engineering and product teams  to help them deliver value to customers in new and exciting ways
  • Travel of 5% - 10% for company retreats that rotate to various cities throughout North America

About Zapier

Zapier helps people across the world automate the boring and tedious parts of their job. We do that by helping everyone connect the web applications they already use and love.

We believe that there are jobs a computer is best at doing and that there are jobs a human is best at doing. We want to empower businesses to create processes and systems that let computers do what they are best at doing and let humans do what they are best at doing.

We believe that with the right tools, you can have a big impact with less hassle.

We believe in small teams. Small teams are fast and nimble. Small teams mean less bureaucracy and less management and more getting things done.

We believe in a safe, welcoming, and inclusive environment. All teammates at Zapier agree to a code of conduct.

The Whole Package

Location: UTC-8 (West Coast Americas) to UTC-3

Compensation:

  • Competitive salary (we don't use remote as an excuse to pay less)
  • Great healthcare + dental + vision coverage*
  • Retirement plan with 4% company match*
  • Profit-sharing
  • 2 annual company retreats to awesome places
  • 14 weeks paid leave for new parents of biological or adopted children
  • Pick your own equipment. We'll set you up with whatever Apple laptop + monitor combo you want plus any software you need.
  • Unlimited vacation policy. Plus we require you to take at least 2 weeks off each year. We see most employees take 4-5 weeks off per year. This isn't a vague policy where unlimited vacation means no vacation.
  • Travel of 5% - 10% for company retreats which rotate to various cities throughout North America
  • Work with awesome companies around the world. We partner with great software companies all over the world and you'll constantly get to interact with people from these great companies

*While we take care of our international folks as best we can, currently, healthcare and retirement plans are only available to UK, Canada, and US-based employees.

How to Apply

We have a non-standard application process. To jump-start the process we ask a few questions we normally would ask at the start of an interview. This helps speed up the process and lets us get to know you a bit better right out of the gate. Please make sure to answer each question.


After you apply, you are going to hear back from us, even if we don't seem like a good fit. In fact, throughout the process, we strive to make sure you never go more than seven days without hearing from us.

Zapier is an equal opportunity employer. We're excited to work with talented and empathetic people no matter their race, color, gender, sexual orientation, religion, national origin, physical or mental disability, or age. Our code of conduct provides a beacon for the kind of company we strive to be, and we celebrate our differences because those differences are what allow us to make a product that serves a global user base.

Engineer, Database Reliability
- Americas

Americas

Hi there!

We're seeking a Database Reliability Engineer (DBRE) to join our team in the Americas! Zapier is on a mission to make everyone more productive at work through the power of automation. Over 3 million professionals already use Zapier to save more time, but there are millions more to reach.  

As we continue to scale our product and grow our team, we're looking for experienced Database-focused Engineers to be stewards of Zapier's data services and technologies. We need Engineers who can help architect our overall data infrastructure landscape but aren't afraid to get into the weeds to solve gnarly operational problems. You'll help drive our best practices around automation, performance, reliability, and disaster recovery.

When bad things happen, you'll have the support of your team to solve problems, unearth contributing causes, learn from failures, and build a more robust and resilient system for our customers. We look for the solutions that automate the problem away, not ones requiring manual effort.

If you’re interested in advancing your career at a fast-growing, profitable, impact-driven company, then read on…

Our Commitment to Applicants

Culture and Values at Zapier

Zapier Guide to Remote Work

Zapier Code of Conduct

Diversity and Inclusivity at Zapier

About You

You're an experienced database technologist. You have at least 7 years of experience in systems administration, systems engineering, or software development, with at least 4 years of experience building and maintaining data services.

You know the cloud. You've designed and maintained highly available, cloud-based database systems in AWS or another cloud offering. We use tools like Postgres, MySQL, and Elastic.

You can code. You're comfortable automating solutions and building tools that eliminate repetitive tasks. You're not afraid to delve into application code to fix a bug or to launch a backend service to address a gap in our data management strategy.

You can solve complex systems challenges. You take ownership of difficult problems, understand how to improve performance, and help uncover opportunities for improvement. You've worked on issues where “just throw more hardware at it” isn't enough for the system to scale.

You're a great communicator. Not only do you know how to share your knowledge with the team and document things well so they can be consumed asynchronous (we do this a lot as a remote company), but you know how to communicate effectively with software and support teams.

You value our values. At Zapier, our values are at the heart of how we work together and how we think about our customers. In our remote setting, they help develop trust and ensure we work together and collaborate to democratize automation. You see how these values can empower meaningful work, you thrive in a collaborative setting, you are eager to continue growing, and you're excited to be part of the team.

Things You’ll Do

  • Identifying and implementing strategies to improve performance and resilience. Our application is continually growing and we need your help to keep scaling.

  • Working with development teams to make architectural decisions about new services and new technologies.
  • Designing solutions for storing our customers’ data across multiple systems, including RDS (MySQL & PostgreSQL), Elasticsearch, RabbitMQ, Memcache, Redis, and more.
  • Optimizing AWS resources as code using Terraform to ensure high performance and resiliency.
  • Monitoring use of our data layer and advising teams on how to use storage in a healthy and scalable way.
  • Acting as a point of escalation for data-layer problems during production incidents.
  • And more!

Things We've Done Recently

  • Performed large-scale migrations of ElasticSearch and Redis clusters
  • Deploying ProxySQL for pooling connections against MySQL databases
  • Rebuilt each of our RDS databases and RabbitMQ instances to provide greater security

The Whole Package

Our flexible, distributed environment lets us work with the best people from around the world. Zapiens live in 40+ countries, including the United Kingdom, Thailand, India, Nigeria, Taiwan, Guatemala, New Zealand, Australia, and more!

  • Competitive salary and profit-sharing program
  • Equity for All: Stock options (or equivalent) for every Zapien
  • Healthcare + dental + vision coverage*
  • Retirement plan with 4% company match*
  • $2,000 annual learning stipend for use on courses, conferences, and more—your choice
  • Two annual all-company retreats
  • 14 weeks paid leave for new parents of biological or adopted children
  • Customized Zapiversary rewards on your 1, 3, 5, 7 and 10 year work anniversaries
  • Leading-edge equipment. We set you up with an Apple laptop and provide an additional budget for you to choose other home office accessories and software you may need.
  • Time to renew. We encourage Zapiens to take at least 2 weeks off each year. Most of us take 4-5 weeks, in addition to locally recognized holidays.
  • Opportunity to work with Zapier’s amazing partners network
 

*While we take care of Zapiens around the world the best we can, healthcare and retirement plans are currently available specifically in the UK, Canada, New Zealand, Australia, and United States.

How to Apply

 
We have a non-standard application process designed to promote inclusion and equity. We first ask a few questions in our application form that would typically be asked at the start of an initial interview. This helps speed up the process and lets us get to know you a bit better right out of the gate. Please be sure to answer each question; the resume and CV fields are optional.
 
After you apply, you are going to hear back from us—even if we don’t see an immediate fit with our team. In fact, throughout the process, we strive to make sure you never go more than seven days without hearing from us.
 
Zapier is an equal opportunity employer. We're excited to work with talented and empathetic people, and do not discriminate based on race, color, sex, gender identity or expression, sexual orientation, religion, national origin, physical or mental disability, military or veteran status, genetic information, pregnancy, age, or any other status protected by local law. Our code of conduct provides a beacon for the kind of company we strive to be, and we celebrate our differences because those differences are what allow us to make a product that serves a global user base.
 
Zapier is is committed to inclusion. As part of this commitment, Zapier will ensure that people with disabilities are provided reasonable accommodations. If reasonable accommodations are needed to participate in the job application or interview process, please contact jobs@zapier.com.
#LI-Remote
 
Verified by
Co-founder
You may also like