How CircleCI Processes 4.5 Million Builds Per Month

41,887
CircleCI
CircleCI’s continuous integration and delivery platform helps software teams rapidly release code with confidence by automating the build, test, and deploy process. CircleCI offers a modern software development platform that lets teams ramp quickly, scale easily, and build confidently every day.

By Rob Zuber, CTO at CircleCI.



CircleCI Workflow


Background

CircleCI is a platform for continuous integration and delivery. Thousands of engineers trust us to run tests and deploy their code, so they can focus on building great software. That trust rests on a solid stack of software that we use to keep people shipping and delivering value to their users.

As CTO of Engineering at CircleCI, I help make the big technical decisions and keep our teams happy and out of trouble. Before this, I was CTO of Copious, where I learned a lot of important lessons about tech in service of building a consumer marketplace. I like snowboarding, Funkadelic, and viscous cappuccino.

The Teams

Engineers are people. People work better in small groups. So we’ve divided our team into several functional units, inspired by Spotify’s pods. We’re much smaller, so we’ve adapted their ideas to meet our needs, while maintaining the core principle that each team has the resources they need to implement a feature across the stack.

But we think of these teams as more of a guideline than actual rules, so folks are free to move around if it means they’ll be more engaged in the work. Flexibility is a key value at CircleCI: it has to be, with the majority of our engineers working remotely across multiple time zones. To keep everyone on the same page, we use Zoom for videoconferencing and screensharing and update statuses in Pingboard to keep track of who’s “in the office”.

We use JIRA to create consistency in our processes across teams. This consistency lets us stay more nimble if engineers ever need or want to switch teams. We use GitHub for version control and Slack for Giphy control. In addition to chat, we use Slack-based integrations with tools like Hubot, PagerDuty, and Looker to give us central access to many day-to-day tasks.

But you didn’t come here to read about how many Slack channels we have (241), you’re here to read about...

The Stack

Languages

Most of CircleCI is written in Clojure. It’s been this way since almost the beginning. While there were some early spikes in Rails, the passion of a sole developer won out; by the time CircleCI was released to the market, it was written entirely in Clojure and has been at our platform’s core ever since.

Our frontend used to be in CoffeeScript, but when Om made a single-page ClojureScript application viable, we opted for consistency and unification. This choice wasn’t that hard to make, given how much we enjoy using Clojure. Having a lingua franca also helps reduce overhead when engineers want to move between layers of the stack.

That doesn’t mean we won’t sharpen other tools when warranted. The build agent for our recently launched 2.0 platform is written in Go, which lets us quickly inject a multi-platform static binary into environments where we can’t lean on a bunch of dependencies. We also use Go for CLI tools where static dependency compilation and fast start-up are more important than our love of Clojure.

But as we pull microservices out of our monolith, Clojure remains our weapon of choice. We’ve already got over ten microservices, and that number is growing rapidly. A major part of this velocity stems from using Clojure, which ensures developers can rapidly move between teams and projects without climbing a huge learning curve.

The Frontend

Our web app’s UI is written in ClojureScript. Specifically, we’re using the framework Om, a ClojureScript interface to Facebook’s React. This is currently in some flux, since we’re upgrading to Om Next, an Om reboot which fixes a lot of its quirks. You can read more about why we’re so excited in this deep dive by one of our engineers, Peter Jaros.


CircleCI Screenshot


The Backend

Two Pools, Both Alike in Dignity

There are two major pools of machines: the first hosts our own services — the systems that serve our site, manage jobs, send notifications, etc. These services are deployed within Docker containers orchestrated in Kubernetes. In 2012, this configuration wasn’t really an option. As functional programmers, though, we were big believers in immutable infrastructure, so we went all in on baking AMIs and rolling them on code changes.

However, rounding boot times and charges to the hour made using full VMs slow and expensive; rolling deploys in Docker with Kubernetes is much more efficient. Kubernetes’ ecosystem and toolchain made it an obvious choice for our fairly statically-defined processes: the rate of change of job types or how many we need in our internal stack is relatively low.

On the other hand, our customers’ jobs are changing constantly. It’s challenging to dynamically predict demand, what types of jobs we’ll be running, and the resource requirements of each of those jobs. We found that Nomad excelled in this area. With a fast, flexible scheduler built-in, Nomad distributes customer jobs across our second pool of machines, reserved specifically for scheduling purposes.

While we did evaluate both Kubernetes and Nomad to do All These Things, neither tool was optimized for such an all-inclusive job. And we treat Nomad’s scheduling role as more a piece of our software stack than as a part of the management or ops layer. So we use Kubernetes to manage the Nomad servers.

We’ve also recently started using Helm to make it easier to deploy new services into Kubernetes. We’ve had to build a couple small services to string the full CD process together with Helm, while also keeping Kubernetes locked down — but the results have been great. We create a chart (i.e. package) for each service. This lets us easily roll back new software and gives us an audit trail of what was installed or upgraded.

Infrastructure

For the last five years, we’ve run our infrastructure on AWS. It started simply because our architecture was simple but evolved into a necessarily complex stack of Linked Accounts, VPCs, Security Groups, and everything else AWS offers to help partition and restrict resources. We’re also running across multiple regions. Our deep investment in AWS led to increasing assumptions in our code about how the software was being managed.

When we introduced CircleCI Enterprise (our on-prem offering), we started supporting a number of different deployment models. We also started separating ourselves further from the system by packaging our code in Docker containers and using cloud-agnostic Kubernetes to manage resources and distribution.

With a much lower level of vendor lock-in, we’ve gained the flexibility to push part of our workload to Google Cloud Platform (GCP) when it suits us. We chose GCP because it’s particularly well-suited for short-lived VMs. Today, if you use our machine executor to run a job, it will run in GCP. This executor type allocates a full VM for tasks that need it.

We’ve also wrapped GCP in a VM service that preallocates machines, then tears everything down once you’re finished. Using an entire VM means you have full control over a much faster machine. We’re pretty happy with this architecture since it smooths out future forays into other platforms: we can just drop in the Go build agent and be on our merry way.

Communication with Frontend

When the frontend needs to talk to the backend, it does so via a dedicated tier of API hosts. These API hosts are also managed by Kubernetes, albeit in a separate cluster to increase isolation. Nearly all our APIs are public, which means we’re using the same interfaces available to our customers. The value of dogfooding your APIs can’t be overstated: it’s enabled us to keep the APIs clean and spot errors before our users find them.

If you’re interacting with our web application, then all of your requests are hitting the API hosts. The majority of our authentication is handled via OAuth from GitHub or Bitbucket. Once you’ve authenticated, you can also generate an API token to get programmatic access to everything we expose in the UI.

Our API hosts once accepted webhooks from GitHub and Bitbucket, but we’ve recently extracted that into its own service. Using a cleanly-separated service that dumps hooks into RabbitMQ allows us to more easily respond to a large array of operational issues. When version control system (VCS) providers are recovering from their own issues, we’ve seen massive spikes in hooks. Now we’re well equipped to deal with that.

Data! Data! Data!

Our primary datastore is MongoDB. We made this decision in CircleCI’s early days — lured like so many others by the simplicity of “schemaless” storage and rapid iteration. Having peaked at over 10TB of bloated storage in MMAP, along with painful, outage-inducing DB-level locks in Mongo 2.4, we’re happy to see progress being made in WiredTiger. Our operations have greatly improved, but we’re still suffering from a legacy of poorly-enforced schemas on a dataset too large to clean efficiently.

So we’re retreating to the structure of PostgreSQL. We’ve got a great opportunity for this migration as we build microservices with their own datastores. We’re also using Redis to cache data we’d never store permanently, as well as to rate-limit our requests to partners’ APIs (like GitHub).

When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We’re well beyond the scale where we could just dump this kind of stuff in a database. We handle any side-effects of S3’s eventual consistency model within our code to ensure that we deal with user requests correctly while writes are in process.

A Build is Born

When we process a webhook from GitHub/Bitbucket telling us that a user pushed some new code, we use the information to create a new build or workflow representation in our datastores, then queue it for processing. In order to get promoted out of this first queue, the organization needs to have enough capacity in its plan to run the build/workflow.

If you’re a customer using all your containers, no new builds or workflows are runnable until enough containers free up. When that happens, we’ll pass the definition of the work to be performed to Nomad, which is responsible for allocating hardware for the work’s duration.

Running the Build

The gritty details of processing a build are executed by the creatively named build agent. It parses configuration, executes commands, and synthesizes actions that create artifacts and test results. Most builds run in a Docker container, or set of containers, which is defined by the customer for a completely tailored build environment.


CircleCI Screenshot


The build agent streams the results of its work over gRPC to the output processor, a secure facade that understands how to write to all our internal systems. This facade approach allows our 1.0 and 2.0 platforms to coexist.

In order to get this live streaming data to your browser, we use WebSockets managed by Pusher. We also use this channel to deliver state change notifications to the browser, e.g. when a build completes. We also store small segments temporarily in Redis while we collect enough to write permanently to S3.

A Hubot Postscript

We have added very little to the CoffeeScript Hubot application – just enough to allow it to talk to our Hubot workers. The hubot workers implement our operational management functionality and expose it to Hubot so we can get chat integration for free. We’ve also tailored the authentication and authorization code of Hubot to meet the needs of roles within our team.

For larger tasks, we’ve got an internal CLI written in Go that talks to the same API as Hubot, giving access to the same functionality we have in Slack, with the addition of scripting, piping, and all of our favorite Unix tools. When the Hubot worker recognizes the CLI is in use, it logs the commands to Slack to maintain visibility of operational changes.

Analytics & Monitoring

Our primary source of monitoring and alerting is Datadog. We’ve got prebuilt dashboards for every scenario and integration with PagerDuty to manage routing any alerts. We’ve definitely scaled past the point where managing dashboards is easy, but we haven’t had time to invest in figuring out their more anomalous features. Nor the willingness to trust that it will just work for us. We capture any unhandled exceptions with Rollbar and, if we realize one will keep happening, we quickly convert the metrics to point back to Datadog, to keep Rollbar as clean as possible. We’re also using LaunchDarkly to safely deploy new and/or incomplete features behind feature flags.

We use Segment to consolidate all of our trackers, the most important of which goes to Amplitude to analyze user patterns. However, if we need a more consolidated view, we push all of our data to our own data warehouse running Postgres; this is available for rapid analytics and dashboard creation through Looker. Many engineers who want to do their own analysis use tools they’re comfortable with, which includes sed and awk but also Pandas and R.

TL; DR

One of the great things about being a CI/CD company is that we get to practice what we preach. Instead of long dry spells between releases, we push several changes per day to keep our feedback loops short and our codebase clean. We’re small enough that we can move quickly, but large enough that our teams have the resources they need.

This is our stack today. As our customers deal with more complex problems, we’ll adapt and adopt new tools to deal with emerging tech. It’s all very exciting, and we can’t wait to see what the future holds.

While we wait for the future, though, there’s no reason you should be waiting for good code. Start building on CircleCI today and ship your code faster. Or come work with us and help us ship our own code faster.

P.S. If you're already a CircleCI customer, head over to our community site and share your stack to get some free swag.


CircleCI Workflow


CircleCI
CircleCI’s continuous integration and delivery platform helps software teams rapidly release code with confidence by automating the build, test, and deploy process. CircleCI offers a modern software development platform that lets teams ramp quickly, scale easily, and build confidently every day.
Tools mentioned in article
Open jobs at CircleCI
Interested in CircleCI Career? Join O...
Japan
<p><span style="font-weight: 400;">Stay connected by joining our Talent Network!&nbsp;</span></p> <p><span style="font-weight: 400;">If we currently do not have any opportunity available that aligns with your career goals or are just curious about hearing more about working at CircleCI Japan, please feel free to submit your resume/Linkedin URL to our Talent Network! We will review all applicants on a regular basis and we will reach out to you when the timing is right or have our casual conversation.&nbsp;</span></p> <p><span style="font-weight: 400;">We look forward to staying connected with you as we will continue to grow and expand our business!</span></p> <p><span style="font-weight: 400;">弊社のキャリアネットワークへ登録しませんか?</span></p> <p><span style="font-weight: 400;">現時点で希望する職種での募集がないけれど、チャンスがあれば今後CircleCIの選考チャレンジしたい方やCircleCIへのキャリアに興味がある方、是非お気軽にレジュメもしくはLinkedin URLを是非ともご登録ください!その後、都度、カジュアル面談・選考の打診をさせていただく流れとなります。</span></p> <p><span style="font-weight: 400;">特定の職種に限定することなくご登録いただけますので、CircleCIキャリアの可能性やキャリア形成についてお伺いしたい方は是非お気軽にご登録ください。</span></p> <p><strong>Prospect positions we are seeking in the future</strong></p> <p><strong>将来募集が検討されるポジション一覧;</strong></p> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">Account Executive, SMB &amp; Enterprise industries</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Account Executive, APAC regions</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Sales Development Representative, Japan or APAC</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Customer Success Representative</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Marketing</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Support Technical Engineer</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">DevOps Engineer</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Solutions Engineer</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Full Stack Engineer</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Back-end Engineer</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Front-end Engineer</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Product Manager</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Project Manager</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Infrastructure Engineer&nbsp; &nbsp; Others...</span></li> </ul> <p><strong>Please note below your registration;</strong></p> <p><span style="font-weight: 400;">*This is not the official application. If you are interested in applying right away, please go to each opening job posting page. The recruiter will check your resume and if we find our jobs that will match with your experience and interests, we will contact you within 2 weeks. If you haven’t received any contact from us, please note that we don’t have any matching positions but we will update you if a position becomes available that is a match to your profile.&nbsp;</span></p> <p><span style="font-weight: 400;">こちらは正式応募ポジションではございません。もし正式な応募をご希望される際は、該当する募集ページからの応募をお願い致します。また、マッチングする募集ポジションがあった場合、2週間以内にリクルーターからポジションのご案内のご連絡を差し上げます。3週間経過しても連絡がなかった場合は、恐れ入りますが現時点でご提案できるポジションが無いとのことでご理解いただけましたら幸いです。その様な場合でも、カジュアルミーティングセッションの実施や月毎に最新情報を配信予定ですので是非チェックください!</span></p> <p><span style="font-weight: 400;">*Please note that this registration is </span><strong><em>only for people who have Japanese residency and who are eligible to work in Japan currently.</em></strong></p>
Support Engineering Manager
Boston, Denver, (Canada), (United States), San Francisco
<p>As Support Engineering Manager at CircleCI, you will lead a team of Support Engineers and guide the extended team of Customer Engineering, Revenue, and Customer Success to deliver a world-class customer experience to CircleCI users and prospective customers.</p> <p>We are looking for an energetic and performance-focused Support Engineering Manager. As the team's manager, you will enable them to support our rapidly growing customer base. You will help the team answer customer questions efficiently and effectively, and improve existing processes. CircleCI takes pride in the fact that our Support Engineers are true engineers, with deep technical skills, and an ability to resolve Tier 1, 2, and often Tier 3 level issues for a very technical user base (developers). Successful candidates that espouse that model can help us attract and retain the unique mix of skills vital to performing at this level.</p> <p>This job will be performed remotely and will be leading a distributed team across North America.</p> <h2><strong>About Support Engineering at CircleCI</strong></h2> <p>Our Support Engineers have a strong understanding of the developer community and are able to confidently communicate with and provide first class support to developers, DevOps leads, architects and decision makers.</p> <h2><strong>What you’ll do: </strong></h2> <ul> <li>Support the Support Engineering team by providing guidance on customer experience and approaches to solve specific challenges, be the point of escalation for the team, as well as represent their team's needs to Customer Engineering Leads and CircleCI leadership.</li> <li>Implement process improvements for consistency across the team.</li> <li>Scale planning for growing the team to meet demand and support CircleCI premium support packages.</li> <li>Represent the Voice of the Customer to CircleCI’s Leadership teams. Help to identify recurring problems, and general trends of user dissatisfaction to drive upstream improvements.</li> <li>Partner with the Customer Success team to ensure alignment across Support and Success initiatives.</li> <li>Coach, develop, and mentor team members by providing constructive feedback, developing career plans and using teaching opportunities.</li> <li>Collaborate with Product and Engineering teams to help influence product roadmap based on customer feedback, and to continue to build product knowledge and awareness within the support engineering team members</li> <li>Set, monitor, and drive success by establishing clear performance metrics.</li> <li>Maintaining high levels of transparency, efficiency, and collaboration in your team.</li> <li>Participate in launch of new products, documentation, and customer integrations to our internal teams and customer community.</li> </ul> <h2><strong>What we’re looking for: </strong></h2> <ul> <li>5+ years experience leading a rapidly growing Support team of 10+ people, preferably in a SaaS environment.</li> <li>Have worked directly with customers in a technical support role, solving complex technical problems.</li> <li>Can tie business problems to technical solutions and understand technology value propositions.</li> <li>Demonstrated and proven capacity to quickly absorb new concepts and technologies.</li> <li>Carry the philosophy that the best way for all to succeed is to honestly discuss product and company abilities and limitations with customers. We do not oversell at CircleCI.</li> <li>Demonstrated ability to prioritize competing opportunities, balancing customer needs with business priorities and the ability to articulate the rationale behind those decisions.</li> <li>Strong use of analytics, user research and intuition for business to drive decisions and improve the customer experience.</li> <li>You are empathetic and able to gracefully handle stressful situations.</li> </ul> <h2><strong>Bonus points!</strong></h2> <ul> <li>Experience with build, test, and deployment automation</li> <li>You have spent a decent amount of time using and scripting *nix</li> <li>You know how to work with Git in general and ideally GitHub in particular</li> <li>You’ve thrown together an app or two in a high-level programming language</li> <li>Knowledge of Docker or Linux containers in general</li> <li>Experience with popular web app frameworks and/or mobile app development (iOS, Android)</li> <li>Experience using and automating a major IaaS like AWS, GCP, or Azure</li> <li>Familiarity deploying and debugging distributed systems</li> </ul> <p><strong>** Not all skills required. If you’re interested in growing with us, please reach out!</strong></p> <p>We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.</p> <h2><strong>About CircleCI</strong></h2> <p>CircleCI is the world’s largest shared continuous integration/continuous delivery (CI/CD) platform, and the hub where code moves from idea to delivery. As one of the most-used DevOps tools - processing more than 1 million builds a day - CircleCI has unique access to data on how engineering teams work, and how their code runs. Companies like Spotify, Coinbase, and BuzzFeed use us to improve engineering team productivity, release better products, and get to market faster.</p> <p>Founded in 2011 and headquartered in downtown San Francisco with a global, remote workforce, CircleCI is venture-backed by Base10, Greenspring Associates, Eleven Prime, IVP, Sapphire Ventures, Top Tier Capital Partners, Baseline Ventures, Threshold Ventures, Scale Venture Partners, Owl Rock Capital, Next Equity Partners, Heavybit and Harrison Metal Capital.</p> <p>CircleCI is an Equal Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.</p>
DevOps Customer Engineer, EMEA
London
<p><span style="font-weight: 400;">As a DevOps Customer Engineer, you will be responsible for providing world class post-sales technical leadership for our highest value customers. Working directly with end users and up to senior decision-makers within customer software engineering teams, you will be the subject matter expert on both the CircleCI platform and Continuous Integration and Deployment as a general practice. You will be responsible for delivering value by driving adoption of CircleCI across the depth and breadth of customer organisation.&nbsp;</span></p> <p><span style="font-weight: 400;">The successful candidate for this job will have a strong technical aptitude and preferably some experience in CI/CD and DevOps practices. You will have a self-starter mentality and the ability to create and maintain deep, lasting relationships with customers.&nbsp; You’re going to be dealing with very technical users and complex issues, but you’re also tasked with creating excitement and loyalty in the customers you interact with.</span></p> <h3><strong>About Customer Engineering at CircleCI</strong></h3> <p>CircleCI’s Customer Engineering organization’s goal is to make life easier for our customers and leave them with the “wow” experience of building and testing their applications with ease. Customer Engineering works with customers to understand their technical and business needs and requirements, from onboarding to implementation to scale. Customer Engineering currently comprises DevOps Customer Engineering (your team), Solutions Engineering (your pre-sales counterparts), and Support Engineering (the global support team). There is opportunity for both horizontal and vertical growth and promotion within, as well.</p> <h3><strong>What you’ll do:&nbsp;</strong></h3> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">Work closely with customers to help them setup CircleCI, both on circleci.com and in their own private cloud, including any custom setup they may need.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Partner with your Customer Success Manager to onboard and support our customers as well as act as the dedicated technical point of contact.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Be creative and scrappy in solving customer technical problems and answering customer questions.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Build best practices for onboarding across different technologies.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Act as the voice of the customer and use customer feedback to help Product and Engineering improve the product.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Code and commit relevant upgrades and changes to the CircleCI codebase.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Work closely with the Product and Engineering teams to improve the customer experience across the whole platform.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Become an expert on the CircleCI solution.</span></li> </ul> <h3><strong>What we’re looking for:&nbsp;</strong></h3> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">3+ years of technical product support, engineering, or experience deploying software in the enterprise.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience working directly with customers to debug common errors.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">You can tie business problems to technical solutions and understand technology value propositions.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Demonstrated and proven capacity to quickly absorb new concepts and technologies.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">You have spent a decent amount of time using and scripting *nix.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Git and GitHub knowledge required.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">You’ve developed and maintained an app or two in a high-level programming language.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">You believe that the best way for all to succeed is to honestly discuss product and company abilities and limitations with customers.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Strong ops / infrastructure knowledge, especially networking and security.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Great relationship building skills and a good people person.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Exceptional written and oral communication skills.</span></li> </ul> <h3><strong>Bonus points!</strong></h3> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience with build, test, and deployment automation, either as a practitioner or in a customer-facing role.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Ability to troubleshoot networking issues that may prevent communication between different components.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Working knowledge of Docker or Linux containers.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience with popular web app frameworks (e.g. Rails, Django) and/or mobile app development (iOS, Android).</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Knowledge of Go and Clojure a plus!</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience using and automating a major IaaS like AWS, GCP, or Azure.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Management of autoscaling technologies (e.g. launch configurations in AWS).</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Familiarity deploying and debugging distributed systems. Familiarity with Nomad, specifically, a plus!</span></li> </ul> <h3><strong>About CircleCI&nbsp;</strong></h3> <p>CircleCI is the world’s largest shared continuous integration and continuous delivery (CI/CD) platform, and the central hub where code moves from idea to delivery. As one of the most-used DevOps tools that processes more than 1 million builds a day, CircleCI has unique access to data on how engineering teams work, and how their code runs. Companies like Spotify, Coinbase, Stitch Fix, and BuzzFeed use us to improve engineering team productivity, release better products, and get to market faster.</p> <p>Founded in 2011 and headquartered in downtown San Francisco with a global, remote workforce, CircleCI is venture-backed by IVP, Sapphire Ventures, Owl Rock, NextEquity Partners, Scale Venture Partners, Threshold Ventures, Baseline Ventures, Top Tier Capital Partners, Industry Ventures, Heavybit, and Harrison Metal Capital. Learn more at&nbsp;<a class="c-link" href="https://circleci.com/" target="_blank">https://circleci.com</a>.</p> <p>CircleCI is proud to be an Equal Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.</p>
Senior Staff Software Engineer
US, Canada
<p><span style="font-weight: 400;">At CircleCI, we enable developers to do better work every day. We focus solely on helping our customers ship quality code, faster. Our organization is growing to meet the demands of our customers, and we need an experienced Backend Architect to join our adventure!</span></p> <p><span style="font-weight: 400;">As a Senior Staff Engineer you will oversee the technical aspects of our direction and quality of delivery within our Application Engine group. This set of teams is accountable for the underlying functions that power our products at scale. Your deep knowledge of API design and backend services will help us deliver functionality that empowers developers worldwide. You'll be at the forefront of crafting powerful solutions and setting the standards for backend engineering across our organization.</span></p> <h2><strong>What you’ll do:</strong></h2> <ul> <li style="font-weight: 400;"><strong>Guide several teams</strong><span style="font-weight: 400;"> in anticipation of future use cases and help them make design decisions that minimize the cost of future changes. </span><span style="font-weight: 400;">You will support and empower these teams to do their best work.</span></li> <li style="font-weight: 400;"><strong>Be a strategic partner </strong><span style="font-weight: 400;">and actively work with engineering teams and cross-functionally with other departments to develop and apply scalable solutions.</span></li> <li style="font-weight: 400;"><strong>Regularly mentor and coach engineers</strong><span style="font-weight: 400;"> to expand their skills and support their career growth, including pair programming.</span></li> <li style="font-weight: 400;"><strong>Take leadership</strong><span style="font-weight: 400;"> on particularly critical code reviews in multiple code languages as well as </span><span style="font-weight: 400;">develop a cross-organizational technical strategy and architectural vision by analyzing cross-team challenges and designing measurable solutions.</span></li> <li style="font-weight: 400;"><strong>Contribute to building the standard methodologies</strong><span style="font-weight: 400;"> of a growing organization.</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Share rotating on-call duties for our </span><strong>incident response.</strong></li> </ul> <h2><strong>What we're looking for:</strong></h2> <p><span style="font-weight: 400;">We’re seeking someone who thrives in a collaborative environment, is naturally curious and interested in learning, has strong communication and collaboration skills, and helps others grow by sharing their expertise and encouragement.</span></p> <ul> <li style="font-weight: 400;"><strong>Experience:</strong><span style="font-weight: 400;"> You bring 10+ years of technical leadership experience working in a modern cloud company and expertise in distributed systems. You're experienced with at least some of our tech stack, are willing to learn new technologies and languages, and aren’t afraid to admit what you don’t know. Learn more about our tech stack</span><a href="https://stackshare.io/circleci/circleci"><span style="font-weight: 400;"> </span><span style="font-weight: 400;">here</span></a><span style="font-weight: 400;">.</span></li> <li style="font-weight: 400;"><strong>Collaboration:</strong><span style="font-weight: 400;"> You thrive in a collaborative environment, are naturally curious and interested in learning, and enjoy helping others grow by sharing your expertise and encouragement.</span></li> <li style="font-weight: 400;"><strong>Communication</strong><span style="font-weight: 400;">: You encourage open communication both asynchronously and synchronously, value feedback loops, and are open to being wrong and having your mind changed. You’re proficient in English in verbal and written form.</span></li> <li style="font-weight: 400;"><strong>Delivery</strong><span style="font-weight: 400;">: You know the ins and outs of modern software engineering practices, agile/scrum methodologies, and how to apply them to drive effective, timely, high-quality delivery. You provide visibility into your work and progress.</span></li> <li style="font-weight: 400;"><strong>Leadership</strong><span style="font-weight: 400;">: You foster a culture of ideas, write problem statements, and proactively engage with and educate both business partners and other engineers cross-functionally to understand the impact of their work. You have experience partnering with product and business leaders while providing technical direction for a team with engineering managers and engineers.</span></li> <li style="font-weight: 400;"><strong>Adaptability</strong><span style="font-weight: 400;">: You're comfortable with ambiguity and figuring things out as they evolve, change, and new requirements emerge.</span></li> <li style="font-weight: 400;"><strong>Remote work</strong><span style="font-weight: 400;">: You are comfortable working with distributed teams across multiple time zones. Being remote-first, we foster a culture that is inclusive of remote workers while allowing everyone to be the most productive. Learn more about our remote-first culture</span><a href="https://circleci.com/blog/what-it-means-to-be-remote-first-vs-remote-friendly/"><span style="font-weight: 400;"> </span><span style="font-weight: 400;">here</span></a><span style="font-weight: 400;">.</span></li> </ul> <h2><strong>CircleCI Engineering Career Growth System:</strong></h2> <p><span style="font-weight: 400;">This role equals level E5 on our</span><a href="https://drive.google.com/file/d/1F3xzmbdsMvfDZwZesvxcEIIBn2TmI4sg/view"><span style="font-weight: 400;"> Engineering Competency Matrix</span></a><span style="font-weight: 400;">, our internal career growth system for engineers. Find more about the matrix in this</span><a href="https://circleci.com/blog/why-we-re-designed-our-engineering-career-paths-at-circleci/"><span style="font-weight: 400;"> blog post</span></a><span style="font-weight: 400;">.</span></p> <p><span style="font-weight: 400;">We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.</span></p> <h2><strong>About CircleCI</strong></h2> <p><span style="font-weight: 400;">CircleCI is the world’s largest shared continuous integration and continuous delivery (CI/CD) platform, and the central hub where code moves from idea to delivery. As one of the most-used DevOps tools that processes more than 1 million builds a day, CircleCI has unique access to data on how engineering teams work, and how their code runs. Companies like Spotify, Coinbase, and BuzzFeed use us to improve engineering team productivity, release better products, and get to market faster.</span></p> <p><span style="font-weight: 400;">CircleCI is proud to be an Equal Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.</span></p> <p>Colorado Salary Range: $160,000 - $200,000/year</p>
Verified by
Developer Evangelist
Support Engineer
Vice President of Marketing
Technical Content Marketing Manager
Head of DevRel & Community
You may also like