How Shippo Built and Maintains Versioned APIs

4,683
Vkm5m6eh
Shippo
Shipping made simple.

By Subhi Beidas, Head of the API team at Shippo. The API Team is responsible for developing the API while delivering great developer experience. Subhi holds a degree in Computer Engineering from the Illinois Institute of Technology.



Shippo API


Shippo is a shipping API that helps developers connect to a global network of carriers like USPS, FedEx, DHL, UPS and others to print shipping labels, track packages, and more. We work with a wide range of developers to help them send packages around the world. Shippo has grown to power over 10,000 businesses everyday, sending millions of packages to and from 196 countries around the world.

An API is a contract to perform a specific service when a specific request comes in. This is especially important for APIs such as Shippo’s, which power essential business infrastructures. If the API doesn’t work, the business can’t run.

After onboarding thousands of new customers in the last two years, we recognized a tremendous need to improve the API design, which in some cases implied introducing backwards incompatible changes to our API. For example, when we launched, we provided developers with a single authorization token which was used for both test and production calls. Whether or not creating a shipping label would result in the user being charged depended on an attribute in the CarrierAccount object. As we started adding more carriers, each of those CarrierAccounts would have its own test flag that could be set independently, which resulted in unnecessary confusion for users: accidental production label purchases, intermixing of test objects and production objects, etc. We wanted to move to a better approach: issuing a separate test token and production token, so developers could very clearly differentiate the calls they were making to our API.

This raised a difficult question: if existing customers built their shipping infrastructure relying on the consistency of our current API specifications, how can we change the API without requiring them to tear out their integration and start over again? We wanted the ability to improve our service without breaking our “contract” with existing users. This challenge led us to API versioning.

API versioning allows us to make backwards incompatible changes packaged into isolated versions of the system. This way, existing developers can choose to opt into newer versions when they are ready, and not be forced.

If they want new features that are available in later versions, they can decide if it’s worthwhile to refactor their existing integration. If they are worried about old features that might lose support in newer versions, they can leave their code as is and trust that the old version of the API will continue to work as it did when they built their software. Moreover, versioning also allowed us to continue to add backwards compatible changes as needed, where we can.

Having carefully studied how other developers versioned their APIs, we wanted to share our considerations, decision-making process, and final designs for our API versioning.

We considered multiples approaches to API versioning. Our top criteria were:

  • How well it enables us to make backwards incompatible API changes that fit with the short term and long term product roadmap.
  • How many engineering resources we need to invest to build this.
  • How testable each independent version can be and how much confidence we can have in that changes in one version do not unintentionally affect others.

Considered approaches:

1. Proxy, which points requests to different versioned codebases

The pros:

  • Freedom to transition to a new model structure / architecture / design

The cons:

  • Building a versioning proxy, requires keeping one version static, and developing a new codebase from scratch, which leads to maintaining different code bases
  • Data migration will be a painful and challenging, not something we can do often
  • Code duplication

2. One router, which points requests to one app with versioned controllers

The pros:

  • No data migration needed since the models are shared

The cons:

  • We'll have to maintain and develop multiple controllers

3. One router, versioned views that share the same controllers

The pros:

  • No data migration needed since the models are shared
  • Least amount of data duplication (compared to 1 & 2), since the models, controllers, routes, and a lot of resources are shared
  • Ease of creating new versions

The cons:

  • Code sharing makes it the most likely that a change in one version will affect other versions

After reviewing the above options, we decided on option #3: One router, versioned views that share the same controllers since:

  • It provides the easiest way to quickly make versioned updates to the API
  • It involves the least code duplication and engineering overhead
  • It addressed all types of backwards incompatible changes posed by the current projects on our roadmap
  • We did not have a pressing business/product need to re-write the data models
  • In case we need to have different controller logic, we can have the versioned views call the controllers with different parameters

With this implementation, as you go deeper in the stack, more resources will be required to maintain the different versions. But since most of the immediate changes that required versioning were either changes in the representation of the JSON objects or in the parameters passed to our controllers, the best implementation for our needs was to go with one router and versioned views that share the same controllers. Here is how we did it.


Shippo Versioning Dashboard


The Routing, the Views, the Serializers

To ensure that every request is dispatched using VersionableView each url endpoint is mapped to a defined VersionableView. As the versioning is intended to be ‘per user’, Shippo persists the version in the API user context, and when an API request is made, the backend maps the request to the corresponding API user.

The VersionableView then determines which version the API user is on and routes to the respective view accordingly. The Views and Serializers (and unit tests) of each view are grouped together in the same folder of each version.

This strategy allows us to keep code duplication minimal. For example, to implement a small incompatible serializer change, the view in the most recent version inherits the view from the previous version and only overrides the serializer it uses.

If there are no changes to an endpoint across versions, then there is no need to copy the code over, the routing module looks up the view from the previous version.

This means that if only a serializer change is needed in v20161025, the views.py file will look like this.


Shippo Folder Structure


Upgrading and Downgrading Versions

Developers can always upgrade their version to the latest available Shippo API version through their API tab in the dashboard.

To keep the flow simple, we do not allow users to upgrade to an intermediate version, and we do not allow a user to downgrade their API version once it’s set.

The primary developer experience use case for API versioning is to help developers test their API integration before making a permanent upgrade. To do that, developers can set a temporary header in their request, which overrides their default API version. They can send requests with a different version until they are ready to permanently upgrade their version.

Documentation and Changelogs

Introducing different versions of the API, also means supporting different versions of documentation and ensuring that our changelog very clearly defines the differences between each version.

If a developer is logged in, we serve the developer docs that correspond to their version. If this version is outdated, the developer has the option to update to the latest version. If the developer is not logged in, then we will show them docs for the latest version.

It’s been a few months since the release of API versioning, and we’ve been quite pleased with the performance and flexibility of the implementation as we’ve built on top of it. By no means is this the end-all to API versioning. If you’re considering building out versioning for your API, have versioned your API differently, or want to leave us feedback on our versioning experience, let us know!

Acknowledgements to the engineers that helped shape this project: Thiago Riberio Ramos and Ankit Jain

Vkm5m6eh
Shippo
Shipping made simple.
Lead Customer Solutions Engineer

Description

Shippo lowers the barriers to shipping for businesses around the world. As free and fast shipping becomes the norm, better access to shipping is a competitive advantage for businesses. Through Shippo, ecommerce businesses, marketplaces, and platforms are able to connect to multiple shipping carriers around the world from one API and dashboard. Businesses can get shipping rates, print labels, automate international documents, track shipments, and facilitate returns. Internally, we think of Shippo as the building blocks of shipping. Shippos are a diverse set of individuals. We look for culture and skills add in every new person. Join us to build the foundations of something great, roll up your sleeves and get important work done everyday. Founded in 2013, we are a proud team of 70 based out of San Francisco. Shippo’s investors include Union Square Ventures, SoftTechVC, VersionOne Ventures, FundersClub and others.

As a Senior/Lead Customer Solutions Engineer at Shippo, you will be the customer’s technical point of contact through the integration process. You will help our largest, most valuable customers get up and running with their Shippo integration by building custom connectors between Shippo and their applications or by building features within Shippo’s codebase. You will work closely with our Business Development, Customer Success, and Engineering teams as well as directly with customers to ensure their integrations’ technical requirements are met. You will excel in this position if you value collaborative cross-functional work environments where you can still operate with a lot of autonomy, see the immediate impact of your work, and quickly shift gears for the next project.

RESPONSIBILITIES:

  • Work with our Business Development and Account Management teams to comprehensively understand our customers’ technical requirements
  • Recommend and develop creative solutions to reduce customer dependency on their own developer teams, accelerating time-to-live
  • Prepare proofs of concept and demo your solutions with customers to ensure your solution will meet their needs
  • Speak to the technical capabilities of Shippo’s API with a variety of stakeholders internally and on the customer side
  • Document the architectural solutions that customers have designed and deployed
  • Communicate customer needs and product feedback to Product Management and Engineering
  • Build custom, one-off, and/or reusable features within the Shippo codebase to accelerate customers’ integration timelines
  • Troubleshoot customer issues and questions to barrel through blockers and keep their implementation moving smoothly

Requirements

  • 5+ years in a technical, customer-facing role
  • Experience working in an agile development environment
  • Deep knowledge of Python or Java, C#, REST API, JavaScript, and SQL with a strong CS foundation
  • Experienced with professional application development. Experience in design, configuration, deployment and debugging of applications on top of REST APIs
  • Excellent verbal, written, and interpersonal communication skills; ability to communicate with non-technical team members and a deep understanding of customer needs and passion for customer success

Benefits

  • Medical, dental, vision (90% covered by the company, incl. dependents)
  • Flexible vacation policy + flexible work hours
  • Free lunch / drinks / snacks
  • Fun team events outside of work hours
  • Awesome team that cares about Shippo's mission, product and co-workers!
Comments
Open jobs at Shippo
Data Engineer

Description

At the core of every successful e-commerce company is a rock solid shipping system. Shippo makes this easy! We provide a RESTful API and powerful UI tools that connect SMBs, marketplaces, and platforms to multiple shipping carriers. Backed by leading investors and with partners such as Godaddy and Shopify we are growing fast and processing thousands of shipments every day.

As a data engineer, you will be responsibility for building systems to collect and process events of massive scale to gain operational and business insight into the performance and optimization of shipping services. The data engineer will work closely with product, engineering, and business leads in generating customer-facing and internal dashboards, ad hoc reports, and models to provide insights and affect platform behavior. This will also include building and maintaining the infrastructure to collect and transform raw data.

  • Design, build, scale, and evolve large scale data systems
  • Integrate data from various data stores to ensure consistency and availability of data insights
  • Prioritize tasks and deliverables, optimizing for a balanced delivery on short-term and long-term goals
  • Articulate and present findings and recommendations at different levels, with a clear bias towards impactful learning and results
  • Drive the usability and impact of the data projects in ad hoc analysis and real-time and batch processing
  • Champion engineering organization’s adoption and ongoing use of the data infrastructure

Requirements

  • BS or MS in Computer Science or related technical discipline or equivalent job experience
  • 3+ years working experience with a scripting language such as Python, Ruby or Perl
  • Experience with RDBMS, such as PostgreSQL or MySQL, and NoSQL and columnar data stores; experience with implementing ETL process
  • Experience with Big Data frameworks such as Hadoop, MapReduce and associated tools
  • Experience building stream-processing systems, using solutions such as Kinesis Stream or Spark-Streaming
  • Experience with statistical analysis and a data visualization package such as R, Mathematica, Stata, Tableau, etc.
  • Experience with cloud environments and devops tools; working experience with AWS and its associated products a plus
  • Experience with machine learning a plus
  • Excellent written, oral communication, and presentation skills

Benefits

  • Benefits: medical, dental, vision (90% covered by the company, incl. dependents)
  • Take-as-much-as-you-need vacation policy + flexible work hours
  • Free lunch / drinks / snacks
  • Fun team events outside of work hours
  • Awesome team that cares about Shippo's mission, product and their co-workers!
Senior Software Engineer

Description

San Francisco, CA

At the core of every successful e-commerce company is a rock solid shipping system. Shippo makes this easy! We provide a RESTful API and powerful UI tools that connect SMBs, marketplaces, and platforms to multiple shipping carriers. Backed by leading investors and with partners such as Godaddy and Shopify we are growing fast and processing thousands of shipments every day.

As an engineer focused on the Shippo application, you strive to create logical, performant code and work to understand the needs of both our internal and external API customers. Shippo applications process millions of packages per month and we are always looking for new ways to use our data to create value for our customers.

Responsibilities

  • Design, implement, test, and support API client libraries in PHP, C#, JavaScript (Node.js).
  • Design, implement, test, and support Application Program Interface (API) Software Development Kits (SDKs), including mobile SDKs for iOS, Android, and React Native implementations.
  • Design and implement software architecture, capable of handling large volume of shipping API requests, including address validation, rating, label creation, and tracking, providing better than industry-standard service-level uptime and response time performance.
  • Build and maintain shipping carrier and shopping cart adapter architecture, such that new integrations can be easily or automatically added, without disruption and impact to core systems.
  • Create technical specifications, design, develop, test, and deploy new API versions and enhancements to existing APIs.
  • Continually improve API availability, fault-tolerance, performance, and latency through automation, performance turning and optimization.


Requirements

  • Bachelor’s degree or foreign equivalent degree in Computer Engineering, Computer Science or related technical field
  • 1 year of work experience in software engineering, software research or software development related occupation

Benefits

  • Benefits: medical, dental, vision (90% covered by the company, incl. dependents)
  • Take-as-much-as-you-need vacation policy + flexible work hours
  • Free lunch / drinks / snacks
  • Fun team events outside of work hours
  • Awesome team that cares about Shippo's mission, product and their co-workers!

How to apply

Interested candidates send resume to: May Feng, Popout, Inc. dba Shippo, 965 Mission Street, Suite 480, San Francisco, CA 94103. Attn.: Job #2.

Senior Data Engineer

Description

At the core of every successful e-commerce company is a rock solid shipping system. Shippo makes this easy! We provide a RESTful API and powerful UI tools that connect SMBs, marketplaces, and platforms to multiple shipping carriers. Backed by leading investors and with partners such as Godaddy and Shopify we are growing fast and processing thousands of shipments every day.

As a data engineer, you will be responsibility for building systems to collect and process events of massive scale to gain operational and business insight into the performance and optimization of shipping services. The data engineer will work closely with product, engineering, and business leads in generating customer-facing and internal dashboards, ad hoc reports, and models to provide insights and affect platform behavior. This will also include building and maintaining the infrastructure to collect and transform raw data.

  • Design, build, scale, and evolve large scale data systems
  • Integrate data from various data stores to ensure consistency and availability of data insights
  • Prioritize tasks and deliverables, optimizing for a balanced delivery on short-term and long-term goals
  • Articulate and present findings and recommendations at different levels, with a clear bias towards impactful learning and results
  • Drive the usability and impact of the data projects in ad hoc analysis and real-time and batch processing
  • Champion engineering organization’s adoption and ongoing use of the data infrastructure

Requirements

  • BS or MS in Computer Science or related technical discipline or equivalent job experience
  • 3+ years working experience with a scripting language such as Python, Ruby or Perl
  • Experience with RDBMS, such as PostgreSQL or MySQL, and NoSQL and columnar data stores; experience with implementing ETL process
  • Experience with Big Data frameworks such as Hadoop, MapReduce and associated tools
  • Experience building stream-processing systems, using solutions such as Kinesis Stream or Spark-Streaming
  • Experience with statistical analysis and a data visualization package such as R, Mathematica, Stata, Tableau, etc.
  • Experience with cloud environments and devops tools; working experience with AWS and its associated products a plus
  • Experience with machine learning a plus
  • Excellent written, oral communication, and presentation skills

Benefits

  • Benefits: medical, dental, vision (90% covered by the company, incl. dependents)
  • Take-as-much-as-you-need vacation policy + flexible work hours
  • Free lunch / drinks / snacks
  • Fun team events outside of work hours
  • Awesome team that cares about Shippo's mission, product and their co-workers!
Sales Engineer

Description

At the core of every successful e-commerce company is a rock solid shipping system. Shippo makes this easy! We provide a RESTful API and powerful UI tools that connect ecommerce companies, marketplaces, and platforms to multiple shipping carriers around the world. Backed by leading investors and with partners such as Weebly, Godaddy and Shopify we are growing fast and processing tens of thousands of shipments every day.


We’re looking for a Sales Engineer to be the product expert and to help our technical prospects understand how Shippo solves their shipping challenges. As one of the charter members, you’ll be able to work with prospects and help acquire significant opportunities. We’re looking for someone with an entrepreneurial drive in a fast growing company.

  • Identify customer requirements and communicate possible solutions to both internal and external stakeholders.
  • Build demos and manage proof of concepts, showing how customers can use the Shippo APIs to solve their problems.
  • Lead projects that drive to faster technical implementation of our solution and activation.
  • Partner with the Business Development, Sales, and Customer Success teams to help customers understand the technological and business value of Shippo.
  • Determine new solutions and provide process improvement feedback to the product management, engineering, and marketing teams.

Requirements

  • BA/BS degree in Engineering, Computer Science, MIS or equivalent practical experience preferred.
  • 3+ years of experience in a technical role. SaaS experience with API exposure is a major plus.
  • Experience working directly with customers, particularly in sales and partnership settings.
  • You have excellent communication skills, with the ability to discuss technical information in a simple, responsive and effective way.
  • Strong understanding of (RESTful APIs, HTTP) and experience in one or more of the following languages (Python, Ruby, JavaScript (Node.js), PHP, Java, C#).
  • You have the ability to manage multiple priorities, projects and relationships at the same time.
  • You can quickly grasp the customer or partner needs and deliver an articulate response in a timely fashion.

Benefits

  • Medical, dental, vision (90% covered by the company, incl. dependents), commuter (transit and parking), 401K
  • Take-as-much-as-you-need vacation policy + flexible work hours, remote working possible
  • Free lunch / drinks / snacks
  • Fun team events outside of work hours - happy hours, “escape the room” adventures, hikes, and more!
You may also like
How Stream Built a Modern RSS Reader With JavaScript
How Heap Built an Analytics Platform that Auto-Tracks Every User Event
How Raygun Processes Millions of Error Events Per Second
Stream & Go: News Feeds for Over 300 Million End Users