By Subhi Beidas, Head of the API team at Shippo. The API Team is responsible for developing the API while delivering great developer experience. Subhi holds a degree in Computer Engineering from the Illinois Institute of Technology.
Shippo is a shipping API that helps developers connect to a global network of carriers like USPS, FedEx, DHL, UPS and others to print shipping labels, track packages, and more. We work with a wide range of developers to help them send packages around the world. Shippo has grown to power over 10,000 businesses everyday, sending millions of packages to and from 196 countries around the world.
An API is a contract to perform a specific service when a specific request comes in. This is especially important for APIs such as Shippo’s, which power essential business infrastructures. If the API doesn’t work, the business can’t run.
After onboarding thousands of new customers in the last two years, we recognized a tremendous need to improve the API design, which in some cases implied introducing backwards incompatible changes to our API. For example, when we launched, we provided developers with a single authorization token which was used for both test and production calls. Whether or not creating a shipping label would result in the user being charged depended on an attribute in the CarrierAccount object. As we started adding more carriers, each of those CarrierAccounts would have its own test flag that could be set independently, which resulted in unnecessary confusion for users: accidental production label purchases, intermixing of test objects and production objects, etc. We wanted to move to a better approach: issuing a separate test token and production token, so developers could very clearly differentiate the calls they were making to our API.
This raised a difficult question: if existing customers built their shipping infrastructure relying on the consistency of our current API specifications, how can we change the API without requiring them to tear out their integration and start over again? We wanted the ability to improve our service without breaking our “contract” with existing users. This challenge led us to API versioning.
API versioning allows us to make backwards incompatible changes packaged into isolated versions of the system. This way, existing developers can choose to opt into newer versions when they are ready, and not be forced.
If they want new features that are available in later versions, they can decide if it’s worthwhile to refactor their existing integration. If they are worried about old features that might lose support in newer versions, they can leave their code as is and trust that the old version of the API will continue to work as it did when they built their software. Moreover, versioning also allowed us to continue to add backwards compatible changes as needed, where we can.
Having carefully studied how other developers versioned their APIs, we wanted to share our considerations, decision-making process, and final designs for our API versioning.
We considered multiples approaches to API versioning. Our top criteria were:
- How well it enables us to make backwards incompatible API changes that fit with the short term and long term product roadmap.
- How many engineering resources we need to invest to build this.
- How testable each independent version can be and how much confidence we can have in that changes in one version do not unintentionally affect others.
Considered approaches:
1. Proxy, which points requests to different versioned codebases
The pros:
- Freedom to transition to a new model structure / architecture / design
The cons:
- Building a versioning proxy, requires keeping one version static, and developing a new codebase from scratch, which leads to maintaining different code bases
- Data migration will be a painful and challenging, not something we can do often
- Code duplication
2. One router, which points requests to one app with versioned controllers
The pros:
- No data migration needed since the models are shared
The cons:
- We'll have to maintain and develop multiple controllers
3. One router, versioned views that share the same controllers
The pros:
- No data migration needed since the models are shared
- Least amount of data duplication (compared to 1 & 2), since the models, controllers, routes, and a lot of resources are shared
- Ease of creating new versions
The cons:
- Code sharing makes it the most likely that a change in one version will affect other versions
After reviewing the above options, we decided on option #3: One router, versioned views that share the same controllers since:
- It provides the easiest way to quickly make versioned updates to the API
- It involves the least code duplication and engineering overhead
- It addressed all types of backwards incompatible changes posed by the current projects on our roadmap
- We did not have a pressing business/product need to re-write the data models
- In case we need to have different controller logic, we can have the versioned views call the controllers with different parameters
With this implementation, as you go deeper in the stack, more resources will be required to maintain the different versions. But since most of the immediate changes that required versioning were either changes in the representation of the JSON objects or in the parameters passed to our controllers, the best implementation for our needs was to go with one router and versioned views that share the same controllers. Here is how we did it.
The Routing, the Views, the Serializers
To ensure that every request is dispatched using VersionableView each url endpoint is mapped to a defined VersionableView. As the versioning is intended to be ‘per user’, Shippo persists the version in the API user context, and when an API request is made, the backend maps the request to the corresponding API user.
The VersionableView then determines which version the API user is on and routes to the respective view accordingly. The Views and Serializers (and unit tests) of each view are grouped together in the same folder of each version.
This strategy allows us to keep code duplication minimal. For example, to implement a small incompatible serializer change, the view in the most recent version inherits the view from the previous version and only overrides the serializer it uses.
If there are no changes to an endpoint across versions, then there is no need to copy the code over, the routing module looks up the view from the previous version.
This means that if only a serializer change is needed in v20161025, the views.py file will look like this.
Upgrading and Downgrading Versions
Developers can always upgrade their version to the latest available Shippo API version through their API tab in the dashboard.
To keep the flow simple, we do not allow users to upgrade to an intermediate version, and we do not allow a user to downgrade their API version once it’s set.
The primary developer experience use case for API versioning is to help developers test their API integration before making a permanent upgrade. To do that, developers can set a temporary header in their request, which overrides their default API version. They can send requests with a different version until they are ready to permanently upgrade their version.
Documentation and Changelogs
Introducing different versions of the API, also means supporting different versions of documentation and ensuring that our changelog very clearly defines the differences between each version.
If a developer is logged in, we serve the developer docs that correspond to their version. If this version is outdated, the developer has the option to update to the latest version. If the developer is not logged in, then we will show them docs for the latest version.
It’s been a few months since the release of API versioning, and we’ve been quite pleased with the performance and flexibility of the implementation as we’ve built on top of it. By no means is this the end-all to API versioning. If you’re considering building out versioning for your API, have versioned your API differently, or want to leave us feedback on our versioning experience, let us know!
Acknowledgements to the engineers that helped shape this project: Thiago Riberio Ramos and Ankit Jain
4
1303
