How Mashape Manages Over 15,000 APIs & Microservices

15,973
Kong
Powering API-Driven Software

Editor's note: By Ahmad Nassri, ‎Head of Engineering at Mashape


Background

Mashape powers API-driven software. Hundreds of thousands of developers use our tools to manage, monitor, consume and provide APIs to their partners, apps, customers and employees. In fact, billions of API requests are processed through Mashape's Marketplace every month.

I'm Ahmad Nassri, Head of Engineering at Mashape. Prior to Mashape I was the Development Manager at The Canadian Broadcasting Corporation (CBC ) and spent many years building Web, Mobile & API products across many startups and businesses.

Engineering Team

The Mashape engineering team is one of the most highly skilled teams I've had the pleasure to work with. While we operate as a single team, our responsibilities are split and organized per project. Currently we have three main projects:

As our team is spread out among San Francisco, Toronto, Montreal, Paris and London, we have faced a number of challenges in scheduling, communications and syncing. To overcome these issues, we follow the open source model of building software and collaborating; ensuring that all communication is clear and verbose, assuming asynchronousity, and keeping a communication channel open at all times on Flowdock, while regularly jumping on Google Hangouts whenever further discussion is needed.

Despite having a focus of responsibilities, the engineering team members are not isolated in silos. We encourage everyone to work and touch different parts of the codebase, across all projects; in fact, most of us will end up working on Mashape Analytics, Marketplace and Mashape ID all in the same week.

Mashape's design team interacts with the engineers on a daily basis. By compiling engineer comments and gathering feedback from users, we're able to design the best Developer Experience (DX) by focusing on our users first. It also helps that we are engineers and developers!

We have regular one hour conference calls every Monday where the previous week's work is reviewed, and new objectives for the coming week are established.


Engineering Team (API marketplace team meeting, mid 2014)


General Architecture

Once upon a time, Mashape's architecture consisted of around 150K lines of Java, a true Monolith! But it's now split into 400 different microservices built with a variety of languages including: Node.js, Lua, Ruby, Java, Python, and OCaml. Everything is an API.

Mashape ID, as an example, is an identity authentication platform operating entirely as an API. Any time a new product is created, we simply plug it into Mashape ID; which provides out of the box account & email management with a companion billing microservice of its own.

One of our new projects is powered by a Ruby API, while the frontend app is built in Ember.js. Conversely, Mashape's marketplace has Java-powered internal APIs which in turn are consumed by a number of Express.js microservices serving as the web frontend; all our applications & APIs follow The 12-factor methodology.

In front of all of our applications and APIs is Kong. We use ElasticSearch for search while also storing data in a number of databases including: MongoDB, PostgreSQL, Redis and Cassandra (each solving a different problem domain). All of our products run on AWS, though on occasion we use Heroku and DigitalOcean for smaller projects to experiment with prototypes or test environments. Splunk is our centralized logging system, Datadog as a unified monitoring dashboard, and PagerDuty for alerts.

Mashape pushes business intelligence metrics from all our products directly into Chart.io. Additionally, we dogfood quite a lot by running Mashape Analytics to monitor and visualize API traffic for the APIs managed by our platforms. TravisCI is utilized for testing, CodeClimate for quality control, while using Chef for deploys. All our code is on Github. For code snippet generation we use our own HTTPsnippet and Mockbin for mocking & testing API prototypes.

All of these applications and systems, alongside all the deployment, monitoring & metrics tools may seem like a lot, but the microservices approach coupled with the 12-factor methodology allows for zen & simplicity in a sea of chaos!

During the course of a slow week we have ~400 internal services running over a cluster of hundreds of EC2 machines (consisting of mainly large instances). Having a microservice architecture allowed us to move much faster and independently, we clearly saw the benefits of the transition. But it's probably helpful to understand why we made the transition.

Scaling challenges

Sometime around 2013, we experienced scaling issues with the dozens of standalone third-party APIs added to Mashape's Marketplace.

Between July 2012 and July 2013 our volume in API transactions grew 50X.


Scaling Problems


Not only was the volume of requests passing through our proxy taking off, Mashape was starting to manage more APIs than anyone else in the market. At that time, the Marketplace had 3,500 public APIs under its belt.

The Marketplace needed to scale fast. Our API proxy had been built in Node.js, with a heavy Java backend. It was a monolithic architecture, and our Node.js proxy could not handle large traffic spikes.

Overtime it became unpredictable and resulted in consuming larger amounts of AWS resources. Our bank account suffered as a consequence and our CEO lost some hair.

We had to find a solution that satisfied not only the massive amount of traffic, but also be best in class in reliability and security. We wanted to move away from a monolithic architecture towards something much more flexible.


Architecture


We've always had a desire to create our own API gateway; targeting the following attributes:

  1. Predictable stability over months and years, not days.
  2. Efficient and easy to scale across multiple geographic regions with little overhead.
  3. Modular and extensible via simple plugins.
  4. Independent: environment, framework & language agnostic.
  5. Easily configurable via a RESTful API.

Basically an "ElasticSearch" for API management. Our CTO began his search and eventually met an engineer at CloudFlare, who introduced us to the amazing world of OpenResty. Built on top of Nginx it allows a developer to extend Nginx functionalities with Lua, an embedded scripting language used by many heavy traffic websites.

CloudFlare built Lua scripts around security and caching. We thought we could do the same but with a focus on API and microservice management. Nginx was already proven worldwide as a stable and trustworthy HTTP server, and was perfect for our base requirements. In combination with OpenResty, we started building Kong.

Kong is born (the API Gateway)

In 2013, Mashape adopted Nginx / OpenResty to implement features on the Marketplace's published APIs. Billing, authentication, throttling and rate-limiting became custom OpenResty plugins written in Lua. We called it Kong (yes, like "King Kong", after our own namesake: MashAPE).


King Kong


Today Kong efficiently manages over 15,000 APIs. Serving both our internal microservices and our Marketplace clients' APIs, all while handling spikes, high concurrent scenarios, and billions of successful API calls per month.

We created our own "ElasticSearch" for API management; much like Elastic's approach to wrapping Lucene in a RESTful interface with some sugar on top, Kong was a wrapper around Nginx.

Below are some examples of Kong's operations within the Marketplace.

Once an API is added to the Mashape platform from the GUI, it sends a POST request to Kong:

curl -i -X POST \
  --url http://localhost:8001/apis/ \
  --data 'name=mockbin' \
  --data 'upstream_url=http://mockbin.com/' \
  --data 'request_host=mockbin.com'

Then, based on the API provider's needs, a collection of various plugins are added on top of it. As an example, if the user needs Key Authentication:

curl -i -X POST \
  --url http://localhost:8001/apis/mockbin/plugins/ \
  --data 'name=key-auth'

User management is a breeze; the Consumer object allows for an abstract entity that can represent Users, Applications, Clients, or any system accessing the API. They are easily created and managed through the RESTful interface.

curl -i -X POST \
  --url http://localhost:8001/consumers/ \
  --data "username=Jason"

Kong's RESTful API allows us to manage thousands of APIs in the Marketplace. With simple and single-focused codebases, we keep our Microservices and individual systems completely independent. Kong itself, by its pluggable nature, can be described as a paragon of microservice architecture. At its core, it consists of no more than a couple thousands lines of code, which handle database abstraction, routing and plugin management. Plugins can live in different code bases and inject themselves anywhere into the lifecycle of a proxied request in a few lines of code. We are very proud of this approach which is quite unique in API management gateways today.

Kong was released to the public in April 2015 - after being partly rewritten for the occasion - and has built a strong community of users and contributors since then. We are very grateful for the contributions brought by developers all around the world, and feel rewarded for our decision to open-source Kong, all the while contributing back to the community by improving many Lua packages used in OpenResty.

We generate Kong packages for many Linux distributions and cloud providers to allow ease of installation and deployment. The most successful distribution so far is Kong on Docker with over 180,000 downloads since it's release.

The Road Ahead

Every month, we release more pieces of Kong to the open-source realm as part of our commitment to drive and grow the open-source API developer community. We are working hard to make Kong reach the 1.0 public milestone so that everybody can enjoy what this internal tool brought to our company and our community.

We are also working on a number of other products that complete the family of services we offer to API developers, and we can't wait for you to hear all the exciting announcements we are going to make over the course of the next few months!

Our Vision

Inside our office we have giant pictures of the first Ford Assembly Line, serving as our inspiration.

Over 100 years ago the second Industrial Revolution happened. Among many new inventions came two in particular: Electricity and The Assembly Line (thanks to electricity). This combination allowed Mass Production for the first time. The result was a dramatic decrease in the cost and time required to bring a product to market. Mass production quickly became a worldwide phenomenon, building companies worth billions. It changed the world.

We see a huge correlation in the software industry today: electricity is cloud computing, while the assembly line is APIs, where you pick up the components / services you need to build your product. This is history repeating itself and we want to be the pioneers of the next revolution, to change the world, yet again.


Assembly Line


If you have questions about the transition we have made and the lessons learned, or about Kong and our open source projects, please drop me a line at: ahmad@mashape.com - always happy to chat!

Kong
Powering API-Driven Software
Tools mentioned in article
Open jobs at Kong
Senior Site Reliability Engineer, Kon...
Sydney
Are you ready to power the World's connections? If you don’t think you meet all of the criteria below but are still interested in the job, please apply.  Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others. About the role: You will be responsible for contributing to a team that is building and maintaining workflows that automate releasing, testing and deploying Kong cloud products. Improving staging and production environments for SaaS distributions. Research and build monitoring/analyze tools to optimize building and deploying code-base, manage distributed systems and application resources. You will build workflows for delivering our software to a variety of platforms including AWS, GCP, Azure, and container technologies like Docker and Kubernetes to automate deployment and scale our products. Why should you want to work at Kong?   Market Opportunity - We are on a quest to build a $10b+ software company over the next few years and need YOUR help! - APIs are driving innovation across every industry! - Strong VC team, Series D, strong year over year revenue growth! Technical Leadership - We are recognized as the leader in innovation in the connectivity space. - Marco, our CTO/co-founder - “We are the Cisco of L4 and L7” - CUBE Conversation, March 2021 - We are the leading innovator in the connectivity space! Amazing Team & Culture - Come be a "Konger" and find out what we mean. - Great Place to Work Certified in 2020 & 2021 - Kong employees discuss our culture at our 2020 Company Kickoff Building Great Products - Learn why the world's largest companies love our tech! - Kong Named a Leader in the 2020 Gartner Magic Quadrant - Over 250m+ downloads of our open source API gateway!  Over 40k+ stars on Github between Kong API and Kong Insomnia! <li>BS degree in Computer Science, similar technical field of study or equivalent practical experience. </li><li>Located in Asia Pacific time zone.</li><li>Experience with continuous/rapid release engineering (CI/CD).</li><li>"Infrastructure as Code" configuration management systems such as Terraform, Chef, Puppet or Ansible. </li><li>Experience building and administering alerting and monitoring systems for API services.</li><li>Strong knowledge of Linux/Unix systems.</li><li>Knowledge of one or more mainstream programming languages (Go, C/C++, Python).</li><li>Strong skills in network services such as DNS, TLS/SSL, HTTP.</li><li>Experience working in a 24/7/365 service environment.</li><li>Design, implement, manage and orchestrate Kubernetes container clusters.</li><li>And any additional tasks required by manager.</li> <li>Experience implementing secure and highly-available distributed systems/microservices.</li><li>Professional experience managing production software in AWS.</li><li>Experience with PostgreSQL in multi-region configuration.</li><li>Experience with Apache Kafka, Datadog, ElasticSearch, Prometheus.</li><li>Experience using CD tools like ArgoCD.</li><li>Experience with Redis in multi-region configuration.</li>
Software Engineer, Sustaining - Shanghai
Shanghai
Are you ready to power the World's connections? If you don’t think you meet all of the criteria below but are still interested in the job, please apply.  Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others. About the role: This team is the technical interface with key customers of our service mesh product. You need a solid technical background, making valuable contributions on debugging tools, and providing escalation support, handling the tough technical issues for our customers. You will have the opportunity to influence the culture, facilitate technical delivery, and work with your team on strategy and execution. Why should you want to work at Kong?   Market Opportunity - We are on a quest to build a $10b+ software company over the next few years and need YOUR help! Why APIs Matter? APIs have been enabling innovation for decades! Strong VC team, Series D, strong year over year revenue growth! Technical Leadership - We are recognized as the leader in innovation in the connectivity space. Marco, our CTO/co-founder - “We are the Cisco of L4 and L7” - CUBE Conversation, March 2021 We are the leading innovator in the connectivity space! Amazing Team & Culture - Come be a "Konger" and find out what we mean. Great Place to Work Certified in 2020 & 2021 Kong employees exemplify our culture at our 2022 Sales Kickoff Building Great Products - Learn why the world's largest companies love our tech! Kong Named a Leader in the 2021 Gartner Magic Quadrant Over 250m+ downloads of our open source API gateway!  Over 40k+ stars on Github between Kong API and Kong Insomnia! <li>Work on the open-source Service Mesh Kuma using Envoy.</li><li>Work on the open core-based enterprise offering Kong Mesh.</li><li>Own the customer escalation system technically, including collaboration with other departments, stakeholders (Sales, Customer Experience, Sales Engineering, etc.), and where necessary customers.</li><li>Organize and prioritize complex problem resolution and bug fixes, debugging distributed services.</li><li>Potentially contribute to go-control-plane, Envoy, or any of our other OSS dependencies.</li><li>Contribute to documentation for Kuma and Kong Mesh.</li><li>Be active in the Kuma community.</li> <li>1+ years of support or development, obsessing over customer experience.</li><li>1+ years programming server-side applications and components (extra points if in Go, C++, Java, or Rust)</li><li>Deep understanding of networking (TCP, HTTP, TLS) and the Linux networking stack.</li><li>Experience working with Kubernetes building and operating operators, controllers, webhooks…</li><li>Proven customer relationship skills sets.</li><li>Business fluent English language skills both written and verbal.</li> <li>Understanding of Envoy and the XDS protocol.</li><li>Experience building or operating a Service Mesh.</li><li>Understanding of Windows Containers and Windows networking.</li><li>Experience with eBPF and CNI.</li><li>Experience in building and fostering an Open-Source community.</li>
Staff Technical Support Engineer - (B...
India
Are you ready to power the World's connections? If you don’t think you meet all of the criteria below but are still interested in the job, please apply.  Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others. About the role: You will be working at a hectic and fast paced Startup as an engineer in the Technical  Support Engineering (TSE) team. This team is responsible for assisting Kong customers in resolving complex technical issues in production and pre-production environments.  You will have the opportunity to work with your teammates and our customers to support many new, leading-edge technologies that solve real challenges. You will work to provide robust feedback and guidance to our Product and Engineering teams while being a voice for our customers, community, partners and the customer success team.. You want to make our customers and partners successful while strengthening their relationship with Kong. You can make a huge impact and have real ownership for the work you do. Why should you want to work at Kong?   Market Opportunity - We are on a quest to build a $10b+ software company over the next few years and need YOUR help! Why APIs Matter? APIs have been enabling innovation for decades! Strong VC team, Series D, strong year over year revenue growth! Technical Leadership - We are recognized as the leader in innovation in the connectivity space. Marco, our CTO/co-founder - “We are the Cisco of L4 and L7” - CUBE Conversation, March 2021 We are the leading innovator in the connectivity space! Amazing Team & Culture - Come be a "Konger" and find out what we mean. Great Place to Work Certified in 2020 & 2021 Kong employees exemplify our culture at our 2022 Sales Kickoff 2022 Forbes Cloud 100 Honoree Building Great Products - Learn why the world's largest companies love our tech! Kong Named a Leader in the 2021 Gartner Magic Quadrant Over 250m+ downloads of our open source API gateway!  Over 40k+ stars on Github between Kong API and Kong Insomnia! This opportunity is hybrid with 3 days in office and 2 days work from home <li>Working with top developers from around the world; helping to implement Kong Products and Technologies including our API Gateway and ServiceMesh solutions</li><li>Leading the investigation of complex technical puzzles and working collaboratively to understand and resolve customer issues</li><li>Creating knowledge articles to compliment our documentation and&nbsp; enable customers to better understand product use cases&nbsp;</li><li>Be the champion of our customers by providing real-time feedback to Kong about product performance and customer needsAssisting other customer facing teams including Education, Technical Account Managers with critical issues and ensuring all solutions and concepts we present are done with a single voice</li><li>Be a driver of innovation within the support team, the company and with our customers, by looking for better ways to do the work we do, improve the tools we use and the how we raise the bar on the service we deliver to our Global 5000 customers</li> <li>You are a voracious learner, who loves to dig into technical mysteries and have the creativity to solve complex technical problems</li><li>You can build relationships with like-minded teammates, smart customers, and global partners</li><li>Some of the skills that you will regularly use include:REST and RESTful APIs</li><li>Coding skills-&nbsp; (at a minimum read and troubleshoot code)</li><li>Containers, cluster deployment and management tools like&nbsp; Docker, Docker Swarm, Kubernetes and others</li><li>You know your way around web architectures including HTTP</li><li>8+ years of experience in a customer facing technical role</li> <li>Familiarity with DNS, Linux/Unix type systems and/or databases</li>
Senior Technical Support Engineer - (...
India
Are you ready to power the World's connections? If you don’t think you meet all of the criteria below but are still interested in the job, please apply.  Nobody checks every box - we’re looking for candidates that are particularly strong in a few areas, and have some interest and capabilities in others. About the role: (You will be working at a hectic and fast paced Startup as an engineer in the Technical Support Engineering (TSE) team. This team is responsible for assisting Kong customers in resolving complex technical issues in production and pre-production environments.  You will have the opportunity to work with your teammates and our customers to support many new, leading-edge technologies that solve real challenges. You will work to provide robust feedback and guidance to our Product and Engineering teams while being a voice for our customers. You want to make our customers and partners successful while strengthening their relationship with Kong. You can make a huge impact and have real ownership for the work you do. Why should you want to work at Kong?   Market Opportunity - We are on a quest to build a $10b+ software company over the next few years and need YOUR help! Why APIs Matter? APIs have been enabling innovation for decades! Strong VC team, Series D, strong year over year revenue growth! Technical Leadership - We are recognized as the leader in innovation in the connectivity space. Marco, our CTO/co-founder - “We are the Cisco of L4 and L7” - CUBE Conversation, March 2021 We are the leading innovator in the connectivity space! Amazing Team & Culture - Come be a "Konger" and find out what we mean. Great Place to Work Certified in 2020 & 2021 Kong employees exemplify our culture at our 2022 Sales Kickoff 2022 Forbes Cloud 100 Honoree Building Great Products - Learn why the world's largest companies love our tech! Kong Named a Leader in the 2021 Gartner Magic Quadrant Over 250m+ downloads of our open source API gateway!  Over 40k+ stars on Github between Kong API and Kong Insomnia! This opportunity is hybrid with 3 days in office and 2 days work from home <li>Working with top developers from around the world; helping to implement Kong Products and Technologies including our API Gateway and ServiceMesh solutions</li><li>Leading the investigation of complex technical puzzles and working collaboratively to understand and resolve customer issues</li><li>Creating knowledge articles to compliment our documentation and&nbsp; enable customers to better understand product use cases&nbsp;</li><li>Be a driver of innovation within the support team, the company and with our customers, by looking for better ways to do the work we do, improve the tools we use and the how we raise the bar on the service we deliver to our Global 5000 customers</li> <li>You are a voracious learner, who loves to dig into technical mysteries and have the creativity to solve complex technical problems</li><li>You can build relationships with like-minded teammates, smart customers, and global partners</li><li>Some of the skills that you will regularly use include:REST and RESTful APIs</li><li>Coding skills-&nbsp; (at a minimum read and troubleshoot code)</li><li>Containers, cluster deployment and management tools like&nbsp; Docker, Docker Swarm, Kubernetes and others</li><li>You know your way around web architectures including HTTP5+ years of experience in a customer facing technical role</li><li>A bachelor's degree in Computer Science/Computer Engineering&nbsp;</li> <li>Familiarity with DNS, Linux/Unix type systems and/or databases</li>
You may also like