How Mashape Manages Over 15,000 APIs & Microservices

12,306

Editor's note: By Ahmad Nassri, ‎Head of Engineering at Mashape


Background

Mashape powers API-driven software. Hundreds of thousands of developers use our tools to manage, monitor, consume and provide APIs to their partners, apps, customers and employees. In fact, billions of API requests are processed through Mashape's Marketplace every month.

I'm Ahmad Nassri, Head of Engineering at Mashape. Prior to Mashape I was the Development Manager at The Canadian Broadcasting Corporation (CBC ) and spent many years building Web, Mobile & API products across many startups and businesses.

Engineering Team

The Mashape engineering team is one of the most highly skilled teams I've had the pleasure to work with. While we operate as a single team, our responsibilities are split and organized per project. Currently we have three main projects:

As our team is spread out among San Francisco, Toronto, Montreal, Paris and London, we have faced a number of challenges in scheduling, communications and syncing. To overcome these issues, we follow the open source model of building software and collaborating; ensuring that all communication is clear and verbose, assuming asynchronousity, and keeping a communication channel open at all times on Flowdock, while regularly jumping on Google Hangouts whenever further discussion is needed.

Despite having a focus of responsibilities, the engineering team members are not isolated in silos. We encourage everyone to work and touch different parts of the codebase, across all projects; in fact, most of us will end up working on Mashape Analytics, Marketplace and Mashape ID all in the same week.

Mashape's design team interacts with the engineers on a daily basis. By compiling engineer comments and gathering feedback from users, we're able to design the best Developer Experience (DX) by focusing on our users first. It also helps that we are engineers and developers!

We have regular one hour conference calls every Monday where the previous week's work is reviewed, and new objectives for the coming week are established.


Engineering Team (API marketplace team meeting, mid 2014)


General Architecture

Once upon a time, Mashape's architecture consisted of around 150K lines of Java, a true Monolith! But it's now split into 400 different microservices built with a variety of languages including: Node.js, Lua, Ruby, Java, Python, and OCaml. Everything is an API.

Mashape ID, as an example, is an identity authentication platform operating entirely as an API. Any time a new product is created, we simply plug it into Mashape ID; which provides out of the box account & email management with a companion billing microservice of its own.

One of our new projects is powered by a Ruby API, while the frontend app is built in Ember.js. Conversely, Mashape's marketplace has Java-powered internal APIs which in turn are consumed by a number of Express.js microservices serving as the web frontend; all our applications & APIs follow The 12-factor methodology.

In front of all of our applications and APIs is Kong. We use ElasticSearch for search while also storing data in a number of databases including: MongoDB, PostgreSQL, Redis and Cassandra (each solving a different problem domain). All of our products run on AWS, though on occasion we use Heroku and DigitalOcean for smaller projects to experiment with prototypes or test environments. Splunk is our centralized logging system, Datadog as a unified monitoring dashboard, and PagerDuty for alerts.

Mashape pushes business intelligence metrics from all our products directly into Chart.io. Additionally, we dogfood quite a lot by running Mashape Analytics to monitor and visualize API traffic for the APIs managed by our platforms. TravisCI is utilized for testing, CodeClimate for quality control, while using Chef for deploys. All our code is on Github. For code snippet generation we use our own HTTPsnippet and Mockbin for mocking & testing API prototypes.

All of these applications and systems, alongside all the deployment, monitoring & metrics tools may seem like a lot, but the microservices approach coupled with the 12-factor methodology allows for zen & simplicity in a sea of chaos!

During the course of a slow week we have ~400 internal services running over a cluster of hundreds of EC2 machines (consisting of mainly large instances). Having a microservice architecture allowed us to move much faster and independently, we clearly saw the benefits of the transition. But it's probably helpful to understand why we made the transition.

Scaling challenges

Sometime around 2013, we experienced scaling issues with the dozens of standalone third-party APIs added to Mashape's Marketplace.

Between July 2012 and July 2013 our volume in API transactions grew 50X.


Scaling Problems


Not only was the volume of requests passing through our proxy taking off, Mashape was starting to manage more APIs than anyone else in the market. At that time, the Marketplace had 3,500 public APIs under its belt.

The Marketplace needed to scale fast. Our API proxy had been built in Node.js, with a heavy Java backend. It was a monolithic architecture, and our Node.js proxy could not handle large traffic spikes.

Overtime it became unpredictable and resulted in consuming larger amounts of AWS resources. Our bank account suffered as a consequence and our CEO lost some hair.

We had to find a solution that satisfied not only the massive amount of traffic, but also be best in class in reliability and security. We wanted to move away from a monolithic architecture towards something much more flexible.


Architecture


We've always had a desire to create our own API gateway; targeting the following attributes:

  1. Predictable stability over months and years, not days.
  2. Efficient and easy to scale across multiple geographic regions with little overhead.
  3. Modular and extensible via simple plugins.
  4. Independent: environment, framework & language agnostic.
  5. Easily configurable via a RESTful API.

Basically an "ElasticSearch" for API management. Our CTO began his search and eventually met an engineer at CloudFlare, who introduced us to the amazing world of OpenResty. Built on top of Nginx it allows a developer to extend Nginx functionalities with Lua, an embedded scripting language used by many heavy traffic websites.

CloudFlare built Lua scripts around security and caching. We thought we could do the same but with a focus on API and microservice management. Nginx was already proven worldwide as a stable and trustworthy HTTP server, and was perfect for our base requirements. In combination with OpenResty, we started building Kong.

Kong is born (the API Gateway)

In 2013, Mashape adopted Nginx / OpenResty to implement features on the Marketplace's published APIs. Billing, authentication, throttling and rate-limiting became custom OpenResty plugins written in Lua. We called it Kong (yes, like "King Kong", after our own namesake: MashAPE).


King Kong


Today Kong efficiently manages over 15,000 APIs. Serving both our internal microservices and our Marketplace clients' APIs, all while handling spikes, high concurrent scenarios, and billions of successful API calls per month.

We created our own "ElasticSearch" for API management; much like Elastic's approach to wrapping Lucene in a RESTful interface with some sugar on top, Kong was a wrapper around Nginx.

Below are some examples of Kong's operations within the Marketplace.

Once an API is added to the Mashape platform from the GUI, it sends a POST request to Kong:

curl -i -X POST \
  --url http://localhost:8001/apis/ \
  --data 'name=mockbin' \
  --data 'upstream_url=http://mockbin.com/' \
  --data 'request_host=mockbin.com'

Then, based on the API provider's needs, a collection of various plugins are added on top of it. As an example, if the user needs Key Authentication:

curl -i -X POST \
  --url http://localhost:8001/apis/mockbin/plugins/ \
  --data 'name=key-auth'

User management is a breeze; the Consumer object allows for an abstract entity that can represent Users, Applications, Clients, or any system accessing the API. They are easily created and managed through the RESTful interface.

curl -i -X POST \
  --url http://localhost:8001/consumers/ \
  --data "username=Jason"

Kong's RESTful API allows us to manage thousands of APIs in the Marketplace. With simple and single-focused codebases, we keep our Microservices and individual systems completely independent. Kong itself, by its pluggable nature, can be described as a paragon of microservice architecture. At its core, it consists of no more than a couple thousands lines of code, which handle database abstraction, routing and plugin management. Plugins can live in different code bases and inject themselves anywhere into the lifecycle of a proxied request in a few lines of code. We are very proud of this approach which is quite unique in API management gateways today.

Kong was released to the public in April 2015 - after being partly rewritten for the occasion - and has built a strong community of users and contributors since then. We are very grateful for the contributions brought by developers all around the world, and feel rewarded for our decision to open-source Kong, all the while contributing back to the community by improving many Lua packages used in OpenResty.

We generate Kong packages for many Linux distributions and cloud providers to allow ease of installation and deployment. The most successful distribution so far is Kong on Docker with over 180,000 downloads since it's release.

The Road Ahead

Every month, we release more pieces of Kong to the open-source realm as part of our commitment to drive and grow the open-source API developer community. We are working hard to make Kong reach the 1.0 public milestone so that everybody can enjoy what this internal tool brought to our company and our community.

We are also working on a number of other products that complete the family of services we offer to API developers, and we can't wait for you to hear all the exciting announcements we are going to make over the course of the next few months!

Our Vision

Inside our office we have giant pictures of the first Ford Assembly Line, serving as our inspiration.

Over 100 years ago the second Industrial Revolution happened. Among many new inventions came two in particular: Electricity and The Assembly Line (thanks to electricity). This combination allowed Mass Production for the first time. The result was a dramatic decrease in the cost and time required to bring a product to market. Mass production quickly became a worldwide phenomenon, building companies worth billions. It changed the world.

We see a huge correlation in the software industry today: electricity is cloud computing, while the assembly line is APIs, where you pick up the components / services you need to build your product. This is history repeating itself and we want to be the pioneers of the next revolution, to change the world, yet again.


Assembly Line


If you have questions about the transition we have made and the lessons learned, or about Kong and our open source projects, please drop me a line at: ahmad@mashape.com - always happy to chat!

Web/UI Engineer (Marketing)
San Francisco, CA
Are you ready to join the API revolution? We are a group of makers, thinkers, and doers focused on helping today’s developers build tomorrow’s technology. Our teams work on the bleeding edge of API innovation to provide our users with a central nervous system for data and services. We put design at the heart of everything we do, and we’re relentlessly focused on creating beautiful experiences for our customers. That’s why technology companies, major banks, e-commerce innovators, and government agencies put Kong in front of their most important web applications. We believe in the power of Open Source and everything it stands for. That’s why developers around the world enthusiastically contribute on top of our open-source platform. We are passionate about solving challenges that will fundamentally shape the future of technology, and we’re looking for the right people to join us on our mission. If you believe in taking ownership of your work, making an impact, and having fun along the way, we would love to talk to you. You are a front-end web engineer who is motivated to combine the art of design with the art of programming. Wielding a toolbox of modern web development skills and the eye of an artist, you deliver clean maintainable web properties and collateral that is pixel-perfect and pleasing the eye.
  • Produce visual elements with pixel perfect accuracy
  • Produce astonishing web animations with clean CSS
  • Create responsive output so the resulting  in optimized web/mobile pages for all screen
  • Bridge the gap between graphical design and technical implementation
  • Own email marketing design and the limitations and pitfalls of delivering content via email
  • Weigh in on design decisions and “just do the right thing” when it comes to pleasing layout, graphical and color choices
  • Mastery of web slicing and animations
  • Familiar with the JAMstack https://jamstack.org/
  • Good understanding of how to create web/mobile sites optimized for speed and readability
  • Demonstrated JS, CSS, template and HTML layout skills. You’re the king of DOM.
  • Sass, Less are your good friends
  • Experience with Wordpress, Jekyll and web frameworks
  • Understanding of caching, image optimization, compression and minimizing HTTP requests
  • Demonstrated experience with email templates and successful cross browser email delivery
  • Skill in Vue, or other modern JS component frameworks
  • Understanding of APIs and API gateways
  • Comments
    Open jobs at Kong
    Customer Support Engineer

    Description

    We're looking for a few great people who want to be part of a fun, highly-collaborative team that has created the fastest growing, and most popular next generation platform for API & Microservices management. Our ideal teammate will thrive in our environment of really smart, driven people, who get to work with many new, leading-edge technologies that solve really big customer challenges. We're looking for people who want to make a huge impact and have real ownership for the work they do. This opportunity is ideal for someone who wants to be in on the ground floor of a special startup company.


    As a Support Engineer you will:

    • Respond to complex technical issues generated by our Enterprise customers through tickets, emails, or phone calls
    • Liaison between Customers and Engineering to resolve complex customer cases and recommend solutions
    • Provide real-time feedback to the Development team about product performance and desired enhancements
    • Take a leadership role in building out Mashape’s Support model, approach and tools -- with the eye toward delivering an exceptional customer experience

    Requirements

    • 3+ yrs experience supporting large clients, bonus if it was a fast-growing Open Source Company
    • Engineering background with a solid understanding of micro-services, API architectures, and networking
    • Curiosity and a strong drive to learn the ins and outs of a complicated technical product
    • Love for explaining technical concepts and the moment where you see things “click”
    • Strong debugging and troubleshooting skills
    • Enthusiasm for working in a high-profile, fast-paced start-up environment


    Desired Experience:

    • Knowledge of JSON and experience with at least one language amongst Node.js, Ruby, Lua, Python or Java
    • Knowledge of networking principles (HTTP/s, Load Balancers, etc) and web server technologies like NGINX/HAProxy.
    • Docker experience (Docker Compose and Swarm a strong plus).
    • Solid experience with the terminal, and CLI HTTP clients: cURL and/or HTTPie
    • Knowledge of AWS (EC2, CloudFormation, AWS RDS, AWS Lambda, etc)
    • Other Open Source technologies including PostgreSQL, Redis and Cassandra a plus.

    Benefits

    Benefits

    • Competitive salaries and benefits, including medical, dental, and vision insurance
    • Vacation days, time off and a flexible working environment
    • The right tools to do your work, and really cool technology in our day to day work
    • We feed our team well, including Italian Food cooked by the founders, free lunch a couple of times a week
    • We have fun! Poker nights, company dinners, happy hours and more!
    • We are your second family -- whatever the joys and challenges outside of the work, we’ll be there
    • And much, much more…
    • As an early employee you will get a true stake in the company



    -------------------------------------------------------------------------------------------------------------------------------------------------------

    Mashape is an Equal Employment employer committed to the principles of equal employment opportunity and affirmative action for all applicants and employees. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status or any other basis protected by federal, state or local law, ordinance or regulation.

    Sales Engineer/Solution Architect

    Description

    Mashape is looking for a seasoned, customer facing software Sales Engineer/Solution Architect who is motivated to help drive new business, evangelize our technology, onboard and train new customers, and establish revenue potential with existing customers. Ideally this candidate should have experience working with open source technology and strong knowledge of NGINX, Docker, HTTP, AWS, Postgres and Cassandra.


    • Support field sales in helping customers understand the value of the Mashape API Management platform
    • Understand and help establish customer requirements and grow trusted client relationships throuhgout the entire sales lifecycle, from initial product reviews to production
    • Support PoCs throghout the entire evaluation process
    • Ensure 100% customer satisfaction across assigned set of accounts
    • Prioritize customer needs and coordinate with support, product, and engineering teams to help support customer needs
    • Create technical documentation for DevOps, implementations, and services engagements

    Requirements

    • 2+ years software engineer and/or sales engineer management experience at SAAS, Open Source or Enterprise Software company.
    • Solid understanding of cloud computing trends and open source business models.
    • You are a strong listener and communicator.
    • You can confidently lead a discussion with a group of Product Managers, Engineers, and Executives.
    • Enthusiasm for working in a high profile, fast-paced startup culture

    Benefits

    - As an early employee you will get a true stake in the company

    - Work in a fun, colloborative environment

    - Competitive salaries

    - Apple equipment

    - Free lunch 2 times a week.

    - Vacation days & time off

    - Medical, dental, & vision insurance

    - 401k

    - Company dinners & happy hour

    - Much more…

    Dev Ops Engineer

    Description

    Mashape is looking for an engineer excellent at scaling systems, networks and servers. You should have strong knowledge of HTTP, AWS, Cassandra, Nginx, and being able to create scripts in Python/Ruby. The Mashape proxy processes many hundreds of millions of requests and many TB of data; therefore, a 1 second, or even 500ms, of downtime greatly affects our API providers and API consumers. You will be responsible for designing and building systems that make Mashape the most reliable service in the world. Additionally, you will monitor and automate every aspect of the production system. Lastly, in a previous position, you made a substantial contribution to a widely used open source project and reduced the company's infrastructure costs.

    Requirements

    Strong experience with AWS

    Strong experience with Cassandra, and being capable of setting up and operating a multi-region Cassandra cluster

    Nginx is your best friend

    Expert in building automated deployment and scaling systems

    Experience with Ansible is a strong plus

    Strong Understanding of HTTP and HTTP caching systems (Varnish, Squid)

    Passionate about designing infrastructures

    Experience in scaling APIs to billion of calls is a big plus

    Experience with ElasticSearch and Splunk is a plus

    Benefits

     - As an early employee you will get a true stake in the company

     - Competitive salaries 

     - Poker nights 

     - Apple equipments 

     - Italian Food cooked by the founders

     - Free lunch 2 times a week.

     - Official mashaper 

     - 1 week/year in a spiritual retreat with the team somewhere in the world

     - A 2nd family  - whatever problem outside of the work, we’ll be there

     - Vacation days & time off

     - Awesome medical, dental, & vision insurance

     - 401k

     - Company dinners & happy hour 

     - Much more…

    Sales Engineer (API/Microservices)

    Description

    Mashape is looking for a seasoned, customer facing software Sales Engineer/Solution Architect who is motivated to help drive new business, evangelize our technology, onboard and train new customers, and establish revenue potential with existing customers. Ideally this candidate should have experience working with open source technology and strong knowledge of NGINX, Docker, HTTP, AWS, Postgres and Cassandra.


    • Support field sales in helping customers understand the value of the Mashape API Management platform
    • Understand and help establish customer requirements and grow trusted client relationships throuhgout the entire sales lifecycle, from initial product reviews to production
    • Support PoCs throghout the entire evaluation process
    • Ensure 100% customer satisfaction across assigned set of accounts
    • Prioritize customer needs and coordinate with support, product, and engineering teams to help support customer needs
    • Create technical documentation for DevOps, implementations, and services engagements

    Requirements

    • 2+ years software engineer and/or sales engineer management experience at SAAS, Open Source or Enterprise Software company.
    • Solid understanding of cloud computing trends and open source business models.
    • You are a strong listener and communicator.
    • You can confidently lead a discussion with a group of Product Managers, Engineers, and Executives.
    • Enthusiasm for working in a high profile, fast-paced startup culture

    Benefits

    - As an early employee you will get a true stake in the company

    - Work in a fun, colloborative environment

    - Competitive salaries

    - Apple equipment

    - Free lunch 2 times a week.

    - Vacation days & time off

    - Medical, dental, & vision insurance

    - 401k

    - Company dinners & happy hour

    - Much more…

    Verified by
    Avatar
    Head of Engineering
    94904
    Director of Ecosystem
    You may also like
    E-Commerce at Scale: Inside Shopify's Tech Stack
    How SendGrid Scaled to 40 Billion Emails Per Month
    How Stream Built a Modern RSS Reader With JavaScript
    How Heap Built an Analytics Platform that Auto-Tracks Every User Event