What is Zookeeper?
Who uses Zookeeper?
Why developers like Zookeeper?
Here are some stack decisions, common use cases and reviews by companies and developers who chose Zookeeper in their tech stack.
In early 2013, Airbnb tackled the problem of service discovery and load balancing in the context of a service oriented architecture (SOA) by building and releasing an open source tool called SmartStack. SmartStack is built on two other open source tools created by Airbnb called Nerve and Synapse.
Nerve is a service registration daemon that performs health checks that “creates ephemeral nodes in Zookeeper which contain information about the address/port combos for a backend available to serve requests for a particular service.”
Synapse is a transparent service discovery framework for connecting an SOA that reads the information in Zookeeper for available backends, and then uses that information to configure a local HAProxy process, which then routes requests between clients and services.
Initially, Stitch only supported real-time updates and addressed this problem with a MapReduce job named The Restorator that performed the following actions:
- Calculated the expected totals
- Queried Cassandra to get the values it had for each counter
- Calculated the increments needed to apply to fix the counters
- Applied the increments
Meanwhile, to stop the sand shifting under its feet, The Restorator needed to coordinate a locking system between itself and the real-time processors, so that the processors did not try to simultaneously apply increments to the same counter, resulting in a race-condition. It used ZooKeeper for this. Zookeeper