etcd vs Zookeeper: What are the differences?
Developers describe etcd as "A distributed consistent key-value store for shared configuration and service discovery". etcd is a distributed key value store that provides a reliable way to store data across a cluster of machines. It’s open-source and available on GitHub. etcd gracefully handles master elections during network partitions and will tolerate machine failure, including the master. On the other hand, Zookeeper is detailed as "Because coordinating distributed systems is a Zoo". A centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications.
etcd and Zookeeper can be primarily classified as "Open Source Service Discovery" tools.
"Service discovery" is the primary reason why developers consider etcd over the competitors, whereas "High performance ,easy to generate node specific config" was stated as the key factor in picking Zookeeper.
etcd is an open source tool with 25.5K GitHub stars and 5.18K GitHub forks. Here's a link to etcd's open source repository on GitHub.
Shopify, SendGrid, and hike are some of the popular companies that use Zookeeper, whereas etcd is used by CNCFlora, Beam, and Giant Swarm. Zookeeper has a broader approval, being mentioned in 116 company stacks & 48 developers stacks; compared to etcd, which is listed in 27 company stacks and 11 developer stacks.
What is etcd?
What is Zookeeper?
Need advice about which tool to choose?Ask the StackShare community!
Sign up to add, upvote and see more prosMake informed product decisions
What are the cons of using etcd?
What are the cons of using Zookeeper?
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions
Initially, Stitch only supported real-time updates and addressed this problem with a MapReduce job named The Restorator that performed the following actions:
- Calculated the expected totals
- Queried Cassandra to get the values it had for each counter
- Calculated the increments needed to apply to fix the counters
- Applied the increments
Meanwhile, to stop the sand shifting under its feet, The Restorator needed to coordinate a locking system between itself and the real-time processors, so that the processors did not try to simultaneously apply increments to the same counter, resulting in a race-condition. It used ZooKeeper for this.
Like many large scale web sites, Pinterest’s infrastructure consists of servers that communicate with backend services composed of a number of individual servers for managing load and fault tolerance. Ideally, we’d like the configuration to reflect only the active hosts, so clients don’t need to deal with bad hosts as often. ZooKeeper provides a well known pattern to solve this problem.
Zookeeper manages our state, and tells each node what version of code it should be running.
Used Zookeeper as the resource management system for Mesos/Marathon services.