Breaking a monolith into microservices and handling the scaling and health of new services as they come only. This should ideally help to reduce the overhead needed to get a service online. We have all of this being handled by custom URLs and health checks being done at the expense of infrastructure setup time and maintenance (VM sprawl). Initially, I am looking at Consul for the TLS proxy and security options as well as the KV store which may prove useful in cross datacenter environments.
What is etcd?
What is Eureka?
Sign up to add, upvote and see more prosMake informed product decisions
What are the cons of using etcd?
What are the cons of using Eureka?
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions
Initially, Stitch only supported real-time updates and addressed this problem with a MapReduce job named The Restorator that performed the following actions:
- Calculated the expected totals
- Queried Cassandra to get the values it had for each counter
- Calculated the increments needed to apply to fix the counters
- Applied the increments
Meanwhile, to stop the sand shifting under its feet, The Restorator needed to coordinate a locking system between itself and the real-time processors, so that the processors did not try to simultaneously apply increments to the same counter, resulting in a race-condition. It used ZooKeeper for this.
Like many large scale web sites, Pinterest’s infrastructure consists of servers that communicate with backend services composed of a number of individual servers for managing load and fault tolerance. Ideally, we’d like the configuration to reflect only the active hosts, so clients don’t need to deal with bad hosts as often. ZooKeeper provides a well known pattern to solve this problem.
Zookeeper manages our state, and tells each node what version of code it should be running.
Used Zookeeper as the resource management system for Mesos/Marathon services.