What is a Microservice Catalog?

Microservices are helping developers satisfy internal and external customers by pushing out software faster with greater efficiency.

And while there have been important advancements to ease this way of building and organizing applications, challenges remain in enabling better visibility and collaboration to properly manage and gain the highest levels of effectiveness and efficiency from the architecture.

To better understand these challenges, let’s review how microservices gained a foothold in enterprise architecture.

The Evolution of the Microservice Architecture

The microservice architecture (microservices) provides both an alternative and a solution to some of the problems posed by the monolithic architecture. Even if it has been developed in a modular way, every time a monolithic application is called up on a web server, all of its other services are called up with it.

Source: https://martinfowler.com/articles/microservices.html

This meant that if one part of the application needs to scale to meet higher demand, everything else has to scale along with it. This can be clunky, inefficient, and expensive for many scenarios. What’s more, every time something new is ready to be pushed into production, everything else in the monolith needs to be tested to make sure it doesn’t break.

Advancements in virtualization technologies allowed for microservices – mini-applications or individual services – to be deployed on different virtual machines. Each of these microservices runs in a container on a virtual machine. That machine runs on a hypervisor on a server in a data center that is part of a network. The services are independent and work and talk to each other over the network by calling up each other’s REST APIs to get what they need to do their job when they need it.

This allows developers to make changes to a single service without changing or testing the whole application. It also allows the business to scale up certain parts of the service to accommodate spikes in demand without touching services that weren’t relevant to that job.

While some companies like Segment have adopted microservices, then moved away from them, they’ve generally become more popular over time:

Microservices often use container technology/tooling to package up their code and dependencies so that they can be deployed independently of the computing environment. This also means the same programming languages, database technologies, tooling and more don’t have to be used by everyone in the company. And a bi-product of all of this is that functional enhancements can be built faster and more efficiently.

But it also means that managing the underlying infrastructure can become incredibly complex. Kubernetes rose to meet this challenge, followed by other technologies like AWS Lambda, which works to abstract all of the underlying infrastructure so that developers can focus on the code. The technologies make it easy to manage thousands or hundreds of thousands of distributed containers and make them portable.

The end game is that it’s now easier to build and containerize microservices, but that also means it’s easier to build and deploy microservices. And so, we confront our next challenge: microservices sprawl.

Enter the Microservice Catalog

Microservices should be reused. In fact, the DevOps Institute says some 80% of them can be reused. But without visibility into what’s out there, it’s pretty hard to do that. And without a culture and tools that encourage collaboration and reuse, it’s almost impossible.

Source: https://devopsinstitute.com/blog-three-basic-truths-about-microservices/

Most IT monitoring tools weren’t built to provide and maintain visibility into containers – and can’t adequately monitor them once they move into production. If you don’t know all of the metadata around the service, who built it, where that person or team is located and how to get in touch with them, it’s hard to quickly remedy inevitable issues that arise with the services.

This lack of visibility presents many challenges to running thousands or hundreds of thousands of microservices effectively and efficiently – undermining the whole goal of the architecture to begin with.

There’s tons of great tooling out there for microservices. But to gain better visibility, teams need to track microservices in use in their organizations – including the name of the service, what it does, who owns it, who has worked on it, its dependencies, changes made, and more.

Microservice catalogs – both manual iterations and now software-enabled catalogs – aim to create this visibility and reduce production incidents, while providing a baseline for reusing microservices. Most tools provide the functionality to search for services and provide the complete audit history of that service. Catalogs erode problems that arise from tribal knowledge and prevent those midnight mad dashes to address services no one is familiar with or which no one knows who owns.

In the past couple of years, lots of viable tools to automate the process of building the catalog have launched. These include products like Effx, Cortex, OpsLevel and the open-source project Backstage.io. These products can connect to third-party application monitoring tools and on-call providers and offer some integration with collaboration technologies.

Why Enterprises Need More Than a Microservice Catalog

While microservice catalog tools provide a really necessary and valuable function, they don’t cover your entire tech stack, only the logical groupings of services that you’re running. To put it more simply: microservice catalogs show you what’s running but they typically don’t give you a full picture of all the technologies you’re using.

Catalogs can also grow quickly and become unmanageable. Not to mention, it’s difficult to keep metadata on every microservice up-to-date.

Microservice catalog tools were built to collect information, while enabling collaboration on that information is typically an afterthought. Though many now boast integration capabilities with third-party collaboration technologies, it’s not a native capability. There’s a risk the catalog software will be substituted for the organizational change management needed to actually be successful with microservices, and simply give visibility into the same silos without ever bringing them together.

For companies that built and attempt to maintain microservices catalogs in-house, the challenge is even greater. Every time a team wants to push a new microservice into production it must ensure everything is up to date in the catalog – taking valuable time away from their core roles.

Perhaps the biggest downside is that microservice catalogs haven’t been built for the Fortune 500s of the world. Catalogs are able to keep track of and monitor microservices, however most large enterprise teams have many different types of application architectures, not just microservices. The typical enterprise has monoliths, microservices, legacy stacks, deprecated technologies not running in production, and a whole host of other infrastructure and application components that microservice catalogs don’t cover.

In short, collaboration is a huge part of delivering a successful microservices architecture. How do you prioritize microservices? How do you track different databases and programming languages in use and determine the value? How do you know what you have across all your app architectures and whose job it is to maintain it? How can you see all the dependencies? Ensure versioning? And how do you encourage the reuse and maintenance of applications?

Private StackShare for Teams: The Enterprise Option

Enter Private StackShare for Teams.

StackShare was purpose-built for collaboration and visibility across all your tech stacks. Private StackShare provides secure and intuitive collaboration capabilities to manage tech stacks internally. Private StackShare instantly connects to your repos to visualize all of the engineering tools and services you’re using across your company in a single dashboard, who knows how to use them, and why they're using them. It’s like an "air traffic control" center for your tech stacks.

Any time someone that is part of your Private StackShare account merges in a pull request that contains a stack change in a connected repo, that change is automatically documented in your Feed and stack profiles in the form of a private Stack Decision and sent out in alerts via email and Slack. If you have multiple repos that all comprise a single application, you can reflect that in your dashboard and profiles.

Don’t let the challenges of visibility and difficulty of collaboration be a roadblock to realizing the value of microservices.

Check out Private StackShare for Teams for free here.