Joshua Dean Küpper

We've already been monitoring Agones for a few years now, but we only adapted Kubernetes in mid 2021, so we could never use it until then. Transitioning to Kubernetes has overall been a blast. There's definitely a steep learning curve associated with it, but for us, it was certainly worth it. And Agones plays definitely a part in it.

We previously scheduled our game servers with Docker Compose and Docker Swarm, but that always felt a little brittle and like a really "manual" process, even though everything was already dockerized. For matchmaking, we didn't have any solution yet.

After we did tons of local testing, we deployed our first production-ready Kubernetes cluster with #kubespray and deployed Agones (with Helm) on it. The installation was very easy and the official chart had just the right amount of knobs for us!

The aspect, that we were the most stunned about, is how seamless Agones integrates into the Kubernetes infrastructure. It reuses existing mechanisms like the Health Pings and extends them with more resource states and other properties that are unique to game servers. But you're still free to use it however you like: One GameServer per Game-Session, one GameServer for multiple Game-Sessions (in parallel or reusing existing servers), custom allocation mechanisms, webhook-based scaling, ... we didn't run into any dead ends yet.

One thing, that I was a little worried about in the beginning, was the SDK integration, as there was no official one for Minecraft/Java. And the two available inofficial ones didn't satisfy our requirements for the SDK. Therefore, we went and developed our own SDK and ... it was super easy! Agones does publish their Protobuf files and so we could generate the stubs with #Protoc. The existing documentation regarding Client-SDKs from Agones was a great help in writing our own documentation for the interface methods.

And they even have excellent tooling for testing your own SDK implementations. With the use of Testcontainers we could just spin up the local SDK testing image for each of the integration tests and could confirm that our SDK is working fine. We discovered a very small inconsistency for one of the interface methods, submitted an issue and a corresponding PR and it was merged within less than 24 hours.

We've now been using Agones for a few months and it has proven to be very reliable, easy to manage and just a great tool in general.

Our old ZNC IRC bouncer was succeeded by TheLounge , because it includes a Web-Interface, is actively maintained, has an out-of-the-box Docker image and overall just feels "cleaner" and more up-to-date.

It is responsive, offers modern chat features (like embeddings) and includes a search along with chat logs.

For our internal team and collaboration panel we use Nuxt.js (with TypeScript that is transpiled into ES6), Webpack and npm. We enjoy the opinionated nature of Nuxt.js over vanilla Vue.js, as we would end up using all of the components Nuxt.js incorporates anyways and we can adhere to the conventions setup by the Nuxt.js project, which allows us to get better support in case we run into any dead ends. Webpack allows us to create reproducable builds and also debug our application with hot reloads, which greately increased the pace at which we are able to perform and test changes. We also incorporated a lot of testing (ESLint, Chai, Jasmine, Nightwatchjs) into our pipelines and can trigger those jobs through GitLab CI. All packages are fetched through npm, so that we can keep our git repositories slim and are notified of new updates aswell as reported security flaws.

We use GraphQL for the communication between our Minecraft-Proxies/Load-Balancers and our global Minecraft-Orchestration-Service JCOverseer.

This connection proved to be especially challenging, as there were so many available options and very specific requirements and we tried our hardest to put as little complexity into this interface as possible.

Initially we considered designing our very own Netty based Packet-Protocol. While the performance of this approach probably would've been noteworthy, we would have had to write a lot of packets as the individual payloads would differ a lot and for the protocol specification a new project would've been needed, so we scrapped that idea.

Our second idea was to use a combination of Redis Key/Value store (in particular the ability to write whole, complex sets as the values of keys) for existing data, Redis Pub-Sub for the synchronization of new/changed/deleted data and a Vert.x based REST API for the mutation requests of the clients. While this would certainly have been possible, we decided against it, as redis offers no real other data types than strings and typing was important to us.

So we finally settled for GraphQL as it would allow us to define dynamic queries and mutations and additionally has subscriptions in store, so we would only need one component instead of three separate. The proxies register as subscribers to the server changes channel and fetch the current data set in advance. If they need to request changes, this is done through a mutation in GraphQL aswell.

The status of the invidiual servers is fetched through Docker healthchecks and a Docker client in the orchestration service, that subscribes to changed HEALTHINESS values in docker. If a service becomes unhealthy it is unregistered and synchronized through GraphQL. The healthcheck is comparable to a ping packet that expects a response in a given time frame.

As the access to our global REST-API "Charon" is bound to OAuth2, we use Keycloak inside Quarkus to authenticate and authorize users of our API. It is not possible to perform any un-authenticated requests against this API, so we wanted to make really sure that the authentication/authorization component is absolutely reliable and tested. We found those attributes within Keycloak, so we used it.

We use Quarkus with native compilation in GraalVM for our global REST-API "Charon", that can be used by every developer to request user, server and game-data (protected through OAuth2). Quarkus offers a reliable framework, library and stack for high-quality APIs and integrates Vert.x into its core.

GraalVM pushes the performance boundaries even further with the ability to perform ahead-of-time native compilation so we can reach an incredible small memory-footprint and fast bootup-times that we need for our microservices architecture.

We use Swagger Inspector in conjunction with our universal REST-API "Charon". Swagger Inspector makes testing edge-cases hassle-free and lets testing look easy. Swagger Inspector was also a great help to explore the Mojang-API, that we are dependent on, because it is the central repository for minecraft-account-data.

We previously used Postman but decided to switch over to Swagger Inspector because it also integrated seamlessly into Swagger UI, which we use for displaying our OpenAPI specification of said REST-API.

We use PMD alongside Checkstyle and FindBugs (Spotbugs) for our static code analysis, as a standard stage in all of our pipelines. PMD offers us insight into various optimization possibilities, best-practice alignment, coding convention compliance and general problems with our code.

OpenLDAP is pretty much unrivaled, when it comes to the LDAP-Protocol. Although it is not hard to spot, that the general structure looks a little bit anachronistic in a time of microservices, cloud-native applications and dockerization, it is very reliable, stable and still actively developed.

We use OpenLDAP within a docker-container und interact with it through another docker-container, housing phpLDAPadmin. LDAP is still the gold-standard regarding a centralized, digital directory of the company accounts, structures and permissions and that's why we use it.

It is necessary, however, to install extensions for some features like enhanced group queries, better password algorithms and a few other features, that I'd like to be shipped by default.

We use Checkstyle for all of our Java projects to perform the linting. The setup was very easy and there are amazing integrations into all IDEs. What also came in very handy was that we could start off with just a basic set of rules and tighten the linting-rules step-by-step, continously improving the readability and uniformity of our codebase.