I think I've tried most of the CI tools out there at some point. It took me a while to get around to Buildkite because at first I didn't see much point given it seemed like you had to run the agent yourself. Eventually it dawned on me why this approach was more ingenious than I realised:

Running my app in a production (or production-like) environment was already a solved problem, because everything was already in some form of "everything as code". Having a test environment where the only difference was adding the Buildkite agent was a trivial addition.

It means that dev/test/prod parity is simple to achieve and maintain. It's also proven to be much easier to support than trying to deal with the problems that come with trying to force an app to fit into the nuances and constraints that are imposed by the containers/runtime of a CI service. When you completely control all of the environment the tests are running in you define those constraints too. It's been a great balance between a managed service and the flexibility of running it yourself.

And while none of my needs have hit the scale of Shopify (I saw one of their engineers speak about it at a conference once, I can't find the video now though 😞) it's good to know I can scale out my worker nodes to hundreds of thousands of workers to reduce the time it takes for my tests to run.

I would recommend you to consider the JFrog Platform that includes JFrog Pipelines - it will allow you to manage the full artifact life cycle for your sbt, docker and other technologies, and automate all of your CI and CD using cloud native declarative yaml pipelines. Will integrate smoothly with all your other toolset.

more configurable to setup ci/cd: * It can provide caching when build sbt, just add this section to yml file * Easy to use, many documentation

Weakness: * Need use gitlab as repository to bring more powerful configuration