Alternatives to Buildbot logo

Alternatives to Buildbot

Jenkins, TeamCity, GitLab CI, Bitbucket, and CircleCI are the most popular alternatives and competitors to Buildbot.
70
116
+ 1
27

What is Buildbot and what are its top alternatives?

BuildBot is a system to automate the compile/test cycle required by most software projects to validate code changes. By automatically rebuilding and testing the tree each time something has changed, build problems are pinpointed quickly, before other developers are inconvenienced by the failure.
Buildbot is a tool in the Continuous Integration category of a tech stack.
Buildbot is an open source tool with 4.8K GitHub stars and 1.6K GitHub forks. Here’s a link to Buildbot's open source repository on GitHub

Top Alternatives to Buildbot

  • Jenkins
    Jenkins

    In a nutshell Jenkins CI is the leading open-source continuous integration server. Built with Java, it provides over 300 plugins to support building and testing virtually any project. ...

  • TeamCity
    TeamCity

    TeamCity is a user-friendly continuous integration (CI) server for professional developers, build engineers, and DevOps. It is trivial to setup and absolutely free for small teams and open source projects. ...

  • GitLab CI
    GitLab CI

    GitLab offers a continuous integration service. If you add a .gitlab-ci.yml file to the root directory of your repository, and configure your GitLab project to use a Runner, then each merge request or push triggers your CI pipeline. ...

  • Bitbucket
    Bitbucket

    Bitbucket gives teams one place to plan projects, collaborate on code, test and deploy, all with free private Git repositories. Teams choose Bitbucket because it has a superior Jira integration, built-in CI/CD, & is free for up to 5 users. ...

  • CircleCI
    CircleCI

    Continuous integration and delivery platform helps software teams rapidly release code with confidence by automating the build, test, and deploy process. Offers a modern software development platform that lets teams ramp. ...

  • GitLab
    GitLab

    GitLab offers git repository management, code reviews, issue tracking, activity feeds and wikis. Enterprises install GitLab on-premise and connect it with LDAP and Active Directory servers for secure authentication and authorization. A single GitLab server can handle more than 25,000 users but it is also possible to create a high availability setup with multiple active servers. ...

  • Bamboo
    Bamboo

    Focus on coding and count on Bamboo as your CI and build server! Create multi-stage build plans, set up triggers to start builds upon commits, and assign agents to your critical builds and deployments. ...

  • Concourse
    Concourse

    Concourse's principles reduce the risk of switching to and from Concourse, by encouraging practices that decouple your project from your CI's little details, and keeping all configuration in declarative files that can be checked into version control. ...

Buildbot alternatives & related posts

Jenkins logo

Jenkins

47.5K
39.8K
2.2K
An extendable open source continuous integration server
47.5K
39.8K
+ 1
2.2K
PROS OF JENKINS
  • 520
    Hosted internally
  • 464
    Free open source
  • 314
    Great to build, deploy or launch anything async
  • 243
    Tons of integrations
  • 210
    Rich set of plugins with good documentation
  • 110
    Has support for build pipelines
  • 72
    Open source and tons of integrations
  • 65
    Easy setup
  • 62
    It is open-source
  • 54
    Workflow plugin
  • 11
    Configuration as code
  • 10
    Very powerful tool
  • 9
    Many Plugins
  • 9
    Continuous Integration
  • 8
    Great flexibility
  • 8
    Git and Maven integration is better
  • 7
    100% free and open source
  • 6
    Github integration
  • 6
    Slack Integration (plugin)
  • 5
    Easy customisation
  • 5
    Self-hosted GitLab Integration (plugin)
  • 4
    Docker support
  • 4
    Pipeline API
  • 3
    Platform idnependency
  • 3
    Excellent docker integration
  • 3
    Fast builds
  • 3
    Hosted Externally
  • 2
    Customizable
  • 2
    AWS Integration
  • 2
    It's Everywhere
  • 2
    JOBDSL
  • 2
    Can be run as a Docker container
  • 2
    It`w worked
  • 1
    Easily extendable with seamless integration
  • 1
    Build PR Branch Only
  • 1
    NodeJS Support
  • 1
    PHP Support
  • 1
    Ruby/Rails Support
  • 1
    Universal controller
  • 1
    Loose Coupling
CONS OF JENKINS
  • 12
    Workarounds needed for basic requirements
  • 9
    Groovy with cumbersome syntax
  • 7
    Plugins compatibility issues
  • 6
    Lack of support
  • 6
    Limited abilities with declarative pipelines
  • 4
    No YAML syntax
  • 3
    Too tied to plugins versions

related Jenkins posts

Tymoteusz Paul
Devops guy at X20X Development LTD · | 23 upvotes · 5.1M views

Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).

It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up or vagrant reload we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.

I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.

We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.

If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.

The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).

Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.

See more
Thierry Schellenbach

Releasing new versions of our services is done by Travis CI. Travis first runs our test suite. Once it passes, it publishes a new release binary to GitHub.

Common tasks such as installing dependencies for the Go project, or building a binary are automated using plain old Makefiles. (We know, crazy old school, right?) Our binaries are compressed using UPX.

Travis has come a long way over the past years. I used to prefer Jenkins in some cases since it was easier to debug broken builds. With the addition of the aptly named “debug build” button, Travis is now the clear winner. It’s easy to use and free for open source, with no need to maintain anything.

#ContinuousIntegration #CodeCollaborationVersionControl

See more
TeamCity logo

TeamCity

1.1K
978
307
TeamCity is an ultimate Continuous Integration tool for professionals
1.1K
978
+ 1
307
PROS OF TEAMCITY
  • 60
    Easy to configure
  • 37
    Reliable and high-quality
  • 31
    On premise
  • 31
    User friendly
  • 31
    Github integration
  • 18
    Great UI
  • 16
    Smart
  • 12
    Can run jobs in parallel
  • 12
    Free for open source
  • 8
    Crossplatform
  • 4
    Chain dependencies
  • 4
    REST API
  • 4
    Great support by jetbrains
  • 4
    Fully-functional out of the box
  • 4
    Projects hierarchy
  • 3
    100+ plugins
  • 3
    Free for small teams
  • 3
    Per-project permissions
  • 3
    Personal notifications
  • 3
    Build templates
  • 2
    Ide plugins
  • 2
    GitLab integration
  • 2
    Upload build artifacts
  • 2
    Artifact dependencies
  • 2
    Smart build failure analysis and tracking
  • 2
    Build progress messages promoting from running process
  • 1
    Built-in artifacts repository
  • 1
    Powerful build chains / pipelines
  • 1
    TeamCity Professional is FREE
  • 1
    Repository-stored, full settings dsl with ide support
  • 0
    Official reliable support
  • 0
    Hosted internally
  • 0
    High-Availability
CONS OF TEAMCITY
  • 2
    User friendly
  • 1
    Proprietary
  • 1
    High costs for more than three build agents
  • 1
    User-friendly

related TeamCity posts

Tymoteusz Paul
Devops guy at X20X Development LTD · | 23 upvotes · 5.1M views

Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).

It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up or vagrant reload we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.

I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.

We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.

If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.

The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).

Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.

See more
Sarah Elson
Product Growth at LambdaTest · | 4 upvotes · 365.8K views

@producthunt LambdaTest Selenium JavaScript Java Python PHP Cucumber TeamCity CircleCI With this new release of LambdaTest automation, you can run tests across an Online Selenium Grid of 2000+ browsers and OS combinations to perform cross browser testing. This saves you from the pain of maintaining the infrastructure and also saves you the licensing costs for browsers and operating systems. #testing #Seleniumgrid #Selenium #testautomation #automation #webdriver #producthunt hunted

See more
GitLab CI logo

GitLab CI

2K
1.4K
71
GitLab integrated CI to test, build and deploy your code
2K
1.4K
+ 1
71
PROS OF GITLAB CI
  • 22
    Robust CI with awesome Docker support
  • 12
    Simple configuration
  • 8
    All in one solution
  • 6
    Source Control and CI in one place
  • 5
    Easy to configure own build server i.e. GitLab-Runner
  • 5
    Integrated with VCS on commit
  • 4
    Free and open source
  • 2
    Hosted internally
  • 1
    Pipeline could be started manually
  • 1
    Built-in support of Review Apps
  • 1
    Enable or disable pipeline by using env variables
  • 1
    Gitlab templates could be shared across logical group
  • 1
    Easy to setup the dedicated runner to particular job
  • 1
    Built-in Docker Registry
  • 1
    Built-in support of Kubernetes
CONS OF GITLAB CI
  • 2
    Works best with GitLab repositories

related GitLab CI posts

Joshua Dean Küpper
CEO at Scrayos UG (haftungsbeschränkt) · | 20 upvotes · 361.8K views

We use GitLab CI because of the great native integration as a part of the GitLab framework and the linting-capabilities it offers. The visualization of complex pipelines and the embedding within the project overview made Gitlab CI even more convenient. We use it for all projects, all deployments and as a part of GitLab Pages.

While we initially used the Shell-executor, we quickly switched to the Docker-executor and use it exclusively now.

We formerly used Jenkins but preferred to handle everything within GitLab . Aside from the unification of our infrastructure another motivation was the "configuration-in-file"-approach, that Gitlab CI offered, while Jenkins support of this concept was very limited and users had to resort to using the webinterface. Since the file is included within the repository, it is also version controlled, which was a huge plus for us.

See more
Sebastian Dellwig
Tech Lead at Porsche Digital GmbH · | 6 upvotes · 152.5K views
Shared insights
on
GitLab CIGitLab CICircleCICircleCICodeshipCodeship

We are using GitLab CI and were very happy with it. The integration of all tools like CI/CD, tickets, etc makes it very easy to stay on top of things. But be aware, Gitlab currently does not have iOS build support. So if you want to exchange that for CircleCI / Codeship to have to invest some effort. We are using a managed Mac OS device and installed the Gitlab runner there, to have iOS builds.

See more
Bitbucket logo

Bitbucket

34.2K
27.1K
2.8K
One place to plan projects, collaborate on code, test and deploy, all with free private repositories
34.2K
27.1K
+ 1
2.8K
PROS OF BITBUCKET
  • 905
    Free private repos
  • 398
    Simple setup
  • 347
    Nice ui and tools
  • 341
    Unlimited private repositories
  • 240
    Affordable git hosting
  • 123
    Integrates with many apis and services
  • 119
    Reliable uptime
  • 86
    Nice gui
  • 84
    Pull requests and code reviews
  • 58
    Very customisable
  • 16
    Mercurial repositories
  • 14
    SourceTree integration
  • 11
    JIRA integration
  • 10
    Track every commit to an issue in JIRA
  • 8
    Best free alternative to Github
  • 8
    Deployment hooks
  • 7
    Automatically share repositories with all your teammates
  • 7
    Compatible with Mac and Windows
  • 6
    Source Code Insight
  • 5
    Price
  • 5
    Login with Google
  • 5
    Create a wiki
  • 5
    Approve pull request button
  • 4
    Customizable pipelines
  • 4
    #2 Atlassian Product after JIRA
  • 3
    Continuous Integration and Delivery
  • 3
    Unlimited Private Repos at no cost
  • 3
    Also supports Mercurial
  • 2
    Teamcity
  • 2
    Mercurial Support
  • 2
    IAM
  • 2
    Issues tracker
  • 2
    Open source friendly
  • 2
    Multilingual interface
  • 2
    Academic license program
  • 2
    IAM integration
  • 0
    Free Private Repositories
CONS OF BITBUCKET
  • 19
    Not much community activity
  • 17
    Difficult to review prs because of confusing ui
  • 15
    Quite buggy
  • 10
    Managed by enterprise Java company
  • 8
    CI tool is not free of charge
  • 7
    Complexity with rights management
  • 6
    Only 5 collaborators for private repos
  • 4
    Slow performance
  • 2
    No AWS Codepipelines integration
  • 1
    No more Mercurial repositories
  • 1
    No server side git-hook support

related Bitbucket posts

Michael Kelly
Senior Software Engineer at StackShare · | 14 upvotes · 674.8K views

I use GitLab when building side-projects and MVPs. The interface and interactions are close enough to those of GitHub to prevent cognitive switching costs between professional and personal projects hosted on different services.

GitLab also provides a suite of tools including issue/project management, CI/CD with GitLab CI, and validation/landing pages with GitLab Pages. With everything in one place, on an #OpenSourceCloud GitLab makes it easy for me to manage much larger projects on my own, than would be possible with other solutions or tools.

It's petty I know, but I can also read the GitLab code diffs far more easily than diffs on GitHub or Bitbucket...they just look better in my opinion.

See more
Shared insights
on
GitHubGitHubGitLabGitLabBitbucketBitbucket

A bit difference in GitHub and GitLab though both are Version Control repository management services which provides key component in the software development workflow. A decision of choosing GitHub over GitLab is major leap extension from code management, to deployment and monitoring alongside looking beyond the code base hosting provided best fitted tools for developer communities.

  • Authentication stages - With GitLab you can set and modify people’s permissions according to their role. In GitHub, you can decide if someone gets a read or write access to a repository.
  • Built-In Continuous Integrations - GitLab offers its very own CI for free. No need to use an external CI service. And if you are already used to an external CI, you can obviously integrate with Jenkins, etc whereas GitHub offers various 3rd party integrations – such as Travis CI, CircleCI or Codeship – for running and testing your code. However, there’s no built-in CI solution at the moment.
  • Import/Export Resources - GitLab offers detailed documentation on how to import your data from other vendors – such as GitHub, Bitbucket to GitLab. GitHub, on the other hand, does not offer such detailed documentation for the most common git repositories. However, GitHub offers to use GitHub Importer if you have your source code in Subversion, Mercurial, TFS and others.

Also when it comes to exporting data, GitLab seems to do a pretty solid job, offering you the ability to export your projects including the following data:

  • Wiki and project repositories
  • Project uploads
  • The configuration including webhooks and services
  • Issues with comments, merge requests with diffs and comments, labels, milestones, snippets, and other project entities.

GitHub, on the other hand, seems to be more restrictive when it comes to export features of existing GitHub repositories. * Integrations - #githubmarketplace gives you an essence to have multiple and competitive integrations whereas you will find less in the GitLab.

So go ahead with better understanding.

See more
CircleCI logo

CircleCI

10.2K
5.9K
962
Automate your development process quickly, safely, and at scale
10.2K
5.9K
+ 1
962
PROS OF CIRCLECI
  • 224
    Github integration
  • 176
    Easy setup
  • 152
    Fast builds
  • 94
    Competitively priced
  • 74
    Slack integration
  • 54
    Docker support
  • 44
    Awesome UI
  • 33
    Great customer support
  • 18
    Ios support
  • 14
    Hipchat integration
  • 13
    SSH debug access
  • 11
    Free for Open Source
  • 5
    Mobile support
  • 5
    Bitbucket integration
  • 4
    Nodejs support
  • 4
    AWS CodeDeploy integration
  • 3
    Free for Github private repo
  • 3
    Great support
  • 3
    YAML configuration
  • 2
    Continuous Deployment
  • 2
    Simple, clean UI
  • 2
    Clojure
  • 2
    Clojurescript
  • 2
    OSX support
  • 1
    Android support
  • 1
    Fair pricing
  • 1
    All inclusive testing
  • 1
    Japanese in rspec comment appears OK
  • 1
    Unstable
  • 1
    Build PR Branch Only
  • 1
    Autoscaling
  • 1
    Helpful documentation
  • 1
    Really easy to use
  • 1
    Favorite
  • 1
    Works
  • 1
    Easy to deploy to private servers
  • 1
    Extremely configurable
  • 1
    So circular
  • 1
    Easy setup, easy to understand, fast and reliable
  • 1
    Parallel builds for slow test suites
  • 1
    Easy setup. 2.0 is fast!
  • 1
    Parallelism
  • 0
    Stable
  • 0
    Stable
CONS OF CIRCLECI
  • 12
    Unstable
  • 6
    Scammy pricing structure
  • 0
    Aggressive Github permissions

related CircleCI posts

Tymoteusz Paul
Devops guy at X20X Development LTD · | 23 upvotes · 5.1M views

Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).

It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up or vagrant reload we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.

I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.

We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.

If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.

The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).

Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.

See more
Tim Abbott
Shared insights
on
Travis CITravis CICircleCICircleCI
at

We actually started out on Travis CI, but we've migrated our main builds to CircleCI, and it's been a huge improvement.

The reason it's been a huge improvement is that Travis CI has a fundamentally bad design for their images, where they start with a standard base Linux image containing tons of packages (several versions of postgres, every programming language environment, etc). This is potentially nice for the "get builds for a small project running quickly" use case, but it's a total disaster for a larger project that needs a decent number of dependencies and cares about the performance and reliability of their build.

This issue is exacerbated by their networking infrastructure being unreliable; we usually saw over 1% of builds failing due to transient networking errors in Travis CI, even after we added retries to the most frequently failing operations like apt update or pip install. And they never install Ubuntu's point release updates to their images. So doing an apt update, apt install, or especially apt upgrade would take forever. We ended up writing code to actually uninstall many of their base packages and pin the versions of hundreds of others to get a semi-fast, semi-reliable build. It was infuriating.

The CircleCI v2.0 system has the right design for a CI system: we can customize the base image to start with any expensive-to-install packages we need for our build, and we can update that image if and when we want to. The end result is that when migrating, we were able to delete all the hacky optimizations mentioned above, while still ending up with a 50% faster build latency. And we've also had 5-10x fewer issues with networking-related flakes, which means one doesn't have to constantly check whether a build failure is actually due to an issue with the code under test or "just another networking flake".

See more
GitLab logo

GitLab

47.7K
39.8K
2.3K
Open source self-hosted Git management software
47.7K
39.8K
+ 1
2.3K
PROS OF GITLAB
  • 497
    Self hosted
  • 423
    Free
  • 335
    Has community edition
  • 239
    Easy setup
  • 238
    Familiar interface
  • 131
    Includes many features, including ci
  • 108
    Nice UI
  • 81
    Good integration with gitlabci
  • 54
    Simple setup
  • 33
    Has an official mobile app
  • 32
    Free private repository
  • 29
    Continuous Integration
  • 19
    Open source, great ui (like github)
  • 15
    Slack Integration
  • 11
    Full CI flow
  • 10
    Free and unlimited private git repos
  • 8
    User, group, and project access management is simple
  • 7
    Built-in CI
  • 7
    Intuitive UI
  • 7
    All in one (Git, CI, Agile..)
  • 4
    Both public and private Repositories
  • 3
    Integrated Docker Registry
  • 3
    Mattermost Chat client
  • 3
    Issue system
  • 2
    One-click install through DigitalOcean
  • 2
    Dockerized
  • 2
    Because is the best remote host for git repositories
  • 2
    Full DevOps suite with Git
  • 2
    Free private repos
  • 2
    Great for team collaboration
  • 2
    Unlimited free repos & collaborators
  • 2
    It's fully integrated
  • 2
    I like the its runners and executors feature
  • 2
    CI
  • 2
    So easy to use
  • 2
    It's powerful source code management tool
  • 2
    Excellent
  • 2
    Build/pipeline definition alongside code
  • 2
    Security and Stable
  • 2
    Low maintenance cost due omnibus-deployment
  • 2
    On-premises
  • 1
    Multilingual interface
  • 1
    Kubernetes integration with GitLab CI
  • 1
    Review Apps feature
  • 1
    Powerful software planning and maintaining tools
  • 1
    Groups of groups
  • 1
    Built-in Docker Registry
  • 1
    Beautiful
  • 1
    Wounderful
  • 1
    Opensource
  • 1
    Not Microsoft Owned
  • 1
    Many private repo
  • 1
    Published IP list for whitelisting (gl-infra#434)
  • 1
    The dashboard with deployed environments
  • 1
    HipChat intergration
  • 1
    Kubernetes Integration
  • 1
    Native CI
  • 1
    Powerful Continuous Integration System
  • 1
    It includes everything I need, all packaged with docker
  • 0
    Supports Radius/Ldap & Browser Code Edits
CONS OF GITLAB
  • 27
    Slow ui performance
  • 7
    Introduce breaking bugs every release
  • 5
    Insecure (no published IP list for whitelisting)
  • 1
    Built-in Docker Registry
  • 0
    Review Apps feature

related GitLab posts

Tim Abbott
Shared insights
on
GitHubGitHubGitLabGitLab
at

I have mixed feelings on GitHub as a product and our use of it for the Zulip open source project. On the one hand, I do feel that being on GitHub helps people discover Zulip, because we have enough stars (etc.) that we rank highly among projects on the platform. and there is a definite benefit for lowering barriers to contribution (which is important to us) that GitHub has such a dominant position in terms of what everyone has accounts with.

But even ignoring how one might feel about their new corporate owner (MicroSoft), in a lot of ways GitHub is a bad product for open source projects. Years after the "Dear GitHub" letter, there are still basic gaps in its issue tracker:

  • You can't give someone permission to label/categorize issues without full write access to a project (including ability to merge things to master, post releases, etc.).
  • You can't let anyone with a GitHub account self-assign issues to themselves.
  • Many more similar issues.

It's embarrassing, because I've talked to GitHub product managers at various open source events about these things for 3 years, and they always agree the thing is important, but then nothing ever improves in the Issues product. Maybe the new management at MicroSoft will fix their product management situation, but if not, I imagine we'll eventually do the migration to GitLab.

We have a custom bot project, http://github.com/zulip/zulipbot, to deal with some of these issues where possible, and every other large project we talk to does the same thing, more or less.

See more
Joshua Dean Küpper
CEO at Scrayos UG (haftungsbeschränkt) · | 20 upvotes · 361.8K views

We use GitLab CI because of the great native integration as a part of the GitLab framework and the linting-capabilities it offers. The visualization of complex pipelines and the embedding within the project overview made Gitlab CI even more convenient. We use it for all projects, all deployments and as a part of GitLab Pages.

While we initially used the Shell-executor, we quickly switched to the Docker-executor and use it exclusively now.

We formerly used Jenkins but preferred to handle everything within GitLab . Aside from the unification of our infrastructure another motivation was the "configuration-in-file"-approach, that Gitlab CI offered, while Jenkins support of this concept was very limited and users had to resort to using the webinterface. Since the file is included within the repository, it is also version controlled, which was a huge plus for us.

See more
Bamboo logo

Bamboo

481
465
17
Tie automated builds, tests, and releases together in a single workflow
481
465
+ 1
17
PROS OF BAMBOO
  • 10
    Integrates with other Atlassian tools
  • 4
    Great notification scheme
  • 2
    Great UI
  • 1
    Has Deployment Projects
CONS OF BAMBOO
  • 5
    Expensive

related Bamboo posts

xie zhifeng
Shared insights
on
BambooBambooJenkinsJenkinsGitLabGitLab
at

I am choosing a DevOps toolset for my team. GitLab is open source and quite cloud-native. Jenkins has a very popular environment system but old-style technicals. Bamboo is very nice but integrated only with Atlassian products.

See more
Concourse logo

Concourse

242
355
53
Pipeline-based CI system written in Go
242
355
+ 1
53
PROS OF CONCOURSE
  • 15
    Real pipelines
  • 10
    Containerised builds
  • 9
    Flexible engine
  • 6
    Fast
  • 4
    Open source
  • 3
    No Snowflakes
  • 3
    Simple configuration management
  • 2
    You have to do everything
  • 1
    Fancy Visualization
CONS OF CONCOURSE
  • 2
    Fail forward instead of rollback pattern

related Concourse posts

We use Docker for our #DeploymentWorkflow along with Concourse for build pipelines and Ansible for deployment together with Vault to manage secrets.

See more