What is Docker?
What is Jenkins?
Want advice about which of these to choose?Ask the StackShare community!
I'm the CTO of a marketing automation SaaS. Because of the continuously increasing load we moved to the AWSCloud. We are using more and more features of AWS: Amazon CloudWatch, Amazon SNS, Amazon CloudFront, Amazon Route 53 and so on.
Our main Database is MySQL but for the hundreds of GB document data we use MongoDB more and more. We started to use Redis for cache and other time sensitive operations.
On the front-end we use jQuery UI + Smarty but now we refactor our app to use Vue.js with Vuetify. Because our app is relatively complex we need to use vuex as well.
On the development side we use GitHub as our main repo, Docker for local and server environment and Jenkins and AWS CodePipeline for Continuous Integration.
We use GitLab CI because of the great native integration as a part of the GitLab framework and the linting-capabilities it offers. The visualization of complex pipelines and the embedding within the project overview made Gitlab CI even more convenient. We use it for all projects, all deployments and as a part of GitLab Pages.
While we initially used the Shell-executor, we quickly switched to the Docker-executor and use it exclusively now.
We formerly used Jenkins but preferred to handle everything within GitLab . Aside from the unification of our infrastructure another motivation was the "configuration-in-file"-approach, that Gitlab CI offered, while Jenkins support of this concept was very limited and users had to resort to using the webinterface. Since the file is included within the repository, it is also version controlled, which was a huge plus for us.
Regarding Continuous Integration - we've started with something very easy to set up - CircleCI , but with time we're adding more & more complex pipelines - we use Jenkins to configure & run those. It's much more effort, but at some point we had to pay for the flexibility we expected. Our source code version control is Git (which probably doesn't require a rationale these days) and we keep repos in GitHub - since the very beginning & we never considered moving out. Our primary monitoring these days is in New Relic (Ruby & SPA apps) and AppSignal (Elixir apps) - we're considering unifying it in New Relic , but this will require some improvements in Elixir app observability. For error reporting we use Sentry (a very popular choice in this class) & we collect our distributed logs using Logentries (to avoid semi-manual handling here).
Docker is the new kid on the block disrupting virtualization nowadays. You're able to save up to 70% of your development cost on AWS (or any other cloud) switching to Docker. For example instead of paying for many small VMs you can spin up a large one with many Docker containers to drastically lower your cost. That alone is only one of the reasons why Docker is the future and it's not even the best feature: isolation, testability, reproducibility, standardization, security, and upgrading / downgrading / application versions to name a few. You can spin up 1000's of Docker containers on an ordinary Laptop, but you would have trouble spinning up 100's of VMs. If you haven't already checked out Docker you're missing out on a huge opportunity to join the movement that will change development/production environments forever
The support for macOS is a fake.
I can't work with docker in macOS because de network and comunications with the container don't works correctly.
Currently experimenting. The idea is to isolate any services where I'm not confident yet in their security/quality. The hope is that if there is an exploit in a given service that an attacker won't be able break out of the docker container and cause damage to my systems.
An example of a service I would isolate in a docker container would be a minecraft browser map application I use. I don't know who wrote it, I don't know who's vetting it, I don't know the source code. I would feel a lot better putting this in a container before I expose it to the internet.
I believe I will follow this process for anything that's not properly maintained (not in an trusted apt-repo or some other sort of confidence)
We are testing out docker at the moment, building images from successful staging builds for all our APIs. Since we operate in a SOA (not quite microservices), developers have a dockerfile that they can run to build the entirety of our api infrastructure on their machines. We use the successful builds from staging to power these instances allowing them to do some more manual integration testing across systems.
Each component of the app was launched in a separate container, so that they wouldn't have to share resources: the front end in one, the back end in another, a third for celery, a fourth for celery-beat, and a fifth for RabbitMQ. Actually, we ended up running four front-end containers and eight back-end, due to load constraints.
Linux containers are so much more lightweight than VMs which is quite important for my limited budget. However, Docker has much more support and tooling for it unlike LXC, hence why I use it. rkt is interesting, although I will probably stick with Docker due to being more widespread.
All of our pull requests are automatically tested using Jenkins' integration with GitHub, and we provision and deploy our servers using Jenkins' interface. This is integrated with HipChat, immediately notifying us if anything goes wrong with a deployment.
Jenkins is our go-to devops automation tool. We use it for automated test builds, all the way up to server updates and deploys. It really helps maintain our homegrown continuous-integration suite. It even does our blue/green deploys.
- Continuous Deploy
- Dev stage: autodeploy by trigger push request from 'develop' branch of Gitlab
- Staging and production stages: Build and rollback quicly with Ansistrano playbook
- Sending messages of job results to Chatwork.
Currently serves as the location that our QA team builds various automated testing jobs.
At one point we were using it for builds, but we ended up migrating away from them to Code Pipelines.