DevOps / Build, Test, Deploy / Version Control System
Avatar of Puciek
Devops guy at X20X Development LTD ·

Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of and wishes of good luck (as it usually is ;)).

It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up or vagrant reload we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.

I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.

We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.

If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.

The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).

Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.

How to design CI/CD pipelines, or rather how I do it. | Tymoteusz Paul - X20X Development (
17 upvotes·2 comments·629.3K views
Avatar of lazercaveman
Creative Web Developer at Ali Soueidan ·

Application and Data: Since my personal website ( ) is a SPA I've chosen to use Vue.js, as a framework to create it. After a short skeptical phase I immediately felt in love with the single file component concept! I also used vuex for state management, which makes working with several components, which are communicating with each other even more fun and convenient to use. Of course, using Vue requires using JavaScript as well, since it is the basis of it.

For markup and style, I used Pug and Sass, since they’re the perfect match to me. I love the clean and strict syntax of both of them and even more that their structure is almost similar. Also, both of them come with an expanded functionality such as mixins, loops and so on related to their “siblings” (HTML and CSS). Both of them require nesting and prevent untidy code, which can be a huge advantage when working in teams. I used JSON to store data (since the data quantity on my website is moderate) – JSON works also good in combo with Pug, using for loops, based on the JSON Objects for example.

To send my contact form I used PHP, since sending emails using PHP is still relatively convenient, simple and easy done.

DevOps: Of course, I used Git to do my version management (which I even do in smaller projects like my website just have an additional backup of my code). On top of that I used GitHub since it now supports private repository for free accounts (which I am using for my own). I use Babel to use ES6 functionality such as arrow functions and so on, and still don’t losing cross browser compatibility.

Side note: I used npm for package management. 🎉

*Business Tools: * I use Asana to organize my project. This is a big advantage to me, even if I work alone, since “private” projects can get interrupted for some time. By using Asana I still know (even after month of not touching a project) what I’ve done, on which task I was at last working on and what still is to do. Working in Teams (for enterprise I’d take on Jira instead) of course Asana is a Tool which I really love to use as well. All the graphics on my website are SVG which I have created with Adobe Illustrator and adjusted within the SVG code or by using JavaScript or CSS (SASS).

17 upvotes·233.4K views
Avatar of johnnyxbell
Senior Software Engineer at StackShare ·

I've been in the #frontend game for about 7 years now. I started coding in Sublime Text because all of the tutorials I was doing back then everyone was using it. I found the speed amazing compared to some other tools at the time. I kept using Sublime Text for about 4-5 years.

I find Sublime Text lacks some functionality, after all it is just a text editor rather than a full fledged IDE. I finally converted over to PhpStorm as I was working with Magento and Magento as you know is mainly #PHP based.

This was amazing all the features in PhpStorm I loved, the debugging features, and the control click feature when you click on a dependency or linked file it will take you to that file. It was great.

PhpStorm is kind of slow, I found that Prettier was taking a long time to format my code, and it just was lagging a lot so I was looking for alternatives. After watching some more tutorial videos I noticed that everyone was using Visual Studio Code. So I gave it a go, and its amazing.

It has support for everything I need with the plugins and the integration with Git is amazing. The speed of this IDE is blazing fast, and I wouldn't go back to using PhpStorm anymore. I highly recommend giving Visual Studio Code a try!

17 upvotes·1 comment·123.2K views
Avatar of ptrthomas
Distinguished Engineer at Intuit ·
Shared insights
Karate DSLKarate DSLGitGit

Karate DSL is extremely effective in those situations where you have a microservice still in development, but the "consumer" web-UI dev team needs to make progress. Just create a mock definition (feature) file, and since it is plain-text - it can easily be shared across teams via Git. Since Karate has a binary stand-alone executable, even teams that are not familiar with Java can use it to stand-up mock services. And the best part is that the mock serves as a "contract" - which the server-side team can use to practice test-driven development.

The World's Smallest Micro Service - ptrthomas Tech Stack (
16 upvotes·2 comments·69.9K views
Avatar of ganesa-vijayakumar
Full Stack Coder | Module Lead ·

I'm planning to create a web application and also a mobile application to provide a very good shopping experience to the end customers. Shortly, my application will be aggregate the product details from difference sources and giving a clear picture to the user that when and where to buy that product with best in Quality and cost.

I have planned to develop this in many milestones for adding N number of features and I have picked my first part to complete the core part (aggregate the product details from different sources).

As per my work experience and knowledge, I have chosen the followings stacks to this mission.

UI: I would like to develop this application using React, React Router and React Native since I'm a little bit familiar on this and also most importantly these will help on developing both web and mobile apps. In addition, I'm gonna use the stacks JavaScript, jQuery, jQuery UI, jQuery Mobile, Bootstrap wherever required.

Service: I have planned to use Java as the main business layer language as I have 7+ years of experience on this I believe I can do better work using Java than other languages. In addition, I'm thinking to use the stacks Node.js.

Database and ORM: I'm gonna pick MySQL as DB and Hibernate as ORM since I have a piece of good knowledge and also work experience on this combination.

Search Engine: I need to deal with a large amount of product data and it's in-detailed info to provide enough details to end user at the same time I need to focus on the performance area too. so I have decided to use Solr as a search engine for product search and suggestions. In addition, I'm thinking to replace Solr by Elasticsearch once explored/reviewed enough about Elasticsearch.

Host: As of now, my plan to complete the application with decent features first and deploy it in a free hosting environment like Docker and Heroku and then once it is stable then I have planned to use the AWS products Amazon S3, EC2, Amazon RDS and Amazon Route 53. I'm not sure about Microsoft Azure that what is the specialty in it than Heroku and Amazon EC2 Container Service. Anyhow, I will do explore these once again and pick the best suite one for my requirement once I reached this level.

Build and Repositories: I have decided to choose Apache Maven and Git as these are my favorites and also so popular on respectively build and repositories.

Additional Utilities :) - I would like to choose Codacy for code review as their Startup plan will be very helpful to this application. I'm already experienced with Google CheckStyle and SonarQube even I'm looking something on Codacy.

Happy Coding! Suggestions are welcome! :)

Thanks, Ganesa

15 upvotes·13 comments·694.7K views
Avatar of adamrneary
Engineer at Airbnb ·

One of the joys I wanted to demonstrate in a GraphQL Summit talk I did is having so many helpful tools at my fingertips while building our product at Airbnb. This includes access to Git in Visual Studio Code, as well as the integrated terminal and tasks for running frequently-needed commands.

Of course, we also had some fun stuff to show for GraphQL and Apollo! The part that most people had not seen was the new Apollo GraphQL VS Code Extension. There is no need for me to copy over all juicy features from their marketing site (there are many!), but I will elaborate on one feature: Schema Tags.

If you are going to lint your queries against the schema you are working on, you will invariably be presented with the decision of “which schema?” The default may be your production schema (“current,” by convention), but as we discuss in the demo, if you need to iterate and explore new ideas, you need the flexibility of targeting a provisional schema.

Since we are using Apollo Engine, publishing multiple schemas using tags allows us this flexibility, and multiple engineers can collaborate on a single proposed schema. Once proposed schema changes for a service are merged upstream and those changes are naturally flowing down in the current production schema, we can flip back to “current” in VS Code. Very cool.


How Airbnb is Moving 10x Faster at Scale with GraphQL and Apollo (
15 upvotes·398.1K views
Avatar of jeromedalbert
Senior Backend Engineer at StackShare ·
Shared insights

Rails is great because "all the decisions are made for you", right? But I wouldn't be writing this decision if this were truly the case. ;-)

rails new and boilerplate generators are nice, but you still have to spend some time adding your favorite gems, tools, formatting files to your standards, removing unused functionality, and configuring deploys. Only after all of this can you truly begin to code.

That’s why I made a shell script (yeah, Rails generators are overkill) that creates a new app from a boilerplate app I preconfigured. Everyone has different tastes and levels of seniority, so I am not sharing this app to the public. Just know that it has different Git branches that my shell script can choose: master (default config), devise, forms, worker, etc. It comes bundled with the tools listed in my personal stack.

All I have to do is run the script, follow the README, and 2 minutes later my app is running a hello world page on Heroku! I cannot stress how powerful it is to have a professional-grade application ready and deployed in 2 minutes. It feels like I’m cheating the system. These superpowers have proved invaluable for personal projects, hackathons and professional projects alike.

Productivity ProjectGenerator PersonalStack

personal-stack - jeromedalbert | StackShare (
12 upvotes·16K views
Avatar of Epistol ·

I use Laravel because it's the most advances PHP framework out there, easy to maintain, easy to upgrade and most of all : easy to get a handle on, and to follow every new technology ! PhpStorm is our main software to code, as of simplicity and full range of tools for a modern application.

Google Analytics Analytics of course for a tailored analytics, Bulma as an innovative CSS framework, coupled with our Sass (Scss) pre-processor.

As of more basic stuff, we use HTML5, JavaScript (but with Vue.js too) and Webpack to handle the generation of all this.

To deploy, we set up Buddy to easily send the updates on our nginx / Ubuntu server, where it will connect to our GitHub Git private repository, pull and do all the operations needed with Deployer .

CloudFlare ensure the rapidity of distribution of our content, and Let's Encrypt the https certificate that is more than necessary when we'll want to sell some products with our Stripe api calls.

Asana is here to let us list all the functionalities, possibilities and ideas we want to implement.

11 upvotes·1 comment·397.3K views

I use Visual Studio Code because at this time is a mature software and I can do practically everything using it.

  • It's free and open source: The project is hosted on GitHub and it’s free to download, fork, modify and contribute to the project.

  • Multi-platform: You can download binaries for different platforms, included Windows (x64), MacOS and Linux (.rpm and .deb packages)

  • LightWeight: It runs smoothly in different devices. It has an average memory and CPU usage. Starts almost immediately and it’s very stable.

  • Extended language support: Supports by default the majority of the most used languages and syntax like JavaScript, HTML, C#, Swift, Java, PHP, Python and others. Also, VS Code supports different file types associated to projects like .ini, .properties, XML and JSON files.

  • Integrated tools: Includes an integrated terminal, debugger, problem list and console output inspector. The project navigator sidebar is simple and powerful: you can manage your files and folders with ease. The command palette helps you find commands by text. The search widget has a powerful auto-complete feature to search and find your files.

  • Extensible and configurable: There are many extensions available for every language supported, including syntax highlighters, IntelliSense and code completion, and debuggers. There are also extension to manage application configuration and architecture like Docker and Jenkins.

  • Integrated with Git: You can visually manage your project repositories, pull, commit and push your changes, and easy conflict resolution.( there is support for SVN (Subversion) users by plugin)

Using Visual Studio Code - Developer How-to (
11 upvotes·187.5K views

This is my stack in Application & Data

JavaScript PHP HTML5 jQuery Redis Amazon EC2 Ubuntu Sass Vue.js Firebase Laravel Lumen Amazon RDS GraphQL MariaDB

My Utilities Tools

Google Analytics Postman Elasticsearch

My Devops Tools

Git GitHub GitLab npm Visual Studio Code Kibana Sentry BrowserStack

My Business Tools


10 upvotes·1 comment·391.9K views