Git has rendered itself to be an integral part of all development at JustChunks. We heavily rely on Git as our version-control-system of choice and use branches, tags and decentralized-development to achieve our software-management-goals.
Our whole DevOps stack consists of the following tools:
- GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
- Respectively Git as revision control system
- SourceTree as Git GUI
- Visual Studio Code as IDE
- CircleCI for continuous integration (automatize development process)
- Prettier / TSLint / ESLint as code linter
- SonarQube as quality gate
- Docker as container management (incl. Docker Compose for multi-container application management)
- VirtualBox for operating system simulation tests
- Kubernetes as cluster management for docker containers
- Heroku for deploying in test environments
- nginx as web server (preferably used as facade server in production environment)
- SSLMate (using OpenSSL) for certificate management
- Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
- PostgreSQL as preferred database system
- Redis as preferred in-memory database/store (great for caching)
The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:
- Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
- Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
- Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
- Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
- Scalability: All-in-one framework for distributed systems.
- Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
So why is your deployment different for your (Heroku) test/dev and your stage/production?
When it comes to testing our web app we do not demand great computational resources and need a very simple, convenient and fast PaaS solution for deploying the app to our testers. In production though, the demand of great computational resources can rise very fast. With Amazon we are able to control that in better way.
Overview: To put it simply, we plan to use the MERN stack to build our web application. MongoDB will be used as our primary database. We will use ExpressJS alongside Node.js to set up our API endpoints. Additionally, we plan to use React to build our SPA on the client side and use Redis on the server side as our primary caching solution. Initially, while working on the project, we plan to deploy our server and client both on Heroku . However, Heroku is very limited and we will need the benefits of an Infrastructure as a Service so we will use Amazon EC2 to later deploy our final version of the application.
Serverside: nodemon will allow us to automatically restart a running instance of our node app when files changes take place. We decided to use MongoDB because it is a non relational database which uses the Document Object Model. This allows a lot of flexibility as compared to a RDMS like SQL which requires a very structural model of data that does not change too much. Another strength of MongoDB is its ease in scalability. We will use Mongoose along side MongoDB to model our application data. Additionally, we will host our MongoDB cluster remotely on MongoDB Atlas. Bcrypt will be used to encrypt user passwords that will be stored in the DB. This is to avoid the risks of storing plain text passwords. Moreover, we will use Cloudinary to store images uploaded by the user. We will also use the Twilio SendGrid API to enable automated emails sent by our application. To protect private API endpoints, we will use JSON Web Token and Passport. Also, PayPal will be used as a payment gateway to accept payments from users.
Client Side: As mentioned earlier, we will use React to build our SPA. React uses a virtual DOM which is very efficient in rendering a page. Also React will allow us to reuse components. Furthermore, it is very popular and there is a large community that uses React so it can be helpful if we run into issues. We also plan to make a cross platform mobile application later and using React will allow us to reuse a lot of our code with React Native. Redux will be used to manage state. Redux works great with React and will help us manage a global state in the app and avoid the complications of each component having its own state. Additionally, we will use Bootstrap components and custom CSS to style our app.
Other: Git will be used for version control. During the later stages of our project, we will use Google Analytics to collect useful data regarding user interactions. Moreover, Slack will be our primary communication tool. Also, we will use Visual Studio Code as our primary code editor because it is very light weight and has a wide variety of extensions that will boost productivity. Postman will be used to interact with and debug our API endpoints.
I like the tech stack you guys have selected. You guys seem to have it all figured out, and well planned. Good luck!
RDBMS like Postgres can also store, index and query schemaless data as JSON fields, while also supporting relations where it makes sense. A document model is actually a downside, since usually data will still have relations, and it makes modeling them inconvenient.
Hi Genius folk, Please advice me on the following. We like for a Jenkins job to start to make use of a webhook on a Git commit. However, the Jenkins job creates a Virtual Machine (with a location), which the committer needs to know to make use of this newly created Virtual Machine. Without the location information, the committer does not know where this Virtual Machine resides.
My question: How is a committer best informed about the outcome of a process that is triggered by the commit?
Git commit information can include the email of the commit author, so you could email them (assuming you already have a email server setup). If the commit author name matches their name on the company messaging tool you could use the messaging tool API to send them a message. If you're using a site like github you can use the github API to post a message on their PR.
If you are using Slack (and if you're not using slack I recommend using slack ) I highly recommend Slack Webhooks integration, every job can alert the specific commiter or a specific channel that updates about all the jobs, that's how my tea, handles updates on job, we get updated when a job starts and then again when it's finished / failed, it`s also very convenient to use a dedicated channel for so you can view the history of jobs very easily.
Ok, a quick pool, then:
Would you be interested in listing (and possibly adopting) opinionated (though validated) code review processes from a centralized collaborative place, instead of researching, studying and defining everything by yourselves? Something somewhat similar to ConventionalCommits?
Hi folks, here is my question:
Given a project with a specific set of tools, including, for example, Git, conventional commits, and applying git-flow.
Besides the project setup, is there a way to generate all the standard guidelines, such as general language conventions, git-flow explanation, and best practices for commit messages, pull requests, code review, etc.?
To sum-up: Does anyone know how not to waste time setting up the same guidelines artefacts for each new project, regardless of the project's technologies?
Cheers - and may the force be us in 2021!
BTW: As English is not my first language, I would really appreciate any correction for eventual mistakes. ;)
Personal Dotfiles management
Given that they are all “configuration management” tools - meaning they are designed to deploy, configure and manage servers - what would be the simplest - and yet robust - solution to manage personal dotfiles - for n00bs.
Ideally, I reckon, it should:
- be containerized (Docker?)
- be versionable (Git)
- ensure idempotency
- allow full automation (tests, CI/CD, etc.)
- be fully recoverable (Linux/ macOS)
- be easier to setup/manage (as much as possible)
Does it make sense?
I recommend whatever you are most comfortable with/whatever might already be installed in the system. Note that, for personal dotfiles, it does not need to be containerized or have full automation/testing. It just needs to handle multiple OS and platform and be idempotent. Git will handle the heavy lifting. Note that you'll have to separate out certain files like the private SSH keys and write your CM so that it will pull it from another store or assist in manually importing them.
I personally use Ansible since it is a serverless design and is in Python, which I prefer to Ruby. Saltstack was too new when I started to port my dotfile management scripts from shell into a configuration management tool. I think any of the above is fine.
For our web application's backend, we have decided to create our server using Node.js and npm as our package manager, as this allows us to utilize a developer's skills and knowledge in JS for both the frontend and backend. ExpressJS provides us an easy to learn framework that saves us effort, time and improves productivity in creating our server, while affording us room to add complexity. Passport will be used to implement Oauth2.0 authentication for our web application, allowing our users to sign in with their existing accounts (no one wants to create a remember the password for yet another account). Mongoose will be used to make calls to our backend, this framework will help make these calls more accessible and organized. We have decided to use Redis on our server for any caching we need. This will greatly speed up retrieval times and reduce calls to external sources for any data that could instead be cached on our server. Lastly, we will use Jest as our unit testing framework for our backend as it is very popular and has support for Node.js . Furthermore, this is the same testing framework we will be using for our frontend, thus allowing use quickly learn and implement testing in both frontend and backend.
We have decided to use Heroku as our hosting platform for our server. Heroku provides clear documentation and a quick and simple process to host Node.js applications with their service, along with great support with our version control Git. Furthermore, Heroku also provides a free tier, which allows us to deploy and test our web application from the beginning of development.
MongoDB is our chosen database as a NoSQL database will give us flexibility in storing different types of data and room for scaling our product. We have decided to use MongoDB Atlas to host our database. As they provide a quick and simple start up along with a free tier to host database. Thus, allowing us to rapidly test our server's uses with the database.
It is not too hard to land on Git for version control nowadays. Although it doesn't have the best user experience or user interface the tooling around git has "won" and there is little reason to look at anything else.
Overall git is fast, has "cheap branches" and the tools and documentation around git make up for what is lost.
As far as RCSs go, Git is no less user friendly than other systems. I've used CVS, RCS, Perforce, SVN, Mercurial, Bitkeeper (the one that caused Git to be born) and others I can't recall. Git is by far the most reliable, fastest, most advanced of any system out there. And if you're not comfortable on the command line, I'd have to wonder what you're doing using Git at all.
And there are always Git GUIs that can take the edge off. Save your time and just get the best one out there, GitKraken: https://www.gitkraken.com/ I use both a GUI when I'm just browsing and the CLI when I'm working.
I am comfortable with the command line and I am strictly speaking of the user experience there. It is partially broken because it exposes quite a bit of the underlying data model. Without understanding that you have to resort to learning certain invocations by heart or looking them up.
One large inconsistency is around `git checkout` being used for multiple actions (not just checking out) - recently this has improved by the addition of `git switch`.
Another big gripe is the inconsistent use of command line arguments and flags, see for instance https://stevelosh.com/blog/2013/04/git-koans/#s4-the-hobgoblin
Mercurial - which you are familiar with - is consistently better than git at everything related to the CLI user interface and experience.
Would you please share few excellent resources based on your experience?