GitHub

GitHub

DevOps / Build, Test, Deploy / Code Collaboration & Version Control
Avatar of johnnyxbell
Senior Software Engineer at StackShare ·

I was building a personal project that I needed to store items in a real time database. I am more comfortable with my Frontend skills than my backend so I didn't want to spend time building out anything in Ruby or Go.

I stumbled on Firebase by #Google, and it was really all I needed. It had realtime data, an area for storing file uploads and best of all for the amount of data I needed it was free!

I built out my application using tools I was familiar with, React for the framework, Redux.js to manage my state across components, and styled-components for the styling.

Now as this was a project I was just working on in my free time for fun I didn't really want to pay for hosting. I did some research and I found Netlify. I had actually seen them at #ReactRally the year before and deployed a Gatsby site to Netlify already.

Netlify was very easy to setup and link to my GitHub account you select a repo and pretty much with very little configuration you have a live site that will deploy every time you push to master.

With the selection of these tools I was able to build out my application, connect it to a realtime database, and deploy to a live environment all with $0 spent.

If you're looking to build out a small app I suggest giving these tools a go as you can get your idea out into the real world for absolutely no cost.

READ MORE
44 upvotes·9 comments·289.9K views

Context: I wanted to create an end to end IoT data pipeline simulation in Google Cloud IoT Core and other GCP services. I never touched Terraform meaningfully until working on this project, and it's one of the best explorations in my development career. The documentation and syntax is incredibly human-readable and friendly. I'm used to building infrastructure through the google apis via Python , but I'm so glad past Sung did not make that decision. I was tempted to use Google Cloud Deployment Manager, but the templates were a bit convoluted by first impression. I'm glad past Sung did not make this decision either.

Solution: Leveraging Google Cloud Build Google Cloud Run Google Cloud Bigtable Google BigQuery Google Cloud Storage Google Compute Engine along with some other fun tools, I can deploy over 40 GCP resources using Terraform!

Check Out My Architecture: CLICK ME

Check out the GitHub repo attached

READ MORE
GitHub - sungchun12/iot-python-webapp: Live, real-time dashboard in a serverless docker web app, and deployed via terraform with a built-in CICD trigger-See Mock Website (github.com)
23 upvotes·2 comments·89.1K views
Avatar of z00b
CTO at CircleCI ·

We use MongoDB as our primary #datastore. Mongo's approach to replica sets enables some fantastic patterns for operations like maintenance, backups, and #ETL.

As we pull #microservices from our #monolith, we are taking the opportunity to build them with their own datastores using PostgreSQL. We also use Redis to cache data we’d never store permanently, and to rate-limit our requests to partners’ APIs (like GitHub).

When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We handle any side-effects of S3’s eventual consistency model within our own code. This ensures that we deal with user requests correctly while writes are in process.

READ MORE
Update: How CircleCI Processes Over 30 Million Builds Per Month - CircleCI Tech Stack (stackshare.io)
22 upvotes·1 comment·431.7K views
Avatar of ruswerner
Lead Engineer at StackShare ·

StackShare Feed is built entirely with React, Glamorous, and Apollo. One of our objectives with the public launch of the Feed was to enable a Server-side rendered (SSR) experience for our organic search traffic. When you visit the StackShare Feed, and you aren't logged in, you are delivered the Trending feed experience. We use an in-house Node.js rendering microservice to generate this HTML. This microservice needs to run and serve requests independent of our Rails web app. Up until recently, we had a mono-repo with our Rails and React code living happily together and all served from the same web process. In order to deploy our SSR app into a Heroku environment, we needed to split out our front-end application into a separate repo in GitHub. The driving factor in this decision was mostly due to limitations imposed by Heroku specifically with how processes can't communicate with each other. A new SSR app was created in Heroku and linked directly to the frontend repo so it stays in-sync with changes.

Related to this, we need a way to "deploy" our frontend changes to various server environments without building & releasing the entire Ruby application. We built a hybrid Amazon S3 Amazon CloudFront solution to host our Webpack bundles. A new CircleCI script builds the bundles and uploads them to S3. The final step in our rollout is to update some keys in Redis so our Rails app knows which bundles to serve. The result of these efforts were significant. Our frontend team now moves independently of our backend team, our build & release process takes only a few minutes, we are now using an edge CDN to serve JS assets, and we have pre-rendered React pages!

#StackDecisionsLaunch #SSR #Microservices #FrontEndRepoSplit

READ MORE
21 upvotes·443.3K views
Avatar of tim_nolet
Founder, Engineer & Dishwasher at Checkly ·

Heroku Docker GitHub Node.js hapi Vue.js AWS Lambda Amazon S3 PostgreSQL Knex.js Checkly is a fairly young company and we're still working hard to find the correct mix of product features, price and audience.

We are focussed on tech B2B, but I always wanted to serve solo developers too. So I decided to make a $7 plan.

Why $7? Simply put, it seems to be a sweet spot for tech companies: Heroku, Docker, Github, Appoptics (Librato) all offer $7 plans. They must have done a ton of research into this, so why not piggy back that and try it out.

Enough biz talk, onto tech. The challenges were:

  • Slice of a portion of the functionality so a $7 plan is still profitable. We call this the "plan limits"
  • Update API and back end services to handle and enforce plan limits.
  • Update the UI to kindly state plan limits are in effect on some part of the UI.
  • Update the pricing page to reflect all changes.
  • Keep the actual processing backend, storage and API's as untouched as possible.

In essence, we went from strictly volume based pricing to value based pricing. Here come the technical steps & decisions we made to get there.

  1. We updated our PostgreSQL schema so plans now have an array of "features". These are string constants that represent feature toggles.
  2. The Vue.js frontend reads these from the vuex store on login.
  3. Based on these values, the UI has simple v-if statements to either just show the feature or show a friendly "please upgrade" button.
  4. The hapi API has a hook on each relevant API endpoint that checks whether a user's plan has the feature enabled, or not.

Side note: We offer 10 SMS messages per month on the developer plan. However, we were not actually counting how many people were sending. We had to update our alerting daemon (that runs on Heroku and triggers SMS messages via AWS SNS) to actually bump a counter.

What we build is basically feature-toggling based on plan features. It is very extensible for future additions. Our scheduling and storage backend that actually runs users' monitoring requests (AWS Lambda) and stores the results (S3 and Postgres) has no knowledge of all of this and remained unchanged.

Hope this helps anyone building out their SaaS and is in a similar situation.

READ MORE
20 upvotes·416.1K views

Quasar Framework FeathersJS Node.js Vue.js SendinBlue Zeit Now GitHub

It was almost too easy to build a complete Feathers Rest API combined with Quasar SSR and reactive form that we are serving through an i-frame within our main site for serving our newsletter signup and opt-in page. Total time: 15 hrs. Check it out:

https://quasar.dev/newsletter

READ MORE
The Quasar Method - Quasar Framework - Medium (medium.com)
20 upvotes·105.4K views
Avatar of kirs
Production Engineer at Shopify ·
Shared insights
on
RailsRailsGitHubGitHub
at

The core Shopify app has remained a Rails monolith, but we also have hundreds of other Rails apps across the organization. These are not microservices, but domain-specific apps: Shipping (talks with various shipping providers), Identity (single sign on across all Shopify stores), and App Store to name a few. Managing a hundred apps and keeping them up to date with security updates can be tough, so we've developed ServicesDB, an internal app that keeps track of all production services and helps developers to make sure that they don't miss anything important.

ServicesDB keeps a checklist for each app: ownership, uptime, logs, on-call rotation, exception reporting, and gem security updates. If there are problems with any of those, ServicesDB opens a GitHub issue and pings owners of the app to ask them to address it. ServicesDB also makes it easy to query the infrastructure and answer questions like, “How many apps are on Rails 4.2? How many apps are using an outdated version of gem X? Which apps are calling this service?”.

READ MORE
E-Commerce at Scale: Inside Shopify's Tech Stack - Shopify Tech Stack | StackShare (stackshare.io)
20 upvotes·85.8K views
Avatar of tabbott
Founder at Zulip ·
Shared insights
on
GitHubGitHubGitLabGitLab
at

I have mixed feelings on GitHub as a product and our use of it for the Zulip open source project. On the one hand, I do feel that being on GitHub helps people discover Zulip, because we have enough stars (etc.) that we rank highly among projects on the platform. and there is a definite benefit for lowering barriers to contribution (which is important to us) that GitHub has such a dominant position in terms of what everyone has accounts with.

But even ignoring how one might feel about their new corporate owner (MicroSoft), in a lot of ways GitHub is a bad product for open source projects. Years after the "Dear GitHub" letter, there are still basic gaps in its issue tracker:

  • You can't give someone permission to label/categorize issues without full write access to a project (including ability to merge things to master, post releases, etc.).
  • You can't let anyone with a GitHub account self-assign issues to themselves.
  • Many more similar issues.

It's embarrassing, because I've talked to GitHub product managers at various open source events about these things for 3 years, and they always agree the thing is important, but then nothing ever improves in the Issues product. Maybe the new management at MicroSoft will fix their product management situation, but if not, I imagine we'll eventually do the migration to GitLab.

We have a custom bot project, http://github.com/zulip/zulipbot, to deal with some of these issues where possible, and every other large project we talk to does the same thing, more or less.

READ MORE
GitHub - zulip/zulip: Zulip server - powerful open source team chat (github.com)
19 upvotes·2 comments·216.9K views
Avatar of sethgrid
Principal Software Developer at SendGrid ·

When we've gone through code reviews (every code and config change goes through a code review) and feel good about the level of automated testing (no one can sign off on their own code's functionality; a quality assurance engineer or other developer has to verify functionality), we merge our changes via a bot that interacts with GitHub (the bot maintains our versions and change logs). After Buildkite has a green build and our binary is shipped to our repo servers, we are good to roll out deploys to our data centers and to keep pushing the needle on the performance of our system.

#Languages #DataStores #Databases #InMemoryDatabases

READ MORE
How SendGrid Scaled to 40 Billion Emails Per Month - SendGrid Tech Stack | StackShare (stackshare.io)
18 upvotes·35.4K views
Avatar of lazercaveman
Creative Web Developer at Ali Soueidan ·

Application and Data: Since my personal website ( https://alisoueidan.com ) is a SPA I've chosen to use Vue.js, as a framework to create it. After a short skeptical phase I immediately felt in love with the single file component concept! I also used vuex for state management, which makes working with several components, which are communicating with each other even more fun and convenient to use. Of course, using Vue requires using JavaScript as well, since it is the basis of it.

For markup and style, I used Pug and Sass, since they’re the perfect match to me. I love the clean and strict syntax of both of them and even more that their structure is almost similar. Also, both of them come with an expanded functionality such as mixins, loops and so on related to their “siblings” (HTML and CSS). Both of them require nesting and prevent untidy code, which can be a huge advantage when working in teams. I used JSON to store data (since the data quantity on my website is moderate) – JSON works also good in combo with Pug, using for loops, based on the JSON Objects for example.

To send my contact form I used PHP, since sending emails using PHP is still relatively convenient, simple and easy done.

DevOps: Of course, I used Git to do my version management (which I even do in smaller projects like my website just have an additional backup of my code). On top of that I used GitHub since it now supports private repository for free accounts (which I am using for my own). I use Babel to use ES6 functionality such as arrow functions and so on, and still don’t losing cross browser compatibility.

Side note: I used npm for package management. 🎉

*Business Tools: * I use Asana to organize my project. This is a big advantage to me, even if I work alone, since “private” projects can get interrupted for some time. By using Asana I still know (even after month of not touching a project) what I’ve done, on which task I was at last working on and what still is to do. Working in Teams (for enterprise I’d take on Jira instead) of course Asana is a Tool which I really love to use as well. All the graphics on my website are SVG which I have created with Adobe Illustrator and adjusted within the SVG code or by using JavaScript or CSS (SASS).

READ MORE
17 upvotes·237.3K views