Alternatives to Cloudinary

Google Analytics is a great tool to analyze your traffic. To debug our software and ask questions, we love to use Postman and Stack Overflow. Google Drive helps our team to share documents. We're able to build our great products through the APIs by Google Maps, CloudFlare, Stripe, PayPal, Twilio, Let's Encrypt, and TensorFlow.

When I first built my portfolio I used GitHub for the source control and deployed directly to Netlify on a push to master. This was a perfect setup, I didn't need any knowledge about #DevOps or anything, it was all just done for me.

One of the issues I had with Netlify was I wanted to gzip my JavaScript files, I had this setup in my #Webpack file, however Netlify didn't offer an easy way to set this.

Over the weekend I decided I wanted to know more about how #DevOps worked so I decided to switch from Netlify to Amazon S3. Instead of creating any #Git Webhooks I decided to use Buddy for my pipeline and to run commands. Buddy is a fantastic tool, very easy to setup builds, copying the files to my Amazon S3 bucket, then running some #AWS console commands to set the content-encoding of the JavaScript files. - Buddy is also free if you only have a few pipelines, so I didn't need to pay anything 🤙🏻.

When I made these changes I also wanted to monitor my code, and make sure I was keeping up with the best practices so I implemented Code Climate to look over my code and tell me where there code smells, issues, and other issues I've been super happy with it so far, on the free tier so its also free.

I did plan on using Amazon CloudFront for my SSL and cacheing, however it was overly complex to setup and it costs money. So I decided to go with the free tier of CloudFlare and it is amazing, best choice I've made for caching / SSL in a long time.

Platform Update: we’ve been using the Performance Test tool provided by KeyCDN for a long time in combination with Pingdom's similar tool and the #WebpageTest and #GoogleInsight - we decided to test out KeyCDN for static asset hosting. The results for the endpoints were superfast - almost 200% faster than CloudFlare in some tests and 370% faster than imgix . So we’ve moved Washington Brown from imgix for hosting theme images, to KeyCDN for hosting all images and static assets (Font, CSS & JS). There’s a few things that we like about “Key” apart from saving $6 a month on the monthly minimum spend ($4 vs $10 for imgix). Key allow for a custom CNAME (no more advertising imgix.com in domain requests and possible SEO improvements - and easier to swap to another host down the track). Key allows JPEG/WebP image requests based on clients ‘accept’ http headers - imgix required a ?auto=format query string on each image resource request - which can break some caches. Key allows for explicitly denying cookies to be set on a zone/domain; cookies are a big strain on limited upload bandwidth so to be able to force these off is great - Cloudflare adds a cookie to every header… for “performance reasons”… but remember “if you’re getting a product something for free…”

In mid-2018 we made a big push for speed on the site. The site, running on PHP, was taking about 7 seconds to load. The site had already been running through CloudFlare for some time but on a shared host in Sydney (which is also where most of the customers are). We found when developing the @TuffTruck site that DigitalOcean was fast - and even though it's located overseas, we still found it 2 seconds faster for Australian users. We found that some Wordpress plugins were really slowing the TTFB - with all plugins off, Wordpress would save respond 1.5-2 seconds faster. With a on/off step through of each plugin we found 2 plugins by Ontraport (a CRM type service that some forms we populating) was the main culprit. Out it went and we built our own WP plugin to do push the data to Ontraport only when required. With the TTFB acceptable, we moved on to getting the completed page load time down. Turning on CloudFlare 's HTML/CSS/JS minifications & Rocket Loader we could get our group of test pages, including the homepage, loading [in full] in just over 2 seconds. We then moved images off to imgix and put the CSS, JS and Fonts onto a mirrored subdomain (so that cookies weren't exchanged), but this only shaved about another 0.2 seconds off. We are keeping it running for the moment, but the $10 minimum a month for imgix is hardly worth it (this would be change if new images were going up all the time and needed processing). The client is overly happy with the ~70% improvement and has already seen the site move up the ranks of Google's SERP and bring down their PPC costs. AND all the new hosting providers still come in at half the price of the previous Sydney hosting service. We have a few ideas that we are testing on our staging site and will roll these out soon.

To provide employees with the critical need of interactive querying, we’ve worked with Presto, an open-source distributed SQL query engine, over the years. Operating Presto at Pinterest’s scale has involved resolving quite a few challenges like, supporting deeply nested and huge thrift schemas, slow/ bad worker detection and remediation, auto-scaling cluster, graceful cluster shutdown and impersonation support for ldap authenticator.

Our infrastructure is built on top of Amazon EC2 and we leverage Amazon S3 for storing our data. This separates compute and storage layers, and allows multiple compute clusters to share the S3 data.

We have hundreds of petabytes of data and tens of thousands of Apache Hive tables. Our Presto clusters are comprised of a fleet of 450 r4.8xl EC2 instances. Presto clusters together have over 100 TBs of memory and 14K vcpu cores. Within Pinterest, we have close to more than 1,000 monthly active users (out of total 1,600+ Pinterest employees) using Presto, who run about 400K queries on these clusters per month.

Each query submitted to Presto cluster is logged to a Kafka topic via Singer. Singer is a logging agent built at Pinterest and we talked about it in a previous post. Each query is logged when it is submitted and when it finishes. When a Presto cluster crashes, we will have query submitted events without corresponding query finished events. These events enable us to capture the effect of cluster crashes over time.

Each Presto cluster at Pinterest has workers on a mix of dedicated AWS EC2 instances and Kubernetes pods. Kubernetes platform provides us with the capability to add and remove workers from a Presto cluster very quickly. The best-case latency on bringing up a new worker on Kubernetes is less than a minute. However, when the Kubernetes cluster itself is out of resources and needs to scale up, it can take up to ten minutes. Some other advantages of deploying on Kubernetes platform is that our Presto deployment becomes agnostic of cloud vendor, instance types, OS, etc.

#BigData #AWS #DataScience #DataEngineering

StackShare Feed is built entirely with React, Glamorous, and Apollo. One of our objectives with the public launch of the Feed was to enable a Server-side rendered (SSR) experience for our organic search traffic. When you visit the StackShare Feed, and you aren't logged in, you are delivered the Trending feed experience. We use an in-house Node.js rendering microservice to generate this HTML. This microservice needs to run and serve requests independent of our Rails web app. Up until recently, we had a mono-repo with our Rails and React code living happily together and all served from the same web process. In order to deploy our SSR app into a Heroku environment, we needed to split out our front-end application into a separate repo in GitHub. The driving factor in this decision was mostly due to limitations imposed by Heroku specifically with how processes can't communicate with each other. A new SSR app was created in Heroku and linked directly to the frontend repo so it stays in-sync with changes.

Related to this, we need a way to "deploy" our frontend changes to various server environments without building & releasing the entire Ruby application. We built a hybrid Amazon S3 Amazon CloudFront solution to host our Webpack bundles. A new CircleCI script builds the bundles and uploads them to S3. The final step in our rollout is to update some keys in Redis so our Rails app knows which bundles to serve. The result of these efforts were significant. Our frontend team now moves independently of our backend team, our build & release process takes only a few minutes, we are now using an edge CDN to serve JS assets, and we have pre-rendered React pages!

#StackDecisionsLaunch #SSR #Microservices #FrontEndRepoSplit

Hi Otensia! I'd definitely recommend using the skills you've already got and building with JavaScript is a smart way to go these days. Most platform services have JavaScript/Node SDKs or NPM packages, many serverless platforms support Node in case you need to write any backend logic, and JavaScript is incredibly popular - meaning it will be easy to hire for, should you ever need to.

My advice would be "don't reinvent the wheel". If you already have a skill set that will work well to solve the problem at hand, and you don't need it for any other projects, don't spend the time jumping into a new language. If you're looking for an excuse to learn something new, it would be better to invest that time in learning a new platform/tool that compliments your knowledge of JavaScript. For this project, I might recommend using Netlify, Vercel, or Google Firebase to quickly and easily deploy your web app. If you need to add user authentication, there are great examples out there for Firebase Authentication, Auth0, or even Magic (a newcomer on the Auth scene, but very user friendly). All of these services work very well with a JavaScript-based application.

For inboxkitten.com, an opensource disposable email service;

We migrated our serverless workload from Cloud Functions for Firebase to CloudFlare workers, taking advantage of the lower cost and faster-performing edge computing of Cloudflare network. Made possible due to our extremely low CPU and RAM overhead of our serverless functions.

If I were to summarize the limitation of Cloudflare (as oppose to firebase/gcp functions), it would be ...

1. <5ms CPU time limit
2. Incompatible with express.js
3. one script limitation per domain

Limitations our workload is able to conform with (YMMV)

For hosting of static files, we migrated from Firebase to CommonsHost

More details on the trade-off in between both serverless providers is in the article