Avatar of StackShare Editors
Shared insights
on
ReduxReduxReactReact
at

By early 2017, Airbnb was seeing around 75 million searches each day. In the midst of that growth, the team decided to rearchitect the web experience.

In the prior architecture, each page of the user’s journey, from searching, to viewing listings, to booking, was delivered standalone via Rails. The goal of the redesign was to maximize speed and fluidity.

In the new configuration, Hypernova is used to server-render React, and Redux is used for all API data and “globals,” like authentication state and experiment configurations.

READ MORE
Rearchitecting Airbnb’s Frontend - Airbnb Engineering & Data Science - Medium (medium.com)
11 upvotes·748.1K views

Around the time of their Series A, Pinterest’s stack included Python and Django, with Tornado and Node.js as web servers. Memcached / Membase and Redis handled caching, with RabbitMQ handling queueing. Nginx, HAproxy and Varnish managed static-delivery and load-balancing, with persistent data storage handled by MySQL.

READ MORE
What is the technology stack behind Pinterest? - Quora (quora.com)
9 upvotes·709.5K views
Shared insights
on
SolrSolrPHPPHPJavaJavaMySQLMySQL
at

One of the earliest public references to Slack’s stack comes from a Twitter conversation. The Slack account states that “the messaging server is java, the app is php, db is mysql and solr for search,” and that uploaded files are “Stored on S3, but private files require authentication so requests go through the app.”

READ MORE
Slack on Twitter: "@markpapadakis No posts yet, but: messaging server is java, the app is php, db is mysql and solr for search. Tried n true." (twitter.com)
9 upvotes·116.6K views

In 2014, PagerDuty struggled with safely releasing reliable mobile applications to users due to some issues with how the code was being packaged and handled.

PagerDuty’s mobile apps are hybrid and used Cordova to share code between platforms. Coding was straightforward but packaging was not, as a separated Gulp-based build process was also being used. The PagerDuty team took a page from Java and started creating software artifacts.

Rather than checking in transformed code or publishing modules to NPM, the team started creating zipped-up build artifacts, which coincided perfectly with GitHub's Releases feature which arrived in 2013. So despite JavaScript lacking a standard packaged app format like a JAR, PagerDuty was still able to improve the build times and sizes of their mobile apps.

READ MORE
Lessons Learned from Creating a Reliable Mobile Build | PagerDuty (pagerduty.com)
8 upvotes·20.2K views
Shared insights
on
KafkaKafkaHadoopHadoop
at

The early data ingestion pipeline at Pinterest used Kafka as the central message transporter, with the app servers writing messages directly to Kafka, which then uploaded log files to S3.

For databases, a custom Hadoop streamer pulled database data and wrote it to S3.

Challenges cited for this infrastructure included high operational overhead, as well as potential data loss occurring when Kafka broker outages led to an overflow of in-memory message buffering.

READ MORE
Scalable and reliable data ingestion at Pinterest - Pinterest Engineering - Medium (medium.com)
7 upvotes·339.2K views
Needs advice
on
npmnpm
and
YarnYarn

From a StackShare Community member: “I’m a freelance web developer (I mostly use Node.js) and for future projects I’m debating between npm or Yarn as my default package manager. I’m a minimalist so I hate installing software if I don’t need to- in this case that would be Yarn. For those who made the switch from npm to Yarn, what benefits have you noticed? For those who stuck with npm, are you happy you with it?"

READ MORE
7 upvotes·164.6K views
Replies (14)
Lead Developer at Chore Champion·

We use Yarn because it allows us to more simply manage our node_modules. It also simplifies commands and increases speed when installing modules. Our teams module download time was cut in half after switching from NPM to Yarn. We now require all employees to use Yarn (to prevent errors with package-lock.json and yarn.lock).

READ MORE
11 upvotes·1 comment·155.6K views
Damian Esteban
Damian Esteban
·
February 15th 2020 at 1:04AM

We recently made the switch to Yarn mostly because of workspaces, speed, and the features in the Yarn v2 release. It has been great.

·
Reply
Recommends
npm

I use npm since new version is pretty fast as well (Yarn may be still faster a bit but the difference isn't huge). No need for other dependency and mainly Yarn sometimes do not work. Sometimes when I want to install project dependencies I got error using Yarn but with npm everything is installed correctly.

READ MORE
8 upvotes·146.2K views
View all (14)
Needs advice
on
TypeScriptTypeScript
and
Flow (JS)Flow (JS)

From a StackShare community member: "We are looking to rewrite our outdated front-end with TypeScript. Right now we have a mix of CoffeeScript and vanilla JavaScript. I have read that adopting TypeScript can help enforce better code quality, and best practices. I also heard good things about Flow (JS). Which one would you recommend and why?"

READ MORE
7 upvotes·133.4K views
Replies (14)
Recommends
TypeScript

I use TypeScript because:

  • incredible developer tooling and community support
  • actively developed and supported by Microsoft (yes, I like Microsoft) ;)
  • easier to make sense of a TS codebase because the annotations provide so much more context than plain JS
  • refactors become easier (VSCode has superb support for TS)

I've switched back and forth between TS and Flow and decided a year ago to abandon Flow completely in favor of TS. I don't want to bash Flow, however, my main grievances are very poor tooling (editor integration leaves much to be desired), a slower release cycle, and subpar docs and community support.

READ MORE
18 upvotes·66.2K views
Recommends
TypeScript

I use TypeScript because it isn't just about validating the types I'm expecting to receive though that is a huge part of it too. Flow (JS) seems to be a type system only. TypeScript also allows you to use the latest features of JavaScript while also providing the type checking. To be fair to Flow (JS), I have not used it, but likely wouldn't have due to the additional features I get from TypeScript.

READ MORE
11 upvotes·95.7K views
View all (14)
Needs advice
on
SlackSlackDiscordDiscord
and
GitterGitter

From a StackShare Community member: “We’re about to start a chat group for our open source project (over 5K stars on GitHub) so we can let our community collaborate more closely. The obvious choice would be Slack (k8s and a ton of major projects use it), but we’ve seen Gitter (webpack uses it) for a lot of open source projects, Discord (Vue.js moved to them), and as of late I’m seeing Spectrum more and more often. Does anyone have experience with these or other alternatives? Is it even worth assessing all these options, or should we just go with Slack? Some things that are important to us: free, all the regular integrations (GitHub, Heroku, etc), mobile & desktop apps, and open source is of course a plus."

READ MORE
6 upvotes·512.1K views
Replies (4)
Head of HR at Rawnet·

We use Slack to increase productivity by simplifying communication and putting Slack in the middle of our communication workflow #Communications #Collaboration

READ MORE
4 upvotes·73.4K views
Expert En Dveloppement Web Et Systmes Dinformations, Designer UX, UI, Co-grant at Wixiweb·

We use Discord to tracking some action and errors (logs / alerting / assertion). it's free and simple to use with mobile application et notifications

READ MORE
4 upvotes·150.8K views
View all (4)

In late 2013, the Operations Engineering team at PagerDuty was made up of 4 engineers, and was comprised of generalists, each of whom had one or two areas of depth. Although the Operations Team ran its own on-call, each engineering team at PagerDuty also participated on the pager.

The Operations Engineering Team owned 150+ servers spanning multiple cloud providers, and used Chef to automate their infrastructure across the various cloud providers with a mix of completely custom cookbooks and customized community cookbooks.

Custom cookbooks were managed by Berkshelf, andach custom cookbook contained its own tests based on ChefSpec 3, coupled with Rspec.

Jenkins was used to GitHub for new changes and to handle unit testing of those features.

READ MORE
Chef at PagerDuty | PagerDuty (pagerduty.com)
6 upvotes·197.9K views
Shared insights
on
KafkaKafkaGolangGolangRedisRedis
at

As of 2017, Slack was handling a peak of 1.4 billion jobs a day, (33,000 jobs every second). Until recently, Slack had continued to depend on their initial job queue implementation system based on Redis. While it had allowed them to grow exponentially and diversify their services, they soon outgrew the existing system. Also, dequeuing jobs required memory that was unavailable. Allowing job workers to scale up further burdened Redis, slowing the entire system.

Slack decided to use Kafka to ease the process and allow them to scale up without getting rid of the existing architecture. To build on it, they added Kafka in front of Redis leaving the existing queuing interface in place. A stateless service called Kafkagate was developed in Go to enqueue jobs to Kafka. It exposes an HTTP POST interface with each request comprising a topic, partition, and content. Kafkagate's design reduces latency while writing jobs and allows greater flexibility in job queue design. JQRelay, a stateless service, is used to relay jobs from a Kafka topic to Redis. It ensures only one relay process is assigned to each topic, failures are self-healing, and job-specific errors are corrected by re-enqueuing the job to Kafka. The new system was rolled out by double writing all jobs to both Redis and Kafka, with JQRelay operating in 'shadow mode' - dropping all jobs after reading it from Kafka. Jobs were verified by being tracked at each part of the system through its lifetime. By using durable storage and JQRelay, the enqueuing rate could be paused or adjusted to give Redis the necessary breathing room and make Slack a much more resilient service.

READ MORE
Scaling Slack’s Job Queue – Several People Are Coding (slack.engineering)
6 upvotes·108.5K views