Stream & Go: News Feeds for Over 300 Million End Users

40,200
Stream
Build scalable feeds, activity streams & chat in a few hours instead of months.

By Thierry Schellenbach, CEO, Stream.


Stream + Go


Stream is an API that enables developers to build news feeds and activity streams (try the API). We are used by over 500 companies and power the feeds of more than 300 million end users. Companies such as Product Hunt, Under Armour, Powerschool, Bandsintown, Dubsmash, Compass and Fabric (Google) rely on Stream to power their news feeds. In addition to the API, the founders of Stream also wrote the most widely used open source solution for building scalable feeds.

Here’s what Stream looks like today:

  • Number of servers: 180
  • Feed updates per month: 34 billion
  • Average API response time: 12ms
  • Average real-time response time: 2ms
  • Regions: 4 (US-East, EU-West, Tokyo and Singapore)
  • 1B API requests per month (~20k/minute)

Given that most of our customers are engineers, we often talk about our stack. Here’s a high level overview:

  1. Go is our primary programming language.
  2. We use a custom solution built on top of RocksDB + Raft for our primary database (we started out on Cassandra but wanted more control over performance). PostgreSQL stores configs, API keys, etc.
  3. OpenTracing and Jaeger handle tracing, StatsD and Grafana provide detailed monitoring and we use the ELK stack for centralized logging.
  4. Python is our language of choice for machine learning, devops and our website https://getstream.io (Django, DRF & React).
  5. Stream uses a combination of fanout-on-write and fanout-on-read. This results in fast read performance when a users open their feed, as well as fast propagation times when a famous user posts an update.

Tommaso Barguli and I (@tschellenbach) are the developers who started Stream nearly 3 years ago. We founded the company in Amsterdam, participated in Techstars NYC 2015 and opened up our Boulder, Colorado office in 2016. It’s been quite a crazy ride in a fairly short amount of time! With over 15 developers and a host of supporting roles, including sales and marketing, the team feels enormous compared to the early days.

The Challenge: News Feeds & Activity Streams

The nature of follow relationships makes it hard to scale feeds. Most of you will remember Facebook’s long load times, Twitter’s fail whale or Tumblr’s year of technical debt. Feeds are hard to scale since there is no clear way to shard the data. Follow relationships connect everyone to everyone else. This makes it difficult to split data across multiple machines. If you want to learn more about this problem, check out these papers:

At a very high level there are 3 different ways to scale your feeds:

  1. Fanout-on-write: basically precompute everything. It’s expensive, but its easy to shard the data.
  2. Fanout-on-read: hard to scale, but more affordable and new activities show up faster.
  3. Combination of the above two approaches: better performance and reduced latency, but increased code complexity.

Stream uses a combination of fanout-on-write and fanout-on-read. This allows us to effectively support both customers with highly connected graphs, as well as customers with a more sparse dataset. This is important since the ways in which our customers use Stream are very different. Have a look at these screenshots from Bandsintown, Unsplash, and Product Hunt:

screenshot examples


Switching from Python to Go

After years of optimizing our existing feed technology we decided to make a larger leap with 2.0 of Stream. While the first iteration of Stream was powered by Python and Cassandra, for Stream 2.0 of our infrastructure we switched to Go. The main reason why we switched from Python to Go is performance. Certain features of Stream such as aggregation, ranking and serialization were very difficult to speed up using Python.

We’ve been using Go since March 2017 and it’s been a great experience so far. Go has greatly increased the productivity of our development team. Not only has it improved the speed at which we develop, it’s also 30x faster for many components of Stream.

The performance of Go greatly influenced our architecture in a positive way. With Python we often found ourselves delegating logic to the database layer purely for performance reasons. The high performance of Go gave us more flexibility in terms of architecture. This led to a huge simplification of our infrastructure and a dramatic improvement of latency. For instance, we saw a 10 to 1 reduction in web-server count thanks to the lower memory and CPU usage for the same number of requests.

Initially we struggled a bit with package management for Go. However, using Dep together with the VG package contributed to creating a great workflow.

Go Stream

If you’ve never tried Go, you’ll want to try this online tour: https://tour.golang.org/welcome/1

Go as a language is heavily focused on performance. The built-in PPROF tool is amazing for finding performance issues. Uber’s Go-Torch library is great for visualizing data from PPROF and will be bundled in PPROF in Go 1.10.

Flame Graph


Switching from Cassandra to RocksDB & Raft

1.0 of Stream leveraged Cassandra for storing the feed. Cassandra is a common choice for building feeds. Instagram, for instance started, out with Redis but eventually switched to Cassandra to handle their rapid usage growth. Cassandra can handle write heavy workloads very efficiently.

Cassandra is a great tool that allows you to scale write capacity simply by adding more nodes, though it is also very complex. This complexity made it hard to diagnose performance fluctuations. Even though we had years of experience with running Cassandra, it still felt like a bit of a black box. When building Stream 2.0 we decided to go for a different approach and build Keevo. Keevo is our in-house key-value store built upon RocksDB, gRPC and Raft.

RocksDB is a highly performant embeddable database library developed and maintained by Facebook’s data engineering team. RocksDB started as a fork of Google’s LevelDB that introduced several performance improvements for SSD. Nowadays RocksDB is a project on its own and is under active development. It is written in C++ and it’s fast. Have a look at how this benchmark handles 7 million QPS. In terms of technology it’s much more simple than Cassandra. This translates into reduced maintenance overhead, improved performance and, most importantly, more consistent performance. It’s interesting to note that LinkedIn also uses RocksDB for their feed.

Our infrastructure is hosted on AWS and is designed to survive entire availability zone outages. Unlike Cassandra, Keevo clusters organizes nodes into leaders and followers . When a leader (master) node becomes unavailable the other nodes in the same deployment will start an election and pick a new leader. Electing a new leader is a fast operation and barely impacts live traffic.

To do this, Keevo implements the Raft consensus algorithm using Hashicorp’s Go implementation. This ensures that every bit stored in Keevo is stored on 3 different servers and operations are always consistent. This site does a great job of visualizing how Raft works: https://raft.github.io/

raft


Not Quite Microservices

By leveraging Go and RocksDB we’re able to achieve great feed performance. The average response time is around 12ms. The architecture lies somewhere between a monolith and a microservice. Stream runs on the following 7 services:

  1. Stream API
  2. Keevo
  3. Real time & Firehose
  4. Analytics
  5. Personalization & Machine learning
  6. Site & Dashboard
  7. Async Workers

To see all our services divided into stacks, head over here.

Personalization & Machine Learning

Almost all large apps with feeds use machine learning and personalization. For instance, LinkedIn prioritizes the items in your feed. Instagram’s explore feed displays pictures outside of the people you follow that you might be interested in. Etsy uses a similar approach to optimize ecommerce conversion. Stream supports the following 5 use cases for personalization:

Documentation for building personalized feeds.

All of these personalization use cases rely on combining feeds with analytics and machine learning. For the machine learning side we generate the models using Python. The models are different for each of our enterprise customers. Typically we’ll use one of these amazing libraries:

Analytics

Analytics data is collected using a tiny Go-based server. In the background, it will spawn go-routines to rollup the data as needed. The resulting metrics are stored in Elastic. In the past, we looked at Druid, which seems like a solid project. For now, we could get away with a simpler solution though.

Dashboard & Site

The dashboard is powered by React and Redux. We also use React and Redux for all of our example applications:

The site, as well as the API for the site, is powered by Python, Django and Django Rest Framework. Stream is sponsoring Django Rest Framework since it’s a pretty great open source project. If you need to build an API quickly there is no better tool than DRF and Python.

We use Imgix to resize the images on our site. For us, Imgix is cost efficient, fast and overall a great service. Thumbor is a good open source alternative.

Real time

Our real time infrastructure is based on Go, Redis and the excellent gorilla websocket library. It implements the Bayeux protocol. In terms of architecture it’s very similar to the node based Faye library.

It was interesting to read the “Ditching Go for Node.js” post on Hacker News. The author moves from Go to Node to improve performance. We actually did the exact opposite and moved from Node to Go for our real time system. The new Go-based infrastructure handles 8x the traffic per node.

Devops, Testing & Multiple Regions

In terms of devops the provisioning and configuration of instances is fully automated using a combination of:

Because our infrastructure is defined in code it has become trivial to launch new regions. We heavily use CloudFormation. Every single piece of our stack is defined in a CloudFormation template. If needed we are able to spawn a new dedicated shard in a few minutes. In addition, AWS Parameter Store is used to hold application settings. Our largest deployment is in US-East, but we also have regions in Tokyo, Singapore and Dublin.

A combination of Puppet and Cloud-init is used to configure our instances. We run our self-contained Go's binaries directly on the EC2 instance without any additional containerization layer.

Releasing new versions of our services is done by Travis. Travis first runs our test suite. Once it passes, it publishes a new release binary to GitHub. Common tasks such as installing dependencies for the Go project, or building a binary are automated using plain old Makefiles. (We know, crazy old school, right?) Our binaries are compressed using UPX.

Tool highlight: Travis

Travis has come a long way over the past years. I used to prefer Jenkins in some cases since it was easier to debug broken builds. With the addition of the aptly named “debug build” button, Travis is now the clear winner. It’s easy to use and free for open source, with no need to maintain anything.


Next we use Fabric to do a rolling deploy to our AWS instances. If anything goes wrong during the deploy it will halt the deploy. We take stability very seriously:

  • A high level of test coverage is required.
  • Releases are created by Travis (making it hard to deploy without running tests).
  • Code is reviewed by at least 2 team members.
  • Our extensive QA integration test suite evaluates if all 7 components still work.

We’ve written about our experience with testing our Go codebase. When things do break we do our best to be transparent about the ongoing issue:

VictorOps is a recent addition to our support stack. It’s made it very easy to collaborate on ongoing issues.

Tool Highlight: VictorOps

The best part about VictorOps is how they use a timeline to collaborate amongst team members. VictorOps is an elegant way to keep our team in the loop about outages. It also integrates well with Slack. This setup enables us to quickly react to any problems that make it into production, work together and resolve them faster.


Victorops


The vast majority of our infrastructure runs on AWS:

The devops responsibilities are shared across our team. While we do have one dedicated devops engineer, all our developers have to understand and own the entire workflow.

Monitoring

Stream uses OpenTracing for tracing and Grafana for beautiful dashboards. The tracking for Grafana is done using StatsD. The end result is this beauty:


monitoring


We track our errors in Sentry and use the ELK stack to centralize our logs.

Tool Highlight: OpenTracing + Jaegar

One new addition to the stack is OpenTracing. In the past we used New Relic, which works like a charm for Python, but isn’t able to automatically measure tracing information for Go. OpenTracing with Jaeger is a great solution that works very well for Stream. It also has, perhaps, the best logo for a tracing solution:

tool highlight opentracing & jaegar

Closing Thoughts

Go is an absolutely amazing language and has been a major win in terms of performance and developer productivity. For tracing we use OpenTracing and Jaeger. Our monitoring is running on StatsD and Graphite. Centralized logging is handled by the ELK stack.

Stream’s main database is a custom solution built on top of RocksDB and Raft. In the past we used Cassandra, which we found hard to maintain and which didn’t give us enough control over performance when compared to RocksDB.

We leverage external tools and solutions for everything that’s not a core competence. Redis hosting is handled by ElastiCache, Postgres by RDS, email by Mailgun, test builds by Travis and error reporting by Sentry.

Thank you for reading about our stack! If you’re a user of Stream, please be sure to add Stream to your stack here on StackShare. If you’re a talented individual, come work with us! And finally, if you haven’t tried out Stream yet, take a look at this quick tutorial for the API.

Stream
Build scalable feeds, activity streams & chat in a few hours instead of months.
Tools mentioned in article
Open jobs at Stream
Technical Director of Engineering
Amsterdam, or
<section> <p>We are looking for a highly skilled technical director of engineering to lead and participate to the execution of our technical roadmap. Between our Feed and Chat products, Stream provides services for more than a billion end users. Internally we have many more technical areas, services and small products.</p> <h2>What you will be doing</h2> <p>You will report to CTO and be responsible for the execution of API products. Inside our engineering, the first two levels of management are still hands-on. You will divide your time between managing projects, and managers as well as working directly on our API product and infrastructure.</p> <h2>Why Join Stream as Technical Director of Engineering</h2> <p>We want to become the AWS of application components, we currently power Chat and Feeds for over a billion end-users. Our API is used by applications with hundreds of millions of users.</p> <ul> <li>Large engineering team ~ 50% of the entire company is part of the engineering department</li> <li>Modern tech stack: we use Golang, RocksDB, Raft, Protobuf, Prometheus</li> <li>The engineering team is very solid</li> <li>Stay hands-on</li> </ul> <h2><strong>Responsibilities</strong></h2> <ul> <li>Execution and delivery of API projects</li> <li>Manage a team of developers and team leads</li> <li>Hiring and growing the best talent</li> <li>Lead planning, prioritization, and design of new API features</li> </ul> <h2>About You</h2> <p>You worked as a director, CTO, technical Co-Founder, tech lead, or team lead in the past and have several years of experience in software development.</p> <p>As a manager, you maintain a hands-on approach and stay involved with technical challenges, you are passionate about building software and technical teams.</p> <p><strong>You have</strong></p> <ul> <li>More than 5 years of experience managing software engineering teams</li> <li>Experience managing developers and leading complex technical projects</li> <li>Hired top talent and grew them under your management</li> <li>CS degree or similar/relevant experience</li> <li>Worked for more than 8 years as a software engineer</li> </ul> <p><strong>Bonus points</strong></p> <ul> <li>Startup experience</li> <li>Prior experience as CTO or as a technical Co-Founder</li> <li>Saas &amp; API products</li> <li>Mobile development background</li> </ul> <p><strong>Our tech stack</strong></p> <p>At Stream we use a wide collection of technologies to offer highly optimized and available features to our customers. Here is a shortlist of the technology that we currently use:</p> <ul> <li>Go, gRPC, RocksDB, Python</li> <li>Postgresql, RabbitMQ</li> <li>AWS, Puppet, CloudFormation</li> <li>Grafana, Graphite, ELK, Jaeger</li> <li>Redis, Memcached</li> </ul> <h2>What we have to offer you</h2> <p>Stream employees enjoy some of the best benefits in the industry:</p> <ul> <li>A team of exceptional engineers (developer positions only)</li> <li>The chance to work on OSS projects (developer positions only)</li> <li>A competitive salary</li> <li>28 days of paid time off plus paid Dutch holidays</li> <li>Company equity</li> <li>A pension scheme</li> <li>A generous Learning and Development budget</li> <li>Commute expenses to Amsterdam covered or the option to use a company bike within the city</li> <li>Gym membership of choice covered and weekly pilates sessions in the office</li> <li>A Macbook Pro and, if necessary, a home office setup package</li> <li>Healthy team lunches and plenty of snacks</li> <li>A generous relocation package</li> <li>An office in the heart of Amsterdam or the possibility to work fully remote</li> <li>The opportunity to attend or present to global conferences and meetups</li> <li>The possibility to visit our office in Boulder, CO</li> </ul> <h2>Our culture</h2> <p>Stream has a casual social culture, our team is diverse and we all have different backgrounds. Our talented developers are highly technical and collaborative, which makes Stream a great place to learn and improve your skills. When it comes to software engineering our culture is oriented towards ownership and quality: our goal is to deliver stable software.</p> <p>If you are interested in becoming a part of what we do, apply now!</p> <p><em>Stream provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.</em></p> <p><em>This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.</em></p> <p><em><strong>No recruiters/agencies please</strong></em></p> </section>
Senior iOS SDK Developer
Boulder, CO or
<p><span style="font-weight: 400;">For our</span><a href="https://github.com/GetStream/stream-chat-swift"><span style="font-weight: 400;"> Chat SDK</span></a><span style="font-weight: 400;"> team we are seeking a Senior iOS SDK Developer to work on our native iOS SDK libraries. This role is 100% remote, preferably in American time zones (MST, ET or close).</span></p> <h2><span style="font-weight: 400;">What you will be doing</span></h2> <p><span style="font-weight: 400;">As an iOS SDK Developer, you strive to simplify how applications are built. You will lead by example, raise the team’s technological proficiency, and influence the rest of the team towards best practices and mature technological choices.</span></p> <p><span style="font-weight: 400;">You will write and maintain our open-source SDKs that are used by hundreds of our customers and consumed by hundreds of millions of their users.</span></p> <p><span style="font-weight: 400;">Most of your day will be dedicated to software design, research, and coding. You will work with modern iOS technologies, such as Swift, SwiftUI, and Combine, with a heavy focus on code quality, API design, testing, and CI/CD processes.</span></p> <h2><span style="font-weight: 400;">You have</span></h2> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">Passion for writing simple and testable code, and for designing APIs meant to be used by other developers</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Deep knowledge of UIKit, experience with SwiftUI, modular UI elements, and design systems</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Proficiency in Swift and the iOS development ecosystem</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">An enthusiasm for all kinds of iOS testing: unit, integration, UI, snapshot tests</span></li> </ul> <p><strong>Bonus points</strong></p> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">Knowledge of CoreData</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience with developing SDK libraries or maintaining open-source projects</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Familiarity with reactive patterns, experience with RxSwift or Combine</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience with Fastlane and CI/CD best practices</span></li> </ul> <h2><span style="font-weight: 400;">What we have to offer you</span></h2> <p><span style="font-weight: 400;">Stream employees enjoy some of the best benefits in the industry:</span></p> <ul> <li> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">A team of exceptional engineers&nbsp;</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">The chance to work on OSS projects&nbsp;</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">A combination of 36 days per year in PTO and public holidays</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Company equity</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Remote work flexibility</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Fitness stipend</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Lunch budget of €25 per week</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">A Macbook Pro and work-from-home stipend</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">The opportunity to attend or present to global conferences and meetups</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">The possibility to visit our offices in Boulder, CO and Amsterdam, NL</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Legal employment in your country of residence</span></li> </ul> </li> </ul> <h2><span style="font-weight: 400;">Our culture</span></h2> <p><span style="font-weight: 400;">Stream has a casual social culture, our team is diverse and we all have different backgrounds. Now, Stream is a team of over 140+ peers from over 35 countries across the globe.</span></p> <p><span style="font-weight: 400;">We value transparency, aim for excellence, and support each other on our way to new victories.</span></p> <p><span style="font-weight: 400;">Our team consists of the strongest talents all over the world, which makes Stream a great place to learn and improve your skills.</span></p> <p><span style="font-weight: 400;">When it comes to software engineering our culture is oriented towards ownership and quality: our goal is to deliver stable software.</span></p> <p><span style="font-weight: 400;">If you are interested in becoming a part of what we do, apply now!</span></p> <p><strong data-stringify-type="bold">Salary:</strong> Our salary ranges are based on national averages. We have wide ranges so we can be flexible and determine compensation based on a number of factors including the candidate's skills, level of experience, and location (which can be US or a remote location that aligns with MST time zone). For the US based candidates, the salary range for this position is up to $110,000. Compensation at all other locations will be based on the factors as stated above.</p> <p><em><span style="font-weight: 400;">Stream provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.</span></em></p> <p><em><span style="font-weight: 400;">This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.</span></em></p> <p>&nbsp;</p>
DevOps Engineer
Amsterdam, or
<section> <p><span style="font-weight: 400;">We are looking for a DevOps Engineer to join Stream on our mission of elevating the quality of apps for billions of users globally. You will be part of the team responsible for our AWS infrastructure. </span></p> <p><span style="font-weight: 400;">This role is open either in our Amsterdam office or remote (EMEA).</span></p> <h2><strong>What you will be doing</strong><span style="font-weight: 400;">&nbsp;</span></h2> <p><span style="font-weight: 400;">Most of your day will be spent working on AWS infrastructure and taking care of monitoring, logging, security, and automation. </span></p> <p><span style="font-weight: 400;">You will also work on improving our system observability, </span><span style="font-weight: 400;">operations for performance and maintenance tasks.</span></p> <p><span style="font-weight: 400;">Our team is made up of very experienced engineers, some with decades of experience. By working together, you will learn from each other along the way. </span></p> <p><span style="font-weight: 400;">Y</span><span style="font-weight: 400;">ou will have an enormous impact on making our API service faster, more scalable, and more reliable. Our AWS infrastructure spans across 13 different regions. We store billions of records and serve over two billion end-users around the world. </span></p> <p><span style="font-weight: 400;">For both Feeds and Chat, we run our own distributed K/V storage written in Golang.</span></p> <h2><strong>About you</strong></h2> <p><strong>You have:</strong></p> <ul> <li>3+ years of commercial experience</li> <li>Managed instances in the cloud</li> <li>Experience with scripting languages</li> <li>Ability to manage network infrastructure (Setting up VLAN &amp; Network segmentation)</li> <li>Skills in Linux system administration and hardening</li> <li>Experience with Infrastructure as a Code and tools such as: Puppet, Docker, Ansible, Chef, Terraform, Cloudformation</li> </ul> <p><strong>&nbsp;Bonus points</strong></p> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">Production experience with AWS, K8S and Docker</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Worked with live distributed systems</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience scaling high traffic websites or API services</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Experience with NewSQL/NoSQL databases</span></li> </ul> <p><strong>Our tech stack</strong></p> <p><span style="font-weight: 400;">&nbsp;</span><span style="font-weight: 400;">At Stream, we use a wide collection of technologies to offer highly optimised and available features to our customers. Here is a shortlist of the technology that we currently use:</span></p> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">Golang, Python</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Postgresql, RocksDB, CockroachDB</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">AWS, Puppet, CloudFormation, Ansible</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Grafana, Graphite, ELK, Prometheus, OpenTelemetry</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Redis, Etcd</span></li> </ul> <h2><strong>Why join Stream?</strong></h2> <ul> <li style="font-weight: 400;"><strong>History of success.</strong><span style="font-weight: 400;"> From Amsterdam to Boulder and Techstars in-between, Stream has raised over $58.25M to build the best Chat Messaging &amp; Activity Feed infrastructure available, with best-in-class support.</span></li> <li style="font-weight: 400;"><strong>Freedom and endless growth opportunities.</strong><span style="font-weight: 400;"> As a rapidly growing startup (since 2020 we have gone from 30 to 150 employees), Stream gives you unique personal and professional growth opportunities. The opportunity of true ownership and accountability has a massive impact on your career. These are the things you can rarely experience in huge corporations.</span></li> <li style="font-weight: 400;"><strong>Be on the front line of progress and innovation.</strong><span style="font-weight: 400;"> While working with cutting-edge technology, we are passionate about tackling difficult tech problems at scale and creating reusable components for them, empowering engineering teams to ship apps faster, more securely, and with a better user experience.</span></li> <li style="font-weight: 400;"><strong>They believe in us:</strong><span style="font-weight: 400;"> Stream is backed by leading VC companies (Felicis Ventures, GGV Capital, 01.Advisors, Techstars, Arthur Ventures), including backers like Dick Costolo (01 Advisors, ex-CEO of Twitter), Olivier Pomel (CEO of Datadog), Tom Preston-Werner (Co-Founder of GitHub), Nicolas Dessaigne (Co-Founder of Algolia), Johnny Boufarhat (Founder and CEO of Hopin).</span></li> <li style="font-weight: 400;"><strong>Complete location and working hours flexibility.</strong><span style="font-weight: 400;"> If you prefer working from the office, working 100% remotely, combining both, or relocating to our AMS office, you have total freedom to choose what works best for you and makes you happy. We’re here to support you in being productive and feeling part of the team, no matter where you are.</span></li> </ul> <h2><strong>What we have to offer you</strong></h2> <p><span style="font-weight: 400;">Stream employees enjoy some of the best benefits in the industry:</span></p> <ul> <li style="font-weight: 400;"><span style="font-weight: 400;">A team of exceptional engineers&nbsp;</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">The chance to work on OSS projects </span><strong><em>&nbsp;</em></strong></li> <li style="font-weight: 400;"><span style="font-weight: 400;">28 days paid time off plus paid Dutch holidays</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Company equity</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">A pension scheme</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Remote work flexibility</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">A Learning and Development budget</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Commute expenses to Amsterdam covered or the option to use a company bike within the city</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Fitness stipend and weekly pilates sessions in the office</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Monthly in-office chair massages by a professional</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">MacBook Pro and work-from-home stipend</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Healthy team lunches and plenty of snacks</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">A generous relocation package</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">An office in the heart of Amsterdam</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">The opportunity to attend or present at global conferences and meetups</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">The possibility to visit our office in Boulder, CO</span></li> <li style="font-weight: 400;"><span style="font-weight: 400;">Parental leave paid at 100%</span></li> </ul> <h2><strong>Our culture</strong></h2> <p><span style="font-weight: 400;">Stream has a casual social culture, our team is diverse and we all have different backgrounds. Now, Stream is a team of over 130+ peers from over 35 countries across the globe.</span></p> <p><span style="font-weight: 400;">We value transparency, aim for excellence, and support each other on our way to new victories.</span></p> <p><span style="font-weight: 400;">Our team consists of the strongest talents worldwide, making Stream a great place to learn and improve your skills.&nbsp;</span></p> <p><span style="font-weight: 400;">When it comes to software engineering, our culture is oriented towards ownership and quality: our goal is to deliver stable software.</span></p> <p><span style="font-weight: 400;">If you are interested in becoming a part of what we do, apply now!</span></p> <p><em><span style="font-weight: 400;">Stream provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.</span></em></p> <p><em><span style="font-weight: 400;">This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.</span></em></p> <p><strong><em>No recruiters/agencies please</em></strong></p> </section>
Backend Software Developer
Amsterdam;
<section> <p class="job-page-module--jobMainDescriptionShort--1eV4E">We are looking for a full time Senior Software Engineer to join our development team. Job duties will include working on Stream's core API technology as well as designing and building high-performance software.</p> </section> <h2 class="heading-module--heading2--1GFVX job-page-module--jobMainDescriptionHeader--174Xj">What you will be doing</h2> <p class="job-page-module--jobMainDescriptionParagraph--3Noss">Most of your day will be dedicated to software design, research, and coding. On typical projects, you will have a lot of freedom and you will be paired with another team member. Our team is made up of very experienced engineers, some with more than 10 years of experience. By working together you will learn from each other along the way. You will have an enormous impact on making our API service faster, more scalable and more flexible.</p> <p class="job-page-module--jobMainDescriptionParagraph--3Noss">You will add new features to the service and find ways to make the existing ones perform orders of magnitude faster. Our customers have millions of users; they use Stream for mission critical features such as showing content and exposing core functionality of their application. Building stable and reliable software is not just an option: as a member of the development team, you will design and write state-of-the-art software, follow best practices, measure everything and be responsible for deployment to production. You will also spend part of your time talking to our customers and help them to use Stream in their app.</p> <h2 class="heading-module--heading3--32XPU job-page-module--jobMainDesscriptionSubtitle--2YQWD">The challenges</h2> <ul class="job-page-module--jobMainDescriptionList--2Ixxi"> <li>Distributed databases: we built our own data store for feeds and for chat</li> <li>Real-time messaging</li> <li>High performance: our API responses are in the 10ms range</li> <li>High scalability: we use sharding, master-master, and master-slave to ensure scalability</li> <li>High availability: our entire infrastructure is designed and operated to survive entire datacenter crashes</li> <li>Multi-region: we deploy our service on 4 different continents</li> </ul> <h2 class="heading-module--heading2--1GFVX job-page-module--jobMainDescriptionHeader--174Xj">You have</h2> <ul class="job-page-module--jobMainDescriptionList--2Ixxi"> <li>5+ years of backend development experience</li> <li>Experience with at least one (preferably few) of the following languages: Go, Rust, Java, C, C++, Erlang, Node.js, Python</li> <li>Proficiency in Go language is strongly preferred</li> <li>Experience with high traffic and high performance applications</li> <li>Solid knowledge of relational databases</li> <li>Experience with building HTTP APIs</li> <li>Experience managing your own projects and work in a team</li> </ul> <h2 class="heading-module--heading2--1GFVX job-page-module--jobMainDescriptionHeader--174Xj">Our tech stack</h2> <p class="job-page-module--jobMainDescriptionParagraph--3Noss">At Stream we use a wide collection of technologies to offer highly optimized and available features to our customers. Over the years we have experimented with different programming languages, frameworks, databases, and libraries. Here is a short list of the technology that we currently use. Do not worry if you do not master them all or if you do not see your favorite tool or language, you will have the chance to be exposed to most and to convince us to expand the list:</p> <ul class="job-page-module--jobMainDescriptionList--2Ixxi"> <li>Go, gRPC, RocksDB, Python</li> <li>Postgresql, RabbitMQ</li> <li>AWS, Puppet, CloudFormation</li> <li>Grafana, Graphite, ELK, Jaeger</li> <li>Redis, Memcached</li> </ul> <h2>Why join Stream?</h2> <ul> <li><strong>History of success.</strong> From Amsterdam to Boulder and Techstars in-between, Stream has raised over $58.25M to build the best Chat Messaging &amp; Activity Feed infrastructure available, with best-in-class support.</li> <li><strong>Freedom and endless growth opportunities.</strong> As a rapidly growing startup (since 2020 we have gone from 30 to 150 employees), Stream gives you unique personal and professional growth opportunities. The opportunity of true ownership and accountability has a massive impact on your career. These are the things you can rarely experience in huge corporations.</li> <li><strong>Be on the front line of progress and innovation.</strong> While working with cutting-edge technology, we are passionate about tackling difficult tech problems at scale and creating reusable components for them, empowering engineering teams to ship apps faster, more securely, and with a better user experience.</li> <li><strong>They believe in us:</strong> Stream is backed by leading VC companies (Felicis Ventures, GGV Capital, 01.Advisors, Techstars, Arthur Ventures), including backers like Dick Costolo (01 Advisors, ex-CEO of Twitter), Olivier Pomel (CEO of Datadog), Tom Preston-Werner (Co-Founder of GitHub), Nicolas Dessaigne (Co-Founder of Algolia), Johnny Boufarhat (Founder and CEO of Hopin).</li> <li><strong>Complete location and working hours flexibility.</strong> If you prefer working from the office, working 100% remotely, combining both, or relocating to our AMS office, you have total freedom to choose what works best for you and makes you happy. We’re here to support you in being productive and feeling part of the team, no matter where you are.</li> </ul> <h2 class="heading-module--heading2--1GFVX job-page-module--jobMainDescriptionHeader--174Xj">What we have to offer you</h2> <p class="job-page-module--jobMainDescriptionParagraph--3Noss">Stream employees enjoy some of the best benefits in the industry:</p> <ul class="job-page-module--jobMainDescriptionList--2Ixxi"> <li>A team of exceptional engineers</li> <li>The chance to work on OSS projects</li> <li>A competitive salary</li> <li>28 days paid time off plus paid Dutch holidays&nbsp;</li> <li>Company equity</li> <li>A pension scheme</li> <li>A generous Learning and Development budget</li> <li>Commute expenses to Amsterdam covered or option to use a company bike within the city</li> <li>Gym membership of choice covered and weekly pilates sessions in the office</li> <li>MacBook Pro or another development setup</li> <li>Healthy team lunches and plenty of snacks</li> <li>A generous relocation package</li> <li>An office in the heart of Amsterdam</li> <li>The opportunity to attend or present to global conferences and meetups</li> <li>The possibility to visit our office in Boulder, CO</li> </ul> <h2 class="heading-module--heading2--1GFVX job-page-module--jobMainDescriptionHeader--174Xj">Our culture</h2> <p>Stream has a casual social culture, our team is diverse and we all have different backgrounds. Now, Stream is a team of over 130+ peers from over 35 countries across the globe.</p> <p>We value transparency, aim for excellence, and support each other on our way to new victories.</p> <p>Our team consists of the strongest talents worldwide, making Stream a great place to learn and improve your skills.</p> <p>When it comes to software engineering, our culture is oriented towards ownership and quality: our goal is to deliver stable software.</p> <p>If you are interested in becoming a part of what we do, apply now!</p> <p><em>Stream provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.</em></p> <p><em>This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation and training.</em></p> <p class="job-page-module--jobMainDescriptionParagraph--3Noss"><strong>No recruiters/agencies please</strong></p>
Verified by
Building cool things on the internet 🛠️
Software Engineer, Data Science
Account Executive
Software Engineer
Software Architect
Javascript Developer
Marketing Manager
You may also like