Wayfair

Building a Vulnerability Management Program for the Cloud

In 2018, Wayfair decided to move to a hybrid cloud infrastructure to host our storefront. We landed on Google Cloud Platform (GCP) as our infrastructure as a service provider, enabling our developer and infrastructure teams to move fast by providing a robust set of APIs and a highly flexible and scalable environment.

A Dynamic Journey to Performance

Excitement buzzed in the virtual air — after multiple years of work, we were finally going live! As we ramped up our feature toggle, we were met with words no engineer wants to hear: “Your service isn’t performing fast enough. We need you to toggle off.”

Your Engineering Leadership Journey in a Nutshell

Before we start: despite my best efforts, this text contains many so-called male default terms, like leader, officer, or manager, which although they are generally believed to be gender-neutral, are considered by most men and sadly also the majority of women as male. I am therefore using pronouns when a male default term appears in the text for the first time.

Case Study: Using Storybook to Enhance our Frontend Developer Experience

Here at Wayfair Engineering, we’re aggressively migrating our monolithic systems into decoupled (independent) libraries and microservices. Decoupled libraries and services allow us to build faster and deploy more frequently, but they can create discoverability and software testing challenges. In this blog post, we’ll show you how we used Storybook to solve some of those problems for our frontend codebase.

Guide to Seamless Kubernetes Deploys

Search Technologies is a highly impactful team at Wayfair looking for experienced engineers. We build fast, reliable and powerful customer-facing search systems across all Wayfair platforms. Our infrastructure provides the most enjoyable browsing and shopping experience for our customers and partners. Search Technologies is very collaborative, as we partner with storefront and app teams, data science, infrastructure teams and operational engineering teams to create a holistic search experience for Wayfair’s customers.

What Does an Architect Do?

Taking on a new role can be an experience that is at once career-definingly thrilling, and a bit nerve-wracking. The inevitable questions of one’s own ability start to creep into mind — “I feel like I only just hit my stride in my last role! Why am I changing everything now?” or “What makes you qualified to be a [your fancy new role here]? You’ve never done that before!” With time, those doubts typically disperse as you remember the reason you pursued your new role in the first place: you’re passionate about the work (and, as it eventually turns out, you’re not half bad at it).

Understanding Envoy Rate Limits

Envoy rate limits is a fairly complex system, built using multiple components. While there are many articles on the Internet explaining basic setup and how each component works, we weren’t able to find something that explains how each component works end-to-end in simple terms.

Sustainable CI With Buildkite

Welcome again, Wayfair Engineering Enthusiasts, to our virtual fireside. Make yourself comfortable and settle in for the second installment of our Fireside Story series on Continuous Integration. If you haven’t had the pleasure, please read part one, a history of GitLab CI at Wayfair. This installment will feature GitLab CI’s successor, Buildkite.

Explainable Fraud Detection

Explainable Artificial Intelligence (XAI) is an emerging field within Machine Learning (ML). There has been considerable advances in the state of the art in recent years.

Predicting New Sort Performance When Data is Sparse / Missing

Goal At Wayfair, we want to predict the performance of new sorting algorithms before we launch them. Why? Sorting algorithms make money!  Think about it: what’s the main difference between scrolling randomly through our ~18 million products versus actually finding the thing you want and buying it? A sort order... Read more

Translation QEM

Wayfair sells products in the USA (wayfair.com), Canada (wayfair.ca), UK (wayfair.co.uk), and Germany (wayfair.de), and works with manufacturers from several countries. Consequently, business is being conducted in many languages. When product data is submitted by our suppliers in a different language than the market where the product is sold, we translate the content to the local language. For example, if a British supplier submits data in UK English and the product is sold both in the Wayfair UK and German stores, we translate the text to show it in German on wayfair.de. 

Feedback Hub – A Python Dash App to Call DS Model Services & Visualize Data

Feedback Hub is Wayfair’s inhouse web application, built by Data Science Analytics, which enables Analysts all across Wayfair to leverage data and insights generated by Natural Language Models developed by our Voice of the Customer DS team. 

The Rise and Fall of GitLab CI

Greetings Wayfair Engineering Enthusiasts: old, new, and returned. We’re gathered today (virtually) to collect a story from Wayfair’s history. I’m Gary White, volunteering historian, frequent worker. I’m going to share with you all the journey we took as an organization to adopt Continuous Integration. This is part one of two. In part two, I’ll tell you the story of Buildkite at Wayfair. In this part (one), you’ll indulge me in learning about Gitlab CI. Where we started, the good, the bad, the decline, and the eventual decision to scrap our critical build infrastructure to do something new.

Hackathon Innovation: Automating Chargeback Workflows

Hackathons usually give people an opportunity to play with some unique ideas, advanced technologies, and even start diving into completely new areas which are small beginnings of big new initiatives. For example, Wayfair is one of the industry leaders in 3D commerce today thanks to what started as a hackathon project a few years ago. Another type of hackathon projects are ones that are solving problems that are never making it to the drawing board, but hackathons bring them quickly to near-production and get support for finalizing them. They are also not unknown at Wayfair and we also followed the path of delivering tangible results working closely with our stakeholders, all the way experimenting with new technologies.

Halo Effects Platform

 By Sarah Cotterill It’s critically important for Wayfair to understand the causal impact on customer’s future spend of various customer actions, like downloading our mobile app, subscribing/unsubscribing from our email list, or purchasing in different product classes. These estimates not only inform strategic business decisions at a high-level, such as... Read more

Quantifying Google Cloud Spanner’s Geographic Latency

Today, Wayfair uses SQL Server to store our customer data. We have a sharded architecture in place which is made up of many shards spread across multiple datacenters located throughout the US and Europe. Over the past few years, we have been pushing the limits of this infrastructure, causing problems like:

How Wayfair Tests Software for the Most-Shopped Catalog of Home Goods in the World

The online retailing market is very competitive and demanding, so you need to deliver new features/functionalities to your customers as quickly as possible. In these realities, every wasted minute can cost you thousands of dollars.

Protected: Scaling Kafka Mirroring Pipelines at Wayfair

In this article, we will take you on a journey of integrating and leveraging LinkedIn’s Brooklin platform as a mirroring technology to replace Kafka Mirror Makers (KMM). 

Thoughts on Dependency Management at Wayfair

Dependency injection has always been a hot topic for debate. At Wayfair, we have always chosen the simpler approach: passing the parameters right into the initializer. While this approach was easy to follow, scalability became a concern once the app started growing to a complicated machinery.

What is Auxiliary Engineering?

If you work in software, this might sound familiar to you: an engineering team has been tasked with designing and building a large, highly visible product or system from the ground up. The team is mostly made up of newer engineers with less institutional knowledge, and the tech lead doesn’t have much experience in this particular domain. However, there’s a senior engineer with domain knowledge and some temporary availability. What’s the most effective way they can help?

Cut Costs and Build Your Own Network Intelligence

Have you ever wanted to be able to simply switch between Internet Service Providers (ISP) dynamically based on network path reliability? Many network engineers will answer yes. This is a common ask in environments of any size. 

TopShelf: How to Arrange a Page of Products, Not Just Rank Them in a List

Many approaches to ranking a list of products or search results are based on assigning a score to each item and sorting in descending order—in other words, greedy sorting approaches. In e-commerce, predictive models place the product that the customer is most likely to be interested in at the top,... Read more

Q&A with Nicholas Dreckshage: Tech Lead on Frontend Framework Team

The Frontend Framework team is responsible for (1) the core frontend libraries that power Wayfair.com and our internal sites; (2) building, testing, and deploying frontend code to production; (3) the frontend development tooling that all Wayfair engineers use to develop frontend code. We are currently leading frontend decoupling which is an essential part of Wayfair engineering’s overall decoupling initiative.

Multi-zone Cluster Management at Wayfair with Kubernetes

Wayfair operates large data-centers in addition to leveraging the public cloud to provide infrastructure services. Over time, Wayfair Infrastructure teams chose to introduce multiple logical zones within a single datacenter to facilitate easier network maintenance and to split the single failure domain into multiple smaller ones.

Q&A with Michael Shearer: Senior Engineer on Machine Learning Platforms Team

Data science and machine learning are incredibly complex domains which require expertise in the fields of mathematics, engineering, and business. We wanted to avoid adding “Dev-Ops” to that list of requirements. In practice, this meant building a solution for all of these “Dev-Ops” challenges which all data scientists can leverage. That’s where my team comes in.

The Importance of Covariates in Causal Inference: Shown in a Comparison of Two Methods

In the last century, scholars across a variety of fields have explored the mathematics of decision-making, or learning to intervene within a system. A challenge that relies on decision-making is fundamentally different, and more complex, than a classification problem. For example, designing a machine to distinguish a photo of a dog from a cat (a classification problem) is different from teaching one to recommend a photo of a dog versus a photo of a cat in order to maximize a key parameter (a decision-making problem). 

The Visual Complements Model (ViCs): Complementary Product Recommendations From Visual Cues

Introduction Creating a recommendation system for home decoration poses unique challenges: each customer has his/her own taste and would like to maintain a cohesive personal style across his/her home. We at Wayfair know that it is hard to describe that taste in words and then to search through huge product... Read more

Q&A with Sarah Carroll: Senior Software Engineer for Storefront Physical Stores 

The Storefront Physical Stores team creates and maintains all of the software that runs our physical retail located in Natick, MA. To learn more about our Natick store and the work my team put into creating the tech behind it, check out this recent blog post!

How to enable data scientists to stop managing ETL pipelines and get back to doing data science: Part I

Part I: How to build tools that you can hand off to data scientists [feature of Hoover tool]   Summary Business moves fast and data science runways are long. What can we do to remove friction and iterate on data science models as quickly as possible? In this two-part blog... Read more

Q&A with John Li: Engineering Manager of Design Services

Design Services is Wayfair’s online interior design service that connects home shoppers with real interior designers. We leverage Wayfair’s technology to create seamless online collaboration between customers and designers on looks that suits people’s unique lifestyles, spaces, and styles. Think industry-standard service, without having to get off the couch.

Bayesian Product Ranking at Wayfair

Wayfair has a huge catalog with over 14 million items. Our site features a diverse array of products for people’s homes, with product categories ranging from “appliances” to “décor and pillows” to “outdoor storage sheds.” Some of these categories include hundreds of thousands of products; this broad offering ensures that... Read more

A Scientific Approach to Capacity Planning

At Wayfair, we have typically discussed scalability from an infrastructure perspective—for example, when considering how many servers to provision for an upcoming holiday season, our usual response has been to calculate based on customer volume projections from the marketing team. However, while this source of data is no doubt useful in making such decisions, it is not always sufficient. Basing additional server capacity needs off of projected customer volume data assumes that the throughput will continue to scale up linearly at a higher user load and that adding additional hardware will always help with absorbing additional load. But, that is not always the case. 

JavaScript Tests You Can Trust

Tests are an invaluable tool for me and my team, I consider them to be a key piece of my engineering toolbelt that I’ve built up over the years. At Wayfair, teams use tests to maintain code quality, as a test-driven development tool, and as a way to self-document new features. All of this utility is possible when you have a set of tests, a “test suite”, which can be trusted; a test suite that works as advertised by producing consistent results.

ReCNet: Deep Learning based Cross-class Recommendations at Wayfair

Introduction The main objective of our recommender systems is to narrow down Wayfair’s vast catalogue to assist customers in finding exactly the products they want. Most recommendation algorithms leverage a customer’s browsing history, which includes various types of interactions with products, such as clicks, add-to-cart actions, and orders. The diversity... Read more

Online to Offline: The Technical Backbone of Wayfair’s First Physical Store

Wayfair has its traditions when it comes to business: It’s rooted in e-commerce through and through. Having been at Wayfair for more than three years with our iOS team, I was able to be part of many stages of growth and expansion, right up to my most recent challenge: Building out the technical backend of our first physical store.

BERT Does Business: Implementing the BERT Model for Natural Language Processing at Wayfair

Introduction Every day, Wayfair receives tens of thousands of messages from customers, telling us about their experiences with our products and services through the likes of product reviews, return comments, and various surveys. Considering that our customers spend valuable time writing these, it is fair to assume that they have... Read more