Instagram Dev Feed

ArticleJul 12

The Instagram Engineering Blog has a new location

In order to streamline our internal blog operations, all future Instagram Engineering content will be posted on the Engineering at Meta…

Read article

ArticleJan 25

Five things I learned about working on content quality at Instagram

Recommended content, which is surfaced in places like Explore or hashtags, is a central part of people’s experience on Instagram. As people browse this “unconnected content” from accounts that they…

Read article

ArticleDec 13

Instagram Data Saver Mode

We recently shipped Data Saver Mode, a new feature on Instagram for Android that helps the app consume less mobile data. In this post, we’ll go over why we decided to work on this feature, our…

Read article

Articleabout

ExploreNov 26

Powered by AI: Instagram’s Explore recommender system

Over half of the Instagram community visits Instagram Explore every month to discover new photos, videos, and Stories relevant to their interests. Recommending the most relevant content out of…

Read article

ArticleNov 8

10 Questions with Shupin Mao, Well-being tech lead

Shupin Mao is a senior software engineer at Facebook. During her last four years at the company, Shupin helped several teams and gained experience across Instagram and Facebook, including the…

Read article

ArticleNov 1

Making instagram.com faster: Code size and execution optimizations (Part 4)

In recent years instagram.com has seen a lot of changes — we’ve launched stories, filters, creation tools, notifications, and direct messaging as well as a myriad of other features and enhancements…

Read article

Articleabout

PythonOct 17

Python at Scale: Strict Modules

Welcome to the third post in our series on Python at scale at Instagram! As we mentioned in the first post in the series, Instagram Server is a several-million-line Python monolith, and it moves…

Read article

ArticleOct 11

Making instagram.com faster: Part 3 — cache first

In recent years instagram.com has seen a lot of changes — we’ve launched stories, filters, creation tools, notifications, and direct messaging as well as a myriad of other features and enhancements…

Read article

Articleabout

Objective-C

iOSOct 8

Implementing Dark Mode in iOS 13

One of the most exciting announcements at WWDC this year was the introduction of platform-wide dark mode in iOS 13. During WWDC a group of enthusiastic iOS engineers and designers from Instagram’s…

Read article

Articleabout

ThingsSep 24

Interview with Tamar Shapiro, Instagram’s Head of Analytics

Tamar is the Head of Analytics at Instagram. She supports a team of data scientists and data engineers responsible for analytics, data science, and experimentation. Tamar also leads the Diversity…

Read article

Articleabout

PythonSep 13

Types for Python HTTP APIs: An Instagram Story

And we’re back! As we mentioned in the first part of our blog post series, Instagram Server is a Python monolith with several million lines of code and a few thousand Django endpoints. This post…

Read article

ArticleSep 6

Making Instagram.com faster: Part 2

In recent years instagram.com has seen a lot of changes — we’ve launched stories, filters, creation tools, notifications, direct messaging and a myriad of other features and enhancements. However…

Read article

ArticleAug 23

From FBU to SWE — My Journey of Proving Myself to Myself and How Instagram Helped Me Do It

Written by Amy Huyen, 2-time intern at Instagram. On my first day of intern orientation last summer, I walked into Facebook wide-eyed and in awe of all the talent that surrounded me. I saw my…

Read article

ArticleAug 15

Static Analysis at Scale: An Instagram Story

Well, mostly. There’s also some Cython, and our dependencies include a fair amount of C++ code exposed to Python as C extensions. Our server app is a monolith, one big codebase of several million…

Read article

ArticleAug 1

Making Instagram.com faster: Part 1

In recent years, instagram.com has seen a lot of changes — we’ve launched stories, filters, creation tools, notifications, and direct messaging as well as myriad other features and enhancements…

Read article

Articleabout

PythonJul 15

Adaptive process and memory management for Python web servers

At Instagram, process and memory management is a frequent point of consideration in building large-scale, distributed, fault-tolerant systems. This need is augmented by many of our long-running…

Read article

ArticleJun 28

Improving Instagram’s Music Audio Quality

Instagram plays a critical part in forming meaningful communities where people can connect with each other and share what matters most to them. To help best facilitate these connections, we craft our…

Read article

ArticleJun 13

Video Upload Latency Improvements at Instagram

In June 2013 Instagram introduced video uploads. At the time the system was simple. To ingest video and make it available for playback, we had the Instagram app upload the entire video file to the…

Read article

ArticleJun 5

Making Direct Messages Reliable and Fast

Over the last few years, Instagram Direct has grown to be a core part of sharing content and experiences with your close friends on Instagram. This isn’t a privilege we take lightly; it’s incredibly…

Read article

ArticleMay 17

Core Modeling at Instagram

At Instagram we have many Machine Learning teams. While they all work on different parts of the product, they all directly contribute to Instagram’s mission of strengthening relationships through…

Read article

ArticleApr 16

Thundering Herds & Promises

Suppose you’re writing a simple service: it handles inbound requests from your users, but doesn’t hold any data itself. This means that in order to handle any inbound requests it needs to refer to a…

Read article

ArticleDec 18

Lessons Learned at Instagram Stories and Feed Machine Learning

Instagram machine learning has grown a lot since we announced Feed ranking back in 2016. Our recommender system serves over 1 billion users on a regular basis. We also now use machine learning for…

Read article

ArticleMay 17

San Francisco!

Today we’re excited to announce that Instagram is opening our third full-stack office in San Francisco. Kevin Systrom and Mike Krieger founded Instagram in San Francisco in 2010, and now — over seven…

Read article

ArticleDec 14

Let your code type-hint itself: introducing open source MonkeyType

Today we are excited to announce we’re open-sourcing MonkeyType, our tool for automatically adding type annotations to your Python 3 code via runtime tracing of types seen. At Instagram we have…

Read article

ArticleNov 27

Organizing for Outcomes: How Instagram Derived Form from Function and Set-Up Engineering for…

Many people working in tech live in fear of new executive hires, and I understand why. Too often, companies onboard leaders to ‘shake things up’ and solve for stagnancy by haphazardly upending…

Read article

ArticleNov 17

Profiling CPython at Instagram

Instagram employs Python in one of the world’s largest settings, using it to implement the “business logic” needed to serve 800 million monthly active users. We use the reference implementation of…

Read article

ArticleNov 3

App modularization and module lazy loading at Instagram and beyond

Instagram prides itself on having a lean app. But as the number of engineers and features grows, so do the challenges to size. We start to face issues like: To tackle these issues, we recently…

Read article

ArticleNov 2

Improving performance with background data prefetching

The Instagram community is bigger and more diverse than ever before. 800m people now visit every month, 80% of whom are outside of the United States. As the community grows, it becomes more and more…

Read article

ME

MarioSoftware Developerat

InstagramsharedNov 5

Direct Connect is a product offered by AWS that allows a customer to establish peering links between Amazon’s data centers and a third party. Using it, we figured that we could link to Facebook’s infrastructure over multiple redundant 10Gbps links. It was during this research when we found the main blocker:

We have no control over IP addressing in EC2.

While this hadn’t been an issue before, it was impassable if we were to establish links with Facebook, as their internal IP space intersected with that of EC2. After much deliberation, we began to understand that we had one option: migrate to VPC first. AWS Direct Connect

119k views119k

Comments

ME

MarioSoftware Developerat

InstagramsharedNov 5

VPC launched in mid 2009 as a companion product to the existing EC2 offering, though it quickly became considered to be EC2 2.0, as it remedied many of the commonly accepted EC2 downfalls. At face value, the migration didn’t seem conceptually difficult, as VPC was just another software abstraction on top of the same hardware, yet it was much more complex, with a few main issues:

You cannot migrate a running instance.
AWS offers no migration plan.
EC2 and VPC do not share security groups.

This last point lingered in our heads as we tried to come up with a solution. What would it take to make EC2 and VPC talk to each other as if the security groups could negotiate? It seemed insurmountable: we had thousands of running instances in EC2 and we could not take any downtime. We were looking for a solution that would allow us to migrate at our own pace, moving partial and full tiers as needed, with secure communication between both sides.

So, we created Neti, a dynamic iptables-based firewall manipulation daemon, written in Python, and backed by Zookeeper. Amazon VPC

122k views122k

Comments