Localytics

Saving Money and Protecting Privacy With Bloom Filters

The EU General Data Protection Regulation (GDPR) went into effect on May 25th, 2018 and stipulates that companies processing personally identifiable information (PII) must carry out requests from end users to be ‘forgotten’ from their systems. Implementing this right to be forgotten (RTBF) for our products means that not only

Introducing Exploranda, a set of tools for exploring complex systems.

At Localytics, data is central to our mission. One of our core principles is to "lead with data." When I envision that, I often think of an analyst using graphs to make a case for a business decision, or a principal engineer using benchmarks to advocate for a

Infrastructure As Code With humidifier-reservoir

A little more than a year ago we made the first commit to the gem that eventually became humidifier. It’s evolved quite a bit in the last year, including integrating AWS’ resource specification which had the side-effect of greatly stabilizing the API. Here at Localytics, we’ve been using

Connecting to Snowflake with Ruby on Rails

At Localytics, one of the tools we use for data processing is the Snowflake data warehouse. We connect to Snowflake in a couple different ways, but our main data retrieval application is a Ruby on Rails API. To accomplish this we use a combination of unixODBC (an open-source implementation of

Cogito - Adding “I think” to “IAM”

Today we are open-sourcing Cogito, an abstraction of AWS IAM syntax. IAM (Identity and Access Management) is AWS’s module that dictates the ability of various users and resources to mutate other resources within the AWS ecosystem. Permissions are described through a JSON-structured policy document that lives within the AWS

Faster Snowflake Queries through Clustering

At Localytics we have petabytes of data that needs to be served at low latencies and we use Snowflake in our mix of data processing technologies. Snowflake, like many other MPP databases, has a way of partitioning data to optimize read-time performance by allowing the query engine to prune unneeded

ODBC and writing your own ActiveRecord adapter

Today we are open-sourcing our ODBC adapter for ActiveRecord, which allows Ruby on Rails applications to communicate with ODBC-compliant databases. The impetus for this work was an effort to update one of our APIs to run with the latest Rails and ruby. Along the way we released Rails 3.2.

Reacting to Change

If you sit quietly enough while reading this article you’ll hear a sound. It’s the faint tapping of fingers on a keyboard, off in the distance in parts unknown. Those self-assured fingers are putting together yet another think piece on JavaScript in this—the year two thousand and

Best Practices and Common Mistakes with Travis CI

What is Travis At Localytics, we’ve worked hard to ensure that all of our services are built and deployed using our own internal best practices. One of those core best practices is that the service in question is using continuous integration (CI). CI greatly reduces the amount of time

Tweaking DynamoDB Tables for Fun and Profit

Amazon’s DynamoDB is a great technology: it’s easy to set up and provides all the benefits of an infinitely scalable distributed database without the operational overhead. Unfortunately, DynamoDB can also be very expensive for high volume, high throughput storage. This post will show how we were able to

Exploring CLI Best Practices

At Localytics we build command line interfaces that manage our infrastructure and processes. This is our list of best practices that we’ve learned.

Six Recipes for Software Managers

The engineering managers at Localytics have been working together for over a year, and through ups and downs, we’ve grown and learned a lot as a team. The goal of this post is to share some of the lessons we’ve learned with the community. Some of the content

When Things Go Wrong

We believe the highest performing engineering teams have a process to identify, triage, fix and learn from service degradations and downtime. At Localytics, we build highly available and scalable systems, and part of the key to our success is learning from failures. One way we foster a learning culture is

Humidifier - CloudFormation made easier

Today we are open-sourcing Humidifier - one of the tools that we use internally to manage infrastructure on Amazon Web Services (AWS) at Localytics. This Ruby gem allows you to programmatically generate domain objects that represent the AWS CloudFormation (CFN) resources you want, handling the details of JSON manipulation under

Testing AWS Scala Microservices

In January and February of this year we open sourced three SBT plugins that have had a huge impact on our ability to test our AWS based Scala microservices architecture: sbt-s3 sbt-dynamodb sbt-sqs This is our story. Stop me if you've heard this one before. After one too many production