Slack

Some Slack customers need tighter control and visibility into their data without disturbing the most essential features. This resulted in the development and release of Slack Enterprise Key Management (Slack EKM). It allows larger visibility into data and more control over the keys used to encrypt and decrypt the data. Slack EKM allows users to bring their own keys into Slack. The initial release supported third-party integration of Amazon Web Services Key Management Service to store keys. Slack EKM works independently of the web application so that other security measures could be added to it. It was written in Go, largely due to its suitability for CPU-intensive cryptographic operations and its top-notch AWS software development kit. KMS key requests are logged on AWS CloudTrail, which key requests created directly from AWS KMS.

Since the beginning, Cal Henderson has been the CTO of Slack. Earlier this year, he commented on a Quora question summarizing their current stack.

Apps

• Web: a mix of JavaScript/ES6 and React.
• Desktop: And Electron to ship it as a desktop application.
• Android: a mix of Java and Kotlin.
• iOS: written in a mix of Objective C and Swift.

Backend

• The core application and the API written in PHP/Hack that runs on HHVM.
• The data is stored in MySQL using Vitess.
• Caching is done using Memcached and MCRouter.
• The search service takes help from SolrCloud, with various Java services.
• The messaging system uses WebSockets with many services in Java and Go.
• Load balancing is done using HAproxy with Consul for configuration.
• Most services talk to each other over gRPC,
• Some Thrift and JSON-over-HTTP
• Voice and video calling service was built in Elixir.

Data warehouse

• Built using open source tools including Presto, Spark, Airflow, Hadoop and Kafka.

Etc

• For server configuration and management we use Terraform, Chef and Kubernetes.
• We use Prometheus for time series metrics and ELK for logging.

By early 2019, a mix of JavaSCript, ES6, and React powered the web app, with the desktop app shipping in Electron. Java and Kotlin powered the Android app with Objective-C and Swift powering iOS.

CTO Cal Henderson goes on to describe the backend: “we have our core application which powers and our API, which is written in PHP/Hacklang running on HHVM. We store data in MySQL using Vitess. For caching, we use Memcached and MCRouter. Our search service is based on SolrCloud, with various Java services for ranking. Our real-time messaging system uses WebSockets and is comprised of many services written in Java and Go.

“We use HAproxy for load balancing and Consul for configuration and some service discovery. Most of our services talk to each other over gRPC, though we have some Thrift and JSON-over-HTTP too. Our voice and video calling service is built in Elixir. A few different services are also written in Node. Our async task queue system is built on Kafka and Redis.

“Our data warehouse is built on open source tools, including Presto, Spark, Airflow, Hadoop and Kafka. For server configuration and management we use Terraform, Chef and Kubernetes. We use Prometheus for time series metrics and ELK for logging. Slack is largely hosted in AWS, in many regions globally.”

They're critical to the business data and operated by an ecosystem of tools. But once the tools have been used, it was important to verify that the data remains as expected at all times. Even with the best efforts to prevent errors, inconsistencies are bound to creep at any stage. In order to test the code in a comprehensive manner, Slack developed a structure known as a consistency check framework.

This is a responsive and personalized framework that can meaningfully analyze and report on your data with a number of proactive and reactive benefits. This framework is important because it can help with repair and recovery from an outage or bug, it can help ensure effective data migration through scripts that test the code post-migration, and find bugs throughout the database. This framework helped prevent duplication and identifies the canonical code in each case, running as reusable code.

The framework was created by creating generic versions of the scanning and reporting code and an interface for the checking code. The checks could be run from the command line and either a single team could be scanned or the whole system. The process was improved over time to further customize the checks and make them more specific. In order to make this framework accessible to everyone, a GUI was added and connected to the internal administrative system. The framework was also modified to include code that can fix certain problems, while others are left for manual intervention. For Slack, such a tool proved extremely beneficial in ensuring data integrity both internally and externally.