Alternatives to Clojure

Lumosity is home to the world's largest cognitive training database, a responsibility we take seriously. For most of the company's history, our analysis of user behavior and training data has been powered by an event stream--first a simple Node.js pub/sub app, then a heavyweight Ruby app with stronger durability. Both supported decent throughput and latency, but they lacked some major features supported by existing open-source alternatives: replaying existing messages (also lacking in most message queue-based solutions), scaling out many different readers for the same stream, the ability to leverage existing solutions for reading and writing, and possibly most importantly: the ability to hire someone externally who already had expertise.

We ultimately migrated to Kafka in early- to mid-2016, citing both industry trends in companies we'd talked to with similar durability and throughput needs, the extremely strong documentation and community. We pored over Kyle Kingsbury's Jepsen post (https://aphyr.com/posts/293-jepsen-Kafka), as well as Jay Kreps' follow-up (http://blog.empathybox.com/post/62279088548/a-few-notes-on-kafka-and-jepsen), talked at length with Confluent folks and community members, and still wound up running parallel systems for quite a long time, but ultimately, we've been very, very happy. Understanding the internals and proper levers takes some commitment, but it's taken very little maintenance once configured. Since then, the Confluent Platform community has grown and grown; we've gone from doing most development using custom Scala consumers and producers to being 60/40 Kafka Streams/Connects.

We originally looked into Storm / Heron , and we'd moved on from Redis pub/sub. Heron looks great, but we already had a programming model across services that was more akin to consuming a message consumers than required a topology of bolts, etc. Heron also had just come out while we were starting to migrate things, and the community momentum and direction of Kafka felt more substantial than the older Storm. If we were to start the process over again today, we might check out Pulsar , although the ecosystem is much younger.

To find out more, read our 2017 engineering blog post about the migration!

Onedot is building an automated data preparation service using probabilistic and statistical methods including artificial intelligence (AI). From the beginning, having a stable foundation while at the same time being able to iterate quickly was very important to us. Due to the nature of compute workloads we face, the decision for a functional programming paradigm and a scalable cluster model was a no-brainer. We started playing with Apache Spark very early on, when the platform was still in its infancy. As a storage backend, we first used Cassandra, but found out that it was not the optimal choice for our workloads (lots of rather smallish datasets, data pipelines with considerable complexity, etc.). In the end, we migrated dataset storage to Amazon S3 which proved to be much more adequate to our case. In the frontend, we bet on more traditional frameworks like React/Redux.js, Blueprint and a number of common npm packages of our universe. Because of the very positive experience with Scala (in particular the ability to write things very expressively, use immutability across the board, etc.) we settled with TypeScript in the frontend. In our opinion, a very good decision. Nowadays, transpiling is a common thing, so we thought why not introduce the same type-safety and mathematical rigour to the user interface?

Why I am using Haskell in my free time?

I have 3 reasons for it. I am looking for:

Fun.

Improve functional programming skill.

Improve problem-solving skill.

Laziness and mathematical abstractions behind Haskell makes it a wonderful language.

It is Pure functional, it helps me to write better Scala code.

Highly expressive language gives elegant ways to solve coding puzzle.

When you think about test automation, it’s crucial to make it everyone’s responsibility (not just QA Engineers'). We started with Selenium and Java, but with our platform revolving around Ruby, Elixir and JavaScript, QA Engineers were left alone to automate tests. Cypress was the answer, as we could switch to JS and simply involve more people from day one. There's a downside too, as it meant testing on Chrome only, but that was "good enough" for us + if really needed we can always cover some specific cases in a different way.

Another major decision was to adopt Elixir and Phoenix Framework - the DX (Developer eXperience) is pretty similar to what we know from RoR, but this tech is running on the top of rock-solid Erlang platform which is powering planet-scale telecom solutions for 20+ years. So we're getting pretty much the best from both worlds: minimum friction & smart conventions that eliminate the excessive boilerplate AND highly concurrent EVM (Erlang's Virtual Machine) that makes all the scalability problems vanish. The transition was very smooth - none of Ruby developers we had decided to leave because of Elixir. What is more, we kept recruiting Ruby developers w/o any requirement regarding Elixir proficiency & we still were able to educate them internally in almost no time. Obviously Elixir comes with some more tools in the stack: Credo , Hex , AppSignal (required to properly monitor BEAM apps).

Another major decision was to adopt Elixir and Phoenix Framework - the DX (Developer eXperience) is pretty similar to what we know from RoR, but this tech is running on the top of rock-solid Erlang platform which is powering planet-scale telecom solutions for 20+ years. So we're getting pretty much the best from both worlds: minimum friction & smart conventions that eliminate the excessive boilerplate AND highly concurrent EVM (Erlang's Virtual Machine) that makes all the scalability problems vanish. The transition was very smooth - none of Ruby developers we had decided to leave because of Elixir. What is more, we kept recruiting Ruby developers w/o any requirement regarding Elixir proficiency & we still were able to educate them internally in almost no time. Obviously Elixir comes with some more tools in the stack: Credo , Hex , AppSignal (required to properly monitor BEAM apps).

Postmates built a tool called Bazaar that helps onboard new partners and handles several routine tasks, like nightly emails to merchants alerting them about items that are out of stock.

Since they ran Bazaar across multiple instances, the team needed to avoid sending multiple emails to their partners by obtaining lock across multiple hosts. To solve their challenge, they created and open sourced ConsulMutEx, and an Elixir module for acquiring and releasing locks with Consul and other backends.

It works with Consul’s KV store, as well as other backends, including ets, Erlang’s in-memory database.

Winds 2.0 is an open source Podcast/RSS reader developed by Stream with a core goal to enable a wide range of developers to contribute.

We chose JavaScript because nearly every developer knows or can, at the very least, read JavaScript. With ES6 and Node.js v10.x.x, it’s become a very capable language. Async/Await is powerful and easy to use (Async/Await vs Promises). Babel allows us to experiment with next-generation JavaScript (features that are not in the official JavaScript spec yet). Yarn allows us to consistently install packages quickly (and is filled with tons of new tricks)

We’re using JavaScript for everything – both front and backend. Most of our team is experienced with Go and Python, so Node was not an obvious choice for this app.

Sure... there will be haters who refuse to acknowledge that there is anything remotely positive about JavaScript (there are even rants on Hacker News about Node.js); however, without writing completely in JavaScript, we would not have seen the results we did.

#FrameworksFullStack #Languages

Our first experience with .NET core was when we developed our OSS feature management platform - Tweek (https://github.com/soluto/tweek). We wanted to create a solution that is able to run anywhere (super important for OSS), has excellent performance characteristics and can fit in a multi-container architecture. We decided to implement our rule engine processor in F# , our main service was implemented in C# and other components were built using JavaScript / TypeScript and Go.

Visual Studio Code worked really well for us as well, it worked well with all our polyglot services and the .Net core integration had great cross-platform developer experience (to be fair, F# was a bit trickier) - actually, each of our team members used a different OS (Ubuntu, macos, windows). Our production deployment ran for a time on Docker Swarm until we've decided to adopt Kubernetes with almost seamless migration process.

After our positive experience of running .Net core workloads in containers and developing Tweek's .Net services on non-windows machines, C# had gained back some of its popularity (originally lost to Node.js), and other teams have been using it for developing microservices, k8s sidecars (like https://github.com/Soluto/airbag), cli tools, serverless functions and other projects...

In our company we have think a lot about languages that we're willing to use, there we have considering Java, Python and C++ . All of there languages are old and well developed at fact but that's not ideology of araclx. We've choose a edge technologies such as Node.js , Rust , Kotlin and Go as our programming languages which is some kind of fun. Node.js is one of biggest trends of 2019, same for Go. We want to grow in our company with growth of languages we have choose, and probably when we would choose Java that would be almost impossible because larger languages move on today's market slower, and cannot have big changes.

As the WeWork footprint continued to expand, in mid-2018 the team began to explore the next generation of identity management to handle the global scale of the business.

The team decided to vet three languages for building microservices: Go, Kotlin, and Ruby. They compared the three by building a component of an identity system in each, and assessing the performance apples-to-apples.

After building out the systems and load testing each one, the team decided to implement the new system in Go for a few reasons. In addition to better performance under heavy loads, Go, according to the team, is a simpler language that will constrain developers to simpler code. Additionally, the development lifecycle is simpler with Go, since “there is little difference between running a service directly on a dev machine, to running it in a container, to running clustered instances of the service.”

In the implementation, they the Go grpc framework to handle various common infrastructure patterns, resulting in “in a clean common server pattern that we can reuse across our microservices.”