What is Clojure?
What is Elixir?
Need advice about which tool to choose?Ask the StackShare community!
Sign up to add, upvote and see more prosMake informed product decisions
Sign up to add, upvote and see more consMake informed product decisions
Sign up to get full access to all the companiesMake informed product decisions
Sign up to get full access to all the tool integrationsMake informed product decisions
Stitch is run entirely on AWS. All of our transactional databases are run with Amazon RDS, and we rely on Amazon S3 for data persistence in various stages of our pipeline. Our product integrates with Amazon Redshift as a data destination, and we also use Redshift as an internal data warehouse (powered by Stitch, of course).
The majority of our services run on stateless Amazon EC2 instances that are managed by AWS OpsWorks. We recently introduced Kubernetes into our infrastructure to run the scheduled jobs that execute Singer code to extract data from various sources. Although we tend to be wary of shiny new toys, Kubernetes has proven to be a good fit for this problem, and its stability, strong community and helpful tooling have made it easy for us to incorporate into our operations.
At Epsagon, we use hundreds of AWS Lambda functions, most of them are written in Python, and the Serverless Framework to pack and deploy them. One of the issues we've encountered is the difficulty to package external libraries into the Lambda environment using the Serverless Framework. This limitation is probably by design since the external code your Lambda needs can be usually included with a package manager.
In order to overcome this issue, we've developed a tool, which we also published as open-source (see link below), which automatically packs these libraries using a simple npm package and a YAML configuration file. Support for Node.js, Go, and Java will be available soon.
The GitHub respoitory: https://github.com/epsagon/serverless-package-external
We are hardcore Kubernetes users and contributors. We loved the automation it provides. However, as our team grew and added more clusters and microservices, capacity and resources management becomes a massive pain to us. We started suffering from a lot of outages and unexpected behavior as we promote our code from dev to production environments. Luckily we were working on our AI-powered tools to understand different dependencies, predict usage, and calculate the right resources and configurations that should be applied to our infrastructure and microservices. We dogfooded our agent (http://github.com/magalixcorp/magalix-agent) and were able to stabilize as the #autopilot continuously recovered any miscalculations we made or because of unexpected changes in workloads. We are open sourcing our agent in a few days. Check it out and let us know what you think! We run workloads on Microsoft Azure Google Kubernetes Engine and Amazon EC2 and we're all about Go and Python!
How Uber developed the open source, end-to-end distributed tracing Jaeger , now a CNCF project:
Distributed tracing is quickly becoming a must-have component in the tools that organizations use to monitor their complex, microservice-based architectures. At Uber, our open source distributed tracing system Jaeger saw large-scale internal adoption throughout 2016, integrated into hundreds of microservices and now recording thousands of traces every second.
Here is the story of how we got here, from investigating off-the-shelf solutions like Zipkin, to why we switched from pull to push architecture, and how distributed tracing will continue to evolve:
At Kong while building an internal tool, we struggled to route metrics to Prometheus and logs to Logstash without incurring too much latency in our metrics collection.
We replaced nginx with OpenResty on the edge of our tool which allowed us to use the lua-nginx-module to run Lua code that captures metrics and records telemetry data during every request’s log phase. Our code then pushes the metrics to a local aggregator process (written in Go) which in turn exposes them in Prometheus Exposition Format for consumption by Prometheus. This solution reduced the number of components we needed to maintain and is fast thanks to NGINX and LuaJIT.
Following its migration from vanilla instances with autoscaling groups to Kubernetes, Postmates began facing challenges while “migrating workloads that needed to scale up very quickly.”
The built-in Horizontal Pod Autoscaler (HPA) automatically scales the number of pods in a replication controller, deployment or replica set based on observed CPU utilization. But the challenges for Postmates is that there’s no way to configure the scale velocity of one particular cluster with an HPA.
For Postmates, which runs at least three different types of applications with distinct performance and scaling characteristics, this proved problematic.
To overcome these challenges, the team created and open sourced the Configurable Horizontal Pod Autoscaler, which allows for fine-grained tuning on a per-HPA object basis. The result is that “you can configure critical services to scale down very slowly, while every other service could be configured to scale down instantly to reduce costs.”
Another major decision was to adopt Elixir and Phoenix Framework - the DX (Developer eXperience) is pretty similar to what we know from RoR, but this tech is running on the top of rock-solid Erlang platform which is powering planet-scale telecom solutions for 20+ years. So we're getting pretty much the best from both worlds: minimum friction & smart conventions that eliminate the excessive boilerplate AND highly concurrent EVM (Erlang's Virtual Machine) that makes all the scalability problems vanish. The transition was very smooth - none of Ruby developers we had decided to leave because of Elixir. What is more, we kept recruiting Ruby developers w/o any requirement regarding Elixir proficiency & we still were able to educate them internally in almost no time. Obviously Elixir comes with some more tools in the stack: Credo , Hex , AppSignal (required to properly monitor BEAM apps).
I adopted Clojure and ClojureScript because:
- it's 1 language, multiple platforms.
- Simple syntax.
- Designed to avoid unwanted side effects and bugs.
- Immutable data-structures.
- Compact code, very expressive.
- Source code is data.
- It has super-flexible macro.
- Has metadata.
Go is a high performance language with simple syntax / semantics. Although it is not as expressive as some other languages, it's still a great language for backend development.
Python is expressive and battery-included, and pre-installed in most linux distros, making it a great language for scripting.
PostgreSQL: Rock-solid RDBMS with NoSQL support.
NATS: fast message queue and easy to deploy / maintain.
Docker makes deployment painless.
Git essential tool for collaboration and source management.
Our new backend micro services are primarily written in Node.js and Go and legacy systems are written in Java. For our new stack decision, we aimed to achieve greater developer productivity, low IO latency and good community so we had couple of technologies in hand to choose but finally we concluded to go for Node.js for API layer and Go for CPU/IO intensive tasks. Currently the inter-services communication is happening via REST but soon to be moved to RPC-based communication.
We have added very little to the CoffeeScript Hubot application – just enough to allow it to talk to our Hubot workers. The Hubot workers implement our operational management functionality and expose it to Hubot so we can get chat integration for free. We’ve also tailored the authentication and authorization code of Hubot to meet the needs of roles within our team.
For larger tasks, we’ve got an internal #CLI written in Go that talks to the same #API as Hubot, giving access to the same functionality we have in Slack, with the addition of scripting, piping, and all of our favorite #Unix tools. When the Hubot worker recognizes the CLI is in use, it logs the commands to Slack to maintain visibility of operational changes.
For the backend of https://www.rsvpkeeper.com I went with Go.
My past few project have been built with Go and I'm really loving it. It was my first statically typed language after many years with PHP and Node.js - and honestly I couldn't be happier to have made the switch.
The biggest thing for me, is that with the forced declaration of types - it's made me feel like I've made a more solid backend. Sometimes with PHP I felt like a stiff breeze could knock the whole thing down. I know that's an exaggeration - but it's kinda how it feels.
Anyways, everyone knows that it almost doesn't even matter what an app is actually made with - what really matters are the design decisions you make a long the way.
At FlowStack we write most of our backend in Go. Go is a well thought out language, with all the right compromises for speedy development of speedy and robust software. It's tooling is part of what makes Go such a great language. Testing and benchmarking is built into the language, in a way that makes it easy to ensure correctness and high performance. In most cases you can get more performance out of Rust and C or C++, but getting everything right is more cumbersome.
In our company we have think a lot about languages that we're willing to use, there we have considering Java, Python and C++ . All of there languages are old and well developed at fact but that's not ideology of araclx. We've choose a edge technologies such as Node.js , Rust , Kotlin and Go as our programming languages which is some kind of fun. Node.js is one of biggest trends of 2019, same for Go. We want to grow in our company with growth of languages we have choose, and probably when we would choose Java that would be almost impossible because larger languages move on today's market slower, and cannot have big changes.
i've give a try to Ruby, Crystal, Python and GO, and yeah, for web development i use Elixir-Phoenix, because idk why just amazing, my phoenix app is very stable (comparing to api that written in other language), Ruby is slow, Crystal has unstable API, GO, umm yeah, you need too complicated (i use golang for microservice)
The first time I actually started using Go was for software on our devices. So on our hotspots we have some custom software running in the firmware. For the first device, that was actually completely built by our manufacturer. But for the second generation most of the parts are built by us in-house and we needed a way to quickly develop software for the device. But we don't have any C programmers in-house, so we were actually looking for something that basically sits in between the friendliness of Ruby, but the performance and the ability to be deployed on an embedded system which you get with C. That's basically what led us to Go and it's been awesome for that. It works so well and so great. Since it works so great, it pushed us into looking into whether we should start using this for some backend services as well.
To complement Java. The REPL lets me interactively exercise Java code. I can write performant and safe libraries in Java, and then use them in Clojure. I also find the data-centric aspect of Clojure (excellent build-in structures, literal syntax for easily creating those structures, functions that act well on abstractions of those structures) good for data processing.
This fits a sweet spot between Ruby and Java.
The following basic API endpoints are implemented on the server written in Go:
- Authorization (Sign Up, Sign In)
- Update user profile
- Community: add post, like post, add comment, delete post, add reply to comment
- Self-diagnosis: send data from the app to the server
- Journal: send user data from the app to the server
- Add groups of community
- Report post, report comment, report reply
- Block user
We wrote our own image processing, resizing, and snapshotting service in Go to allow our clients to send photos and GIFs to each other. Files are stored in S3, resized on the fly using OpenCV, and then cached in GroupCache before being served to clients.
Go allows it all to be quite fast and efficient, and entirely non-blocking on uploads!
We use Clojure mostly for its "Minority Report"-like interactive development in situations that require 'semi-automatic programming' (data inspection, admin tasks, API exploration, scrapers, etc.). We have also used Clojure successfully to build some components of our stack very quickly and reliably, in the backend and the frontend.
just started learning clojure, maybe around two weeks or so. i'm addicted. this is what i want to be working with and learning for the foreseeable future. the elegance of the language is refreshing. the community is really amazing. i've finally found a language that fits my passion for programming.
Clojure simplifies and reduces the coding efforts involved in creating CloudRepo. The fact that it runs in the JVM gives us access to all the libraries that we could ever need. Our code base is much smaller and easier to reason about than it would have been had we gone with pure Java.
Our main web scraping engine is built usign Golang because of the way how efficiently and fast this language is. Also out compilation facility let people who dont know Golang build fast as flash scrapers to run ourside of our platform without any knowledge in programming in Golang.
For some of our more taxing parts of our applications, something able to handle high I/O load quickly and with fast processing is needed. Go has completely filled that gap, allowing us to break down walls that would've been completely impossible with other languages.
Huge boon to productivity when coupled with Phoenix. Moreover, it has made background jobs and all the unseen aspects of a business easily abstracted.