By mid-2015, around the time of the Series E, the Digital department at WeWork had grown to more than 40 people to support the company’s growing product needs.

By then, they’d migrated the main website off of WordPress to Ruby on Rails, and a combination React, Angular, and jQuery, though there were efforts to move entirely to React for the front-end.

The backend was structured around a microservices architecture built partially in Node.js, along with a combination of Ruby, Python, Bash, and Go. Swift/Objective-C and Java powered the mobile apps.

These technologies power the listings on the website, as well as various internal tools, like community manager dashboards as well as RFID hardware for access management.

Multiple systems means there is a requirement to cart data across them.

Started off with Talend scripts. This was great as what we initially had were PHP/Python script - allowed for a more systematic approach to ETL.

But ended up with a massive repository of scripts, complex crontab entries and regular failures due to memory issues.

Using Stitch or similar services is a better approach: - no need to worry about the infrastructure needed for the ETL processes - a more formal mapping of data from source to destination as opposed to script developer doing his/her voodoo magic - lot of common sources and destination integrations are already builtin and out of the box

etl @{etlasaservice}|topic:1323|

I use Visual Studio Code because at this time is a mature software and I can do practically everything using it.

• It's free and open source: The project is hosted on GitHub and it’s free to download, fork, modify and contribute to the project.

• Multi-platform: You can download binaries for different platforms, included Windows (x64), MacOS and Linux (.rpm and .deb packages)

• LightWeight: It runs smoothly in different devices. It has an average memory and CPU usage. Starts almost immediately and it’s very stable.

• Extended language support: Supports by default the majority of the most used languages and syntax like JavaScript, HTML, C#, Swift, Java, PHP, Python and others. Also, VS Code supports different file types associated to projects like .ini, .properties, XML and JSON files.

• Integrated tools: Includes an integrated terminal, debugger, problem list and console output inspector. The project navigator sidebar is simple and powerful: you can manage your files and folders with ease. The command palette helps you find commands by text. The search widget has a powerful auto-complete feature to search and find your files.

• Extensible and configurable: There are many extensions available for every language supported, including syntax highlighters, IntelliSense and code completion, and debuggers. There are also extension to manage application configuration and architecture like Docker and Jenkins.

• Integrated with Git: You can visually manage your project repositories, pull, commit and push your changes, and easy conflict resolution.( there is support for SVN (Subversion) users by plugin)

Multiple systems means there is a requirement to cart data across them.

Started off with Talend scripts. This was great as what we initially had were PHP/Python script - allowed for a more systematic approach to ETL.

But ended up with a massive repository of scripts, complex crontab entries and regular failures due to memory issues.

Using Stitch or similar services is a better approach: - no need to worry about the infrastructure needed for the ETL processes - a more formal mapping of data from source to destination as opposed to script developer doing his/her voodoo magic - lot of common sources and destination integrations are already builtin and out of the box

The algorithms and data infrastructure at Stitch Fix is housed in #AWS. Data acquisition is split between events flowing through Kafka, and periodic snapshots of PostgreSQL DBs. We store data in an Amazon S3 based data warehouse. Apache Spark on Yarn is our tool of choice for data movement and #ETL. Because our storage layer (s3) is decoupled from our processing layer, we are able to scale our compute environment very elastically. We have several semi-permanent, autoscaling Yarn clusters running to serve our data processing needs. While the bulk of our compute infrastructure is dedicated to algorithmic processing, we also implemented Presto for adhoc queries and dashboards.

Beyond data movement and ETL, most #ML centric jobs (e.g. model training and execution) run in a similarly elastic environment as containers running Python and R code on Amazon EC2 Container Service clusters. The execution of batch jobs on top of ECS is managed by Flotilla, a service we built in house and open sourced (see https://github.com/stitchfix/flotilla-os).

At Stitch Fix, algorithmic integrations are pervasive across the business. We have dozens of data products actively integrated systems. That requires serving layer that is robust, agile, flexible, and allows for self-service. Models produced on Flotilla are packaged for deployment in production using Khan, another framework we've developed internally. Khan provides our data scientists the ability to quickly productionize those models they've developed with open source frameworks in Python 3 (e.g. PyTorch, sklearn), by automatically packaging them as Docker containers and deploying to Amazon ECS. This provides our data scientist a one-click method of getting from their algorithms to production. We then integrate those deployments into a service mesh, which allows us to A/B test various implementations in our product.

For more info:

• Our Algorithms Tour: https://algorithms-tour.stitchfix.com/
• Our blog: https://multithreaded.stitchfix.com/blog/
• Careers: https://multithreaded.stitchfix.com/careers/

#DataScience #DataStack #Data

Django or NodeJS? Hi, I’m thinking about which software I should use for my web-app. What about Node.js or Django for the back-end? I want to create an online preparation course for the final school exams in my country. At the beginning for maths. The course should contain tutorials and a lot of exercises of different types. E.g. multiple choice, user text/number input and drawing tasks. The exercises should change (different levels) with the learning progress. Wrong questions should asked again with different numbers. I also want a score system and statistics. So far, I have got only limited web development skills. (some HTML, CSS, Bootstrap and Wordpress). I don’t know JavaScript or Python.

Possible pros for Python / Django: - easy syntax, easier to learn for me as a beginner - fast development, earlier release - libraries for mathematical and scientific computation

Possible pros for JavaScript / Node.js: - great performance, better choice for real time applications: user should get the answer for a question quickly

Which software would you use in my case? Are my arguments for Python/NodeJS right? Which kind of database would you use?

Thank you for your answer!

Node.js JavaScript Django Python

Go is a high performance language with simple syntax / semantics. Although it is not as expressive as some other languages, it's still a great language for backend development.

Python is expressive and battery-included, and pre-installed in most linux distros, making it a great language for scripting.

PostgreSQL: Rock-solid RDBMS with NoSQL support.

TypeScript saves you from all nonsense semantics of JavaScript , LOL.

NATS: fast message queue and easy to deploy / maintain.

Docker makes deployment painless.

Git essential tool for collaboration and source management.

I'm #Fullstack here and work with Vue.js, React and Node.js in some projects but also C# for other clients. Also started learning Python. And all this with just one tool!: #Vscode I have used Atom and Sublime Text in the past and they are very good too, but for me now is just vscode. I think the combination of vscode with the free available extensions that the community is creating makes a powerful tool and that's why vscode became the most popular IDE for software development. You can match it to your own needs in a couple of minutes. Did I mention you can style it your way? Amazing tool!

Our most used programming languages are JavaScript / Node.js for it's lightweight and fast use, PHP because everyone knows it, HTML5 because you can't live without it and Sass to write great CSS. Occasionally, we use nginx as a web server and proxy, React for our UX, PostgreSQL as fast relational database, Ubuntu as server OS, ES6 and TypeScript for Node, Google Compute Engine for our infrastructure, and Socket.IO and Electron for specific use cases. We also use Python for some of our backends.

We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.

To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas

To build #Webapps we decided to use Angular 2 with RxJS

#Devops - GitHub , Travis CI , Terraform , Docker , Serverless

I love Python and JavaScript . You can do the same JavaScript async operations in Python by using asyncio. This is particularly useful when you need to do socket programming in Python. With streaming sockets, data can be sent or received at any time. In case your Python program is in the middle of executing some code, other threads can handle the new socket data. Libraries like asyncio implement multiple threads, so your Python program can work in an asynchronous fashion. PubNub makes bi-directional data streaming between devices even easier.

Python is a excellent tool for #DataScience , but up to now is very poor in #uxdesign . To do some design I'm using JavaScript and #nodejs , #electron stack. The possibility of use CSS 3 to draw interfaces is very awesome and fast. Unfortunatelly Python don't have (yet) a good way to make a #UXdesign .

I'm working as one of the engineering leads in RunaHR. As our platform is a Saas, we thought It'd be good to have an API (We chose Ruby and Rails for this) and a SPA (built with React and Redux ) connected. We started the SPA with Create React App since It's pretty easy to start.

We use Jest as the testing framework and react-testing-library to test React components. In Rails we make tests using RSpec.

Our main database is PostgreSQL, but we also use MongoDB to store some type of data. We started to use Redis  for cache and other time sensitive operations.

We have a couple of extra projects: One is an Employee app built with React Native and the other is an internal back office dashboard built with Next.js for the client and Python in the backend side.

Since we have different frontend apps we have found useful to have Bit to document visual components and utils in JavaScript.

Since you said that your middleware will be accessing DB and expose API, you can go with Node.js. It will make your development fast and easy. Suppose in future you will add some business logic you can choose Java with Spring Boot or Python with Flask / Django. NOTE: Language or framework doesn't matter. Choose based on your programming knowledge.

In our company we have think a lot about languages that we're willing to use, there we have considering Java, Python and C++ . All of there languages are old and well developed at fact but that's not ideology of araclx. We've choose a edge technologies such as Node.js , Rust , Kotlin and Go as our programming languages which is some kind of fun. Node.js is one of biggest trends of 2019, same for Go. We want to grow in our company with growth of languages we have choose, and probably when we would choose Java that would be almost impossible because larger languages move on today's market slower, and cannot have big changes.