AWS Lambda

In a couple of recent projects we had an opportunity to try out the new Serverless approach to building web applications. It wasn't necessarily a question if we should use any particular vendor but rather "if" we can consider serverless a viable option for building apps. Obviously our goal was also to get a feel for this technology and gain some hands-on experience.

We did consider AWS Lambda, Firebase from Google as well as Azure Functions. Eventually we went with AWS Lambdas.

• No servers to manage (obviously!)
• Limited fixed costs – you pay only for used time
• Automated scaling and balancing
• Automatic failover (or, at this level of abstraction, no failover problem at all)
• Security easier to provide and audit
• Low overhead at the start (with the certain level of knowledge)
• Short time to market
• Easy handover - deployment coupled with code
• Perfect choice for lean startups with fast-paced iterations
• Augmentation for the classic cloud, server(full) approach

• Not much know-how and best practices available about structuring the code and projects on the market
• Not suitable for complex business logic due to the risk of producing highly coupled code
• Cost difficult to estimate (helpful tools: serverlesscalc.com)
• Difficulty in migration to other platforms (Vendor lock⚠️)
• Little engineers with experience in serverless on the job market
• Steep learning curve for engineers without any cloud experience

More details are on our blog: https://evojam.com/blog/2018/12/5/should-you-go-serverless-meet-the-benefits-and-flaws-of-new-wave-of-cloud-solutions I hope it helps 🙌 & I'm curious of your experiences.

At Compass, we’re big proponents of using NPM and semver (semantic versioning) when distributing our shared components as packages. NPM provides us with an industry-standard platform to publish our internal dependencies. The tools and technologies someone learns while working on a package at Compass are the same ones they’ll use in projects in the open source community. Meanwhile, semantic versioning itself plays a huge role in providing peace of mind. Users of shared components know when updates are safe enough to upgrade to, and component authors can make big updates without the fear of silently breaking the contracts they’ve made with their users. We wanted to build out a way to provide these same benefits to more than just JS libraries, and we ended up creating a lightning-fast form of semantic versioning for our CSS implementation that utilized Lambda@Edge, NPM, and some clever work by our engineers.

AWS Lambda Amazon CloudFront npm #lambdaatedge #semver #serverless

CodeFactor being a #SAAS product, our goal was to run on a cloud-native infrastructure since day one. We wanted to stay product focused, rather than having to work on the infrastructure that supports the application. We needed a cloud-hosting provider that would be reliable, economical and most efficient for our product.

CodeFactor.io aims to provide an automated and frictionless code review service for software developers. That requires agility, instant provisioning, autoscaling, security, availability and compliance management features. We looked at the top three #IAAS providers that take up the majority of market share: Amazon's Amazon EC2 , Microsoft's Microsoft Azure, and Google Compute Engine.

AWS has been available since 2006 and has developed the most extensive services ant tools variety at a massive scale. Azure and GCP are about half the AWS age, but also satisfied our technical requirements.

It is worth noting that even though all three providers support Docker containerization services, GCP has the most robust offering due to their investments in Kubernetes. Also, if you are a Microsoft shop, and develop in .NET - Visual Studio Azure shines at integration there and all your existing .NET code works seamlessly on Azure. All three providers have serverless computing offerings (AWS Lambda, Azure Functions, and Google Cloud Functions). Additionally, all three providers have machine learning tools, but GCP appears to be the most developer-friendly, intuitive and complete when it comes to #Machinelearning and #AI.

The prices between providers are competitive across the board. For our requirements, AWS would have been the most expensive, GCP the least expensive and Azure was in the middle. Plus, if you #Autoscale frequently with large deltas, note that Azure and GCP have per minute billing, where AWS bills you per hour. We also applied for the #Startup programs with all three providers, and this is where Azure shined. While AWS and GCP for startups would have covered us for about one year of infrastructure costs, Azure Sponsorship would cover about two years of CodeFactor's hosting costs. Moreover, Azure Team was terrific - I felt that they wanted to work with us where for AWS and GCP we were just another startup.

In summary, we were leaning towards GCP. GCP's advantages in containerization, automation toolset, #Devops mindset, and pricing were the driving factors there. Nevertheless, we could not say no to Azure's financial incentives and a strong sense of partnership and support throughout the process.

Bottom line is, IAAS offerings with AWS, Azure, and GCP are evolving fast. At CodeFactor, we aim to be platform agnostic where it is practical and retain the flexibility to cherry-pick the best products across providers.

At Volta we use AWS CodePipeline and AWS CodeBuild to automatically ship new AWS Lambda services without any effort. And we do it all with AWS CloudFormation, since configuration is easier to maintain than code.

We are starting to build one shirt data logic, structure and as an online clothing store we believe good ux and ui is a goal to drive a lot of click through. The problem is, how do we fetch data and how do we abstract the gap between the Front-end devs and backend-devs as we are just two in the technical unit. We decided to go for GraphQL as our application-layer tool and Prisma for our database-layer abstracter.

Reasons :

GraphQL :

1. GraphQL makes fetching of data less painful and organised.

2. GraphQL gives you 100% assurance on data you getting back as opposed to the Rest design .

3. GraphQL comes with a bunch of real-time functionality in form of. subscriptions and finally because we are using React (GraphQL is not React demanding, it's doesn't require a specific framework, language or tool, but it definitely makes react apps fly )

Prisma :

1. Writing revolvers can be fun, but imagine writing revolvers nested deep down, curry braces flying around. This is sure a welcome note to bugs and as a small team we need to focus more on what that matters more. Prisma generates this necessary CRUD resolves, mutations and subscription out of the box.

2. We don't really have much budget at the moment so we are going to run our logic in a scalable cheap and cost effective cloud environment. Oh! It's AWS Lambda and deploying our schema to Lambda is our best bet to minimize cost and same time scale.

We are still at development stage and I believe, working on this start up will increase my dev knowledge. Off for Lunch :)

I'm currently building out a Twitter analysis tool that's using AWS Lambda to stream data into Amazon Kinesis Firehose, which in turns saves the result to Amazon S3. The plan is to have Amazon S3 operate as both a data store and quasi-messaging bus with any post-processing work (e.g., notifications of new tweets going into Slack) fanning out from there. I went with this approach as I can get things up and running quickly and only pay for things on a pay-per-use basis rather than having lots of worker nodes sitting around waiting for work. Amazon Kinesis Firehose also makes it easy to add a different or additional data store in the future.

Working on a project recently, wanted an easy modern frontend to work with, decoupled from our backend. To get things going quickly, decided to go with React, Redux.js, redux-saga, Bootstrap.

On the backend side, Go is a personal favourite, and wanted to minimize server overheads so went with a #serverless architecture leveraging AWS Lambda, AWS CloudFormation, Amazon DynamoDB, etc.

For IDE/tooling I tend to stick to the #JetBrains tools: WebStorm / Goland.

Obviously using Git, with GitLab private repo's for managing code/issues/etc.

In order to accurately measure & track user behaviour on our platform we moved over quickly from the initial solution using Google Analytics to a custom-built one due to resource & pricing concerns we had.

While this does sound complicated, it’s as easy as clients sending JSON blobs of events to Amazon Kinesis from where we use AWS Lambda & Amazon SQS to batch and process incoming events and then ingest them into Google BigQuery. Once events are stored in BigQuery (which usually only takes a second from the time the client sends the data until it’s available), we can use almost-standard-SQL to simply query for data while Google makes sure that, even with terabytes of data being scanned, query times stay in the range of seconds rather than hours. Before ingesting their data into the pipeline, our mobile clients are aggregating events internally and, once a certain threshold is reached or the app is going to the background, sending the events as a JSON blob into the stream.

In the past we had workers running that continuously read from the stream and would validate and post-process the data and then enqueue them for other workers to write them to BigQuery. We went ahead and implemented the Lambda-based approach in such a way that Lambda functions would automatically be triggered for incoming records, pre-aggregate events, and write them back to SQS, from which we then read them, and persist the events to BigQuery. While this approach had a couple of bumps on the road, like re-triggering functions asynchronously to keep up with the stream and proper batch sizes, we finally managed to get it running in a reliable way and are very happy with this solution today.

#ServerlessTaskProcessing #GeneralAnalytics #RealTimeDataProcessing #BigDataAsAService

Whenever we need to notify a user of something happening on our platform, whether it’s a personal push notification from one user to another, a new Dub, or a notification going out to millions of users at the same time that new content is available, we rely on AWS Lambda to do this task for us. When we started implementing this feature 2 years ago we were luckily able to get early access to the Lambda Beta and are still happy with the way things are running on there, especially given all the easy to set up integrations with other AWS services.

Lambda enables us to quickly send out million of pushes within a couple of minutes by acting as a multiplexer in front of Amazon SNS. We simply call a first Lambda function with a batch of up to 300 push notifications to be sent, which then calls a subsequent Lambda function with 20 pushes each, which then does the call to SNS to actually send out the push notifications.

This multi-tier process of sending push notifications enables us to quickly adjust our sending volume while keeping costs & maintenance overhead, on our side, to a bare minimum.

#ApplicationHosting