I am using Twilio as my base application, which one should I prefer Twilio Functions or AWS Lambda function to host my functions which are called from Twilio plugins?
AWS Lambda
Which one is the best between Serverless framework and Claudia for AWS Lambda deployment? I just need the detailed difference between them to understand which is better.
If you're ONLY using Lambda I would recommend Claudia. It is a lot simpler of a framework and it is also pretty easy to configure and use without having to learn much. If you plan on using the wider AWS ecosystem I would recommend Serverless or SST.
Both Serverless and SST will give you more fine grained control over AWS resources and makes it easier to work with either through CloudFormation or CDK.
Hey,
I want to stream incremental document updates (like a price change) to a ton of end users using SSE. What is the best tool for that? AWS Lambda's / Cloud Functions terminate too fast and Containers / Servers seem too expensive.
Users should authenticate and "subscribe" to a specific resource. I am planning to use a MongoDB Instance as a Database and Lambdas to write to it.
Hi Marc,
For the com part, depending of more details not provided, i'd use SSE, OR i'd run either Mosquitto or RabbitMQ running on Amazon EC2 instances and leverage MQTT or amqp 's subscribe/publish features with my users running mqtt or amqp clients (tcp or websockets) somehow. (publisher too.. you don't say how and who gets to update the document(s).
I find "a ton of end users", depending on how you define a ton (1k users ;) ?) and how frequent document updates are, that can mean a ton of ressources, can't cut it at some point, even using SSE
how many, how big, how persistant do the document(s) have to be ? Db-wise,can't say for lack of details and context, yeah could also be Redis, any RDBMS or nosql or even static json files stored on an Amazon S3 bucket .. anything really
Good luck!
What's the difference between AWS Lambda and AWS Step Functions?
I intend to use a programming language which I'll use as AWS runtime and write a script that will comb through tons of files in a directory and its subdirectories and search for simple text regular expressions and process and write the matches in a file as output. I have heard that Perl is good for regex based search but I also want the performance to be good as it will have to go through tons of files for IO. In this post: https://filia-aleks.medium.com/aws-lambda-battle-2021-performance-comparison-for-all-languages-c1b441005fd1, I see that Rust works well as AWS Lambda runtime with very good performance. Which one should I choose as my AWS lambda runtime for this problem? Golang is also an option as it is fast as per the above link.
I used to work in a Perl shop and must admit that the language is very simple for tasks like these, but as you mentioned it's not fast at execution time. I'm now a Go programmer professionally but I taught myself the language while in college purely out of interest and eventually found my way to the job, not the other way around. I've recently been learning a little rust because of how much that language comes up in conversations around Go. I find the concept of the borrow checker nice but I have to admit I feel lost like I am in most flavors of new fancy framework js. That's not to say Rust is really anything like js, but the learning appears the same to me as someone who's convinced they could learn just about any programming language if it was necessary (over time I've seen procedural, OOP, declarative and functional stuff but never programming logic outside of the prolog code I wrote in school).
Go isn't made for your specific task at hand but it's a very easy language to pick up and it has good directory traversal standard library code and good regex (even though with time perl's has been optimized to be faster and I think it's written in C++) but more than anything Go is "cloud native" programming in that an awful lot of new microservice tech stacks are centered around it, docker and kubernetes are written in it, and there's a thriving community whose focus is generally web-first and performance-oriented. This means for your use case there might already be a large cohort of gophers that have asked the stackoverflow questions for you
I personally would push you towards the NYT Profiler for Perl before I would towards Rest, but that's because I know you wouldn't waste any time being able to get to the task at hand and then make it go faster, and I expect all but a few rustaceans would be able to do so with the same speed.
Whatever you pick I wish you the very best of luck!
Perl is super fast to write but only if you have lot of experience in it
Go is maintainable and faster in performance
Rust is fastest in performance
Go is "maintainable" up to a point, I find that error handling and interfaces typically require more dev and maintainance time than in other languages, because they are too simple.
We are working on a transaction processing application and everything needs to happen in real-time. I have used AWS Lambda with Quarkus + Java, and struggled with cold start time however Lambda is still cost effective. so wanted to understand which one will be suitable as alternative to lambda services.
The main question here is your request rate. - If it's usually very high, then in most cases you shouldn't run into cold starts - and for the rare gaps, you can send a steady stream of "blank" requests that will keep the lambda's alive. - You can also consider pushing all your incoming requests to an SQS, and then triggering lambdas based on the sqs capacity - this way you'll be able to provide a constant level of service even with the cold starts
I am currently working on a long term mobile app project. Current stack: Frontend: Dart/Flutter Backend: Go, AWS Resources (AWS Lambda, Amazon DynamoDB, etc.) Since there are only two developers and we have limited time and resources, we are looking for a BAAS like Firebase or AWS Amplify to handle auth and push notifications for now. We are prioritizing developing speed so we can iterate quickly. The only problem is that AWS amplify support for flutter is in developer preview and has limited capabilities (We have tested it out in our app). Firebase is the more mature option. It has great support for flutter and has more than we need for auth, notifications, etc. My question is that, if we choose firebase, we would be stuck with using two different cloud providers. Is this bad, or is this even a problem? I am willing to change anything on the backend architecture wise, so any suggestions would be greatly appreciated as I am somewhat unfamiliar with Google Cloud Platform. Thank you.
I have never understood why using different vendors is such a big problem for many people. Each vendor is better than others for certain aspects, but no vendor is better than others for everything. At Kokoen, we usually host the applications we develop in AWS (EC2, RDS, S3), use Firebase (usually with OneSignal.com) for notifications, use Netlify for hosting certain SPAs, use Heroku to provide APIs directly from Git while we are still developing, MongoDB Cloud when we need NonSQL databases, etc..
I am trying to write a cloud function that needs to perform a specific task. I will be using AWS Lambda for executing the function. I have 3 choices for stacks to write the lambda function: Plain Java, Node.js, or a Spring Boot stack. I have an idea of the time that the lambda would take to execute for a Hello World example for each of these stacks, and based on that, it is a close competition between Java and Node.js. I want to get some insight into what to chose and also how it would affect if my lambda would need scaling.
I find Spring Boot is the most productive environment, and written properly, it is fast. However, speed is the least of your concerns. You should be considering support and maintenance, along with time to market. The best tool for the job is the one that you can support 5 years in to the future. It is the one that will return dollars from customers soonest. You can always scale up your AWS Lambda servers to get more power.
Hi Nikhil, I agree either Node.js or Java can be used to write the AWS lambda function. As you didn't say the type of task is going to be performed in AWS function. I would recommend Node.js if the task has to run asynchronous (i.e) Don't wait and simple continue with the next request.
Some difference between Node JS and Java
- Node JS is open source where as Public updates for Oracle Java SE 8 released after January 2019 will not be available for business, commercial, or production use without a commercial license, as Oracle announced.
- Node.JS uses asynchronous programming language, so its easier to write and executions is non blocking.
Agreed, however my task is not asynchronous in nature. Although, its still a lightweight task so I think this argument would still apply. Thanks for your insight!
I planned to do a project in Cloud Firestore, which will store about 100GB of data. Shall I go for Cloud Firestore or traditional AWS RDS MS-SQL SERVER with AWS Lambda? Please I need your suggestion.
I am building an API that can be achieved from either.
It's a simple CRUD API.
Is there a well-known public API in production by a known company powered by AWS Lambda?
I see that everybody uses containers instead.
We have microservices in production with both technologies (hosting simple CRUDs, more complex async processes, GraphQL endpoints, etc). As a matter of preference, for greenfield projects, we tend to go full serverless (lambda, dynamo, aurora...), because is not only cheaper in general, but way easier to continuously integrate with bitbucket pipelines, github actions, etc. The drawback is still the cold starts, despite the last improvements in VPC cold starts; it's way less noticeable in non statically typed languages though (Node.js, Python...). You will have to keep the lambdas warmed with Cloudwatch events if you can't wait for a few seconds after the first invocation, or use the new feature Provisioned Concurrency (which we haven't), that kind of defeat the purpose of being serverless.
As to how to host different methods of an API in lambdas, there are different approaches: we have for example stacks of cloud formation where every lambda host unique functionalities (one for detecting faces in images, another for cropping, another for comparing, etc), and also other stacks where the entire GraphQL API is hosted with Fastify in just one lambda. It really depends on your use-case.
If you host your API in ECS with an ALB, you won't have cold starts, it will automatically handle blue/green deployments with no disruption in the service while you deploy, but your bill will increase, and the infrastructure will be more difficult to automate in something like a template.yaml.
Give it a try with aws sam (Serverless Application Model), with a few commands, you can have a sample API running in a couple of minutes.
thanks for the great advice.
I actually did play with serverless and rebuilt an api to see how it works.
I wasn't using vpc so the cold start was not noticeable. However, what was no able to achieve:
lambda to lambda communication.
I was easy with microservices. gPRC and rest based microservices can communicate with each other easily. How would we achieve something similar in lambda?
Secondly, CRUD api: would you recommend serverless over ecs/eks?