Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

AWS Batch
AWS Batch

19
6
+ 1
0
AWS Lambda
AWS Lambda

5K
3.5K
+ 1
383
Add tool

AWS Batch vs AWS Lambda: What are the differences?

Developers describe AWS Batch as "Fully Managed Batch Processing at Any Scale". It enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS. It dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted. On the other hand, AWS Lambda is detailed as "Automatically run code in response to modifications to objects in Amazon S3 buckets, messages in Kinesis streams, or updates in DynamoDB". AWS Lambda is a compute service that runs your code in response to events and automatically manages the underlying compute resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own back-end services that operate at AWS scale, performance, and security.

AWS Batch and AWS Lambda belong to "Serverless / Task Processing" category of the tech stack.

- No public GitHub repository available -
- No public GitHub repository available -

What is AWS Batch?

It enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS. It dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted.

What is AWS Lambda?

AWS Lambda is a compute service that runs your code in response to events and automatically manages the underlying compute resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own back-end services that operate at AWS scale, performance, and security.
Get Advice Icon

Need advice about which tool to choose?Ask the StackShare community!

Why do developers choose AWS Batch?
Why do developers choose AWS Lambda?
    Be the first to leave a pro

    Sign up to add, upvote and see more prosMake informed product decisions

      Be the first to leave a con
      What companies use AWS Batch?
      What companies use AWS Lambda?

      Sign up to get full access to all the companiesMake informed product decisions

      What tools integrate with AWS Batch?
      What tools integrate with AWS Lambda?
        No integrations found

        Sign up to get full access to all the tool integrationsMake informed product decisions

        What are some alternatives to AWS Batch and AWS Lambda?
        Beanstalk
        A single process to commit code, review with the team, and deploy the final result to your customers.
        Airflow
        Use Airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The Airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command lines utilities makes performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress and troubleshoot issues when needed.
        Serverless
        Build applications comprised of microservices that run in response to events, auto-scale for you, and only charge you when they run. This lowers the total cost of maintaining your apps, enabling you to build more logic, faster. The Framework uses new event-driven compute services, like AWS Lambda, Google CloudFunctions, and more.
        Cloud Functions for Firebase
        Cloud Functions for Firebase lets you create functions that are triggered by Firebase products, such as changes to data in the Realtime Database, uploads to Cloud Storage, new user sign ups via Authentication, and conversion events in Analytics.
        Google Cloud Functions
        Construct applications from bite-sized business logic billed to the nearest 100 milliseconds, only while your code is running
        See all alternatives
        Decisions about AWS Batch and AWS Lambda
        Kestas Barzdaitis
        Kestas Barzdaitis
        Entrepreneur & Engineer · | 14 upvotes · 79.2K views
        atCodeFactorCodeFactor
        Kubernetes
        Kubernetes
        CodeFactor.io
        CodeFactor.io
        Amazon EC2
        Amazon EC2
        Microsoft Azure
        Microsoft Azure
        Google Compute Engine
        Google Compute Engine
        Docker
        Docker
        AWS Lambda
        AWS Lambda
        Azure Functions
        Azure Functions
        Google Cloud Functions
        Google Cloud Functions
        #SAAS
        #IAAS
        #Containerization
        #Autoscale
        #Startup
        #Automation
        #Machinelearning
        #AI
        #Devops

        CodeFactor being a #SAAS product, our goal was to run on a cloud-native infrastructure since day one. We wanted to stay product focused, rather than having to work on the infrastructure that supports the application. We needed a cloud-hosting provider that would be reliable, economical and most efficient for our product.

        CodeFactor.io aims to provide an automated and frictionless code review service for software developers. That requires agility, instant provisioning, autoscaling, security, availability and compliance management features. We looked at the top three #IAAS providers that take up the majority of market share: Amazon's Amazon EC2 , Microsoft's Microsoft Azure, and Google Compute Engine.

        AWS has been available since 2006 and has developed the most extensive services ant tools variety at a massive scale. Azure and GCP are about half the AWS age, but also satisfied our technical requirements.

        It is worth noting that even though all three providers support Docker containerization services, GCP has the most robust offering due to their investments in Kubernetes. Also, if you are a Microsoft shop, and develop in .NET - Visual Studio Azure shines at integration there and all your existing .NET code works seamlessly on Azure. All three providers have serverless computing offerings (AWS Lambda, Azure Functions, and Google Cloud Functions). Additionally, all three providers have machine learning tools, but GCP appears to be the most developer-friendly, intuitive and complete when it comes to #Machinelearning and #AI.

        The prices between providers are competitive across the board. For our requirements, AWS would have been the most expensive, GCP the least expensive and Azure was in the middle. Plus, if you #Autoscale frequently with large deltas, note that Azure and GCP have per minute billing, where AWS bills you per hour. We also applied for the #Startup programs with all three providers, and this is where Azure shined. While AWS and GCP for startups would have covered us for about one year of infrastructure costs, Azure Sponsorship would cover about two years of CodeFactor's hosting costs. Moreover, Azure Team was terrific - I felt that they wanted to work with us where for AWS and GCP we were just another startup.

        In summary, we were leaning towards GCP. GCP's advantages in containerization, automation toolset, #Devops mindset, and pricing were the driving factors there. Nevertheless, we could not say no to Azure's financial incentives and a strong sense of partnership and support throughout the process.

        Bottom line is, IAAS offerings with AWS, Azure, and GCP are evolving fast. At CodeFactor, we aim to be platform agnostic where it is practical and retain the flexibility to cherry-pick the best products across providers.

        See more
        Nitzan Shapira
        Nitzan Shapira
        at Epsagon · | 11 upvotes · 117.3K views
        atEpsagonEpsagon
        Python
        Python
        Serverless
        Serverless
        npm
        npm
        Node.js
        Node.js
        Go
        Go
        Java
        Java
        GitHub
        GitHub
        AWS Lambda
        AWS Lambda

        At Epsagon, we use hundreds of AWS Lambda functions, most of them are written in Python, and the Serverless Framework to pack and deploy them. One of the issues we've encountered is the difficulty to package external libraries into the Lambda environment using the Serverless Framework. This limitation is probably by design since the external code your Lambda needs can be usually included with a package manager.

        In order to overcome this issue, we've developed a tool, which we also published as open-source (see link below), which automatically packs these libraries using a simple npm package and a YAML configuration file. Support for Node.js, Go, and Java will be available soon.

        The GitHub respoitory: https://github.com/epsagon/serverless-package-external

        See more
        Michal Nowak
        Michal Nowak
        Co-founder at Evojam · | 7 upvotes · 74.3K views
        atEvojamEvojam
        Serverless
        Serverless
        AWS Lambda
        AWS Lambda
        Firebase
        Firebase
        Azure Functions
        Azure Functions

        In a couple of recent projects we had an opportunity to try out the new Serverless approach to building web applications. It wasn't necessarily a question if we should use any particular vendor but rather "if" we can consider serverless a viable option for building apps. Obviously our goal was also to get a feel for this technology and gain some hands-on experience.

        We did consider AWS Lambda, Firebase from Google as well as Azure Functions. Eventually we went with AWS Lambdas.

        PROS
        • No servers to manage (obviously!)
        • Limited fixed costs – you pay only for used time
        • Automated scaling and balancing
        • Automatic failover (or, at this level of abstraction, no failover problem at all)
        • Security easier to provide and audit
        • Low overhead at the start (with the certain level of knowledge)
        • Short time to market
        • Easy handover - deployment coupled with code
        • Perfect choice for lean startups with fast-paced iterations
        • Augmentation for the classic cloud, server(full) approach
        CONS
        • Not much know-how and best practices available about structuring the code and projects on the market
        • Not suitable for complex business logic due to the risk of producing highly coupled code
        • Cost difficult to estimate (helpful tools: serverlesscalc.com)
        • Difficulty in migration to other platforms (Vendor lock⚠️)
        • Little engineers with experience in serverless on the job market
        • Steep learning curve for engineers without any cloud experience

        More details are on our blog: https://evojam.com/blog/2018/12/5/should-you-go-serverless-meet-the-benefits-and-flaws-of-new-wave-of-cloud-solutions I hope it helps 🙌 & I'm curious of your experiences.

        See more
        Jeyabalaji Subramanian
        Jeyabalaji Subramanian
        CTO at FundsCorner · | 12 upvotes · 356.4K views
        atFundsCornerFundsCorner
        Amazon SQS
        Amazon SQS
        Python
        Python
        Flask
        Flask
        MongoDB
        MongoDB
        Swagger UI
        Swagger UI
        Vue.js
        Vue.js
        Vuetify
        Vuetify
        vuex
        vuex
        Zappa
        Zappa
        AWS Lambda
        AWS Lambda
        Netlify
        Netlify
        Google Compute Engine
        Google Compute Engine
        Slack
        Slack
        GitLab CI
        GitLab CI
        Sentry
        Sentry

        At FundsCorner, we are on a mission to enable fast accessible credit to India’s Kirana Stores. We are an early stage startup with an ultra small Engineering team. All the tech decisions we have made until now are based on our core philosophy: "Build usable products fast".

        Based on the above fundamentals, we chose Python as our base language for all our APIs and micro-services. It is ultra easy to start with, yet provides great libraries even for the most complex of use cases. Our entire backend stack runs on Python and we cannot be more happy with it! If you are looking to deploy your API as server-less, Python provides one of the least cold start times.

        We build our APIs with Flask. For backend database, our natural choice was MongoDB. It frees up our time from complex database specifications - we instead use our time in doing sensible data modelling & once we finalize the data model, we integrate it into Flask using Swagger UI. Mongo supports complex queries to cull out difficult data through aggregation framework & we have even built an internal framework called "Poetry", for aggregation queries.

        Our web apps are built on Vue.js , Vuetify and vuex. Initially we debated a lot around choosing Vue.js or React , but finally settled with Vue.js, mainly because of the ease of use, fast development cycles & awesome set of libraries and utilities backing Vue.

        You simply cannot go wrong with Vue.js . Great documentation, the library is ultra compact & is blazing fast. Choosing Vue.js was one of the critical decisions made, which enabled us to launch our web app in under a month (which otherwise would have taken 3 months easily). For those folks who are looking for big names, Adobe, and Alibaba and Gitlab are using Vue.

        By choosing Vuetify, we saved thousands of person hours in designing the CSS files. Vuetify contains all key material components for designing a smooth User experience & it just works! It's an awesome framework. All of us at FundsCorner are now lifelong fanboys of Vue.js and Vuetify.

        On the infrastructure side, all our API services and backend services are deployed as server less micro-services through Zappa. Zappa makes your life super easy by packaging everything that is required to deploy your code as AWS Lambda. We are now addicted to the single - click deploys / updates through Zappa. Try it out & you will convert!

        Also, if you are using Zappa, you can greatly simplify your CI / CD pipelines. Do try it! It's just awesome! and... you will be astonished by the savings you have made on AWS bills at end of the month.

        Our CI / CD pipelines are built using GitLab CI. The documentation is very good & it enables you to go from from concept to production in minimal time frame.

        We use Sentry for all crash reporting and resolution. Pro tip, they do have handlers for AWS Lambda , which made our integration super easy.

        All our micro-services including APIs are event-driven. Our background micro-services are message oriented & we use Amazon SQS as our message pipe. We have our own in-house workflow manager to orchestrate across micro - services.

        We host our static websites on Netlify. One of the cool things about Netlify is the automated CI / CD on git push. You just do a git push to deploy! Again, it is super simple to use and it just works. We were dogmatic about going server less even on static web sites & you can go server less on Netlify in a few minutes. It's just a few clicks away.

        We use Google Compute Engine, especially Google Vision for our AI experiments.

        For Ops automation, we use Slack. Slack provides a super-rich API (through Slack App) through which you can weave magical automation on boring ops tasks.

        See more
        Jeyabalaji Subramanian
        Jeyabalaji Subramanian
        CTO at FundsCorner · | 24 upvotes · 340.4K views
        atFundsCornerFundsCorner
        MongoDB
        MongoDB
        PostgreSQL
        PostgreSQL
        MongoDB Stitch
        MongoDB Stitch
        Node.js
        Node.js
        Amazon SQS
        Amazon SQS
        Python
        Python
        SQLAlchemy
        SQLAlchemy
        AWS Lambda
        AWS Lambda
        Zappa
        Zappa

        Recently we were looking at a few robust and cost-effective ways of replicating the data that resides in our production MongoDB to a PostgreSQL database for data warehousing and business intelligence.

        We set ourselves the following criteria for the optimal tool that would do this job: - The data replication must be near real-time, yet it should NOT impact the production database - The data replication must be horizontally scalable (based on the load), asynchronous & crash-resilient

        Based on the above criteria, we selected the following tools to perform the end to end data replication:

        We chose MongoDB Stitch for picking up the changes in the source database. It is the serverless platform from MongoDB. One of the services offered by MongoDB Stitch is Stitch Triggers. Using stitch triggers, you can execute a serverless function (in Node.js) in real time in response to changes in the database. When there are a lot of database changes, Stitch automatically "feeds forward" these changes through an asynchronous queue.

        We chose Amazon SQS as the pipe / message backbone for communicating the changes from MongoDB to our own replication service. Interestingly enough, MongoDB stitch offers integration with AWS services.

        In the Node.js function, we wrote minimal functionality to communicate the database changes (insert / update / delete / replace) to Amazon SQS.

        Next we wrote a minimal micro-service in Python to listen to the message events on SQS, pickup the data payload & mirror the DB changes on to the target Data warehouse. We implemented source data to target data translation by modelling target table structures through SQLAlchemy . We deployed this micro-service as AWS Lambda with Zappa. With Zappa, deploying your services as event-driven & horizontally scalable Lambda service is dumb-easy.

        In the end, we got to implement a highly scalable near realtime Change Data Replication service that "works" and deployed to production in a matter of few days!

        See more
        Julien DeFrance
        Julien DeFrance
        Principal Software Engineer at Tophatter · | 2 upvotes · 16K views
        atSmartZipSmartZip
        Serverless
        Serverless
        AWS Lambda
        AWS Lambda
        Amazon Machine Learning
        Amazon Machine Learning
        Amazon SageMaker
        Amazon SageMaker
        #PaaS
        #GCP
        #FaaS

        Which #IaaS / #PaaS to chose? Not all #Cloud providers are created equal. As you start to use one or the other, you'll build around very specific services that don't have their equivalent elsewhere.

        Back in 2014/2015, this decision I made for SmartZip was a no-brainer and #AWS won. AWS has been a leader, and over the years demonstrated their capacity to innovate, and reducing toil. Like no other.

        Year after year, this kept on being confirmed, as they rolled out new (managed) services, got into Serverless with AWS Lambda / FaaS And allowed domains such as #AI / #MachineLearning to be put into the hands of every developers thanks to Amazon Machine Learning or Amazon SageMaker for instance.

        Should you compare with #GCP for instance, it's not quite there yet. Building around these managed services, #AWS allowed me to get my developers on a whole new level. Where they know what's under the hood. Where they know they have these services available and can build around them. Where they care and are responsible for operations and security and deployment of what they've worked on.

        See more
        Aviad Mor
        Aviad Mor
        CTO & Co-Founder at Lumigo · | 5 upvotes · 11.5K views
        atLumigoLumigo
        AWS Lambda
        AWS Lambda
        CircleCI
        CircleCI
        Serverless
        Serverless

        Our backend is serverless based, with many AWS Lambda , with CI/CD, using CircleCI and Serverless. This allows to develop with awesome agility and move fast. Since we update our lambdas daily, we needed a way to make sure we did not run into AWS's max limit of versions per lambda. We use the open source in link below to clear them out and stay clear of the limit.

        See more
        Google Cloud Functions
        Google Cloud Functions
        AWS Lambda
        AWS Lambda

        I use Google Cloud Functions because it's the AWS Lambda equivalent on GCP. It's not as mature compared to lambda because it doesn't have VPC enablement unless done through VPC Service Controls which can be pretty cumbersome.

        Although it feels bare bones compared to lambda, it still gets the job done when you want backend tasks done via serverless.

        Example Use Case

        See more
        Tim Nolet
        Tim Nolet
        Founder, Engineer & Dishwasher at Checkly · | 5 upvotes · 26.5K views
        atChecklyHQChecklyHQ
        AWS Lambda
        AWS Lambda
        Serverless
        Serverless
        Amazon CloudWatch
        Amazon CloudWatch
        Azure Functions
        Azure Functions
        Google Cloud Functions
        Google Cloud Functions
        Node.js
        Node.js

        AWS Lambda Serverless Amazon CloudWatch Azure Functions Google Cloud Functions Node.js

        In the last year or so, I moved all Checkly monitoring workloads to AWS Lambda. Here are some stats:

        • We run three core functions in all AWS regions. They handle API checks, browser checks and setup / teardown scripts. Check our docs to find out what that means.
        • All functions are hooked up to SNS topics but can also be triggered directly through AWS SDK calls.
        • The busiest function is a plumbing function that forwards data to our database. It is invoked anywhere between 7000 and 10.000 times per hour with an average duration of about 179 ms.
        • We run separate dev and test versions of each function in each region.

        Moving all this to AWS Lambda took some work and considerations. The blog post linked below goes into the following topics:

        • Why Lambda is an almost perfect match for SaaS. Especially when you're small.
        • Why I don't use a "big" framework around it.
        • Why distributed background jobs triggered by queues are Lambda's raison d'être.
        • Why monitoring & logging is still an issue.

        https://blog.checklyhq.com/how-i-made-aws-lambda-work-for-my-saas/

        See more
        Praveen Mooli
        Praveen Mooli
        Technical Leader at Taylor and Francis · | 11 upvotes · 216.3K views
        MongoDB Atlas
        MongoDB Atlas
        Java
        Java
        Spring Boot
        Spring Boot
        Node.js
        Node.js
        ExpressJS
        ExpressJS
        Python
        Python
        Flask
        Flask
        Amazon Kinesis
        Amazon Kinesis
        Amazon Kinesis Firehose
        Amazon Kinesis Firehose
        Amazon SNS
        Amazon SNS
        Amazon SQS
        Amazon SQS
        AWS Lambda
        AWS Lambda
        Angular 2
        Angular 2
        RxJS
        RxJS
        GitHub
        GitHub
        Travis CI
        Travis CI
        Terraform
        Terraform
        Docker
        Docker
        Serverless
        Serverless
        Amazon RDS
        Amazon RDS
        Amazon DynamoDB
        Amazon DynamoDB
        Amazon S3
        Amazon S3
        #Backend
        #Microservices
        #Eventsourcingframework
        #Webapps
        #Devops
        #Data

        We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.

        To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas

        To build #Webapps we decided to use Angular 2 with RxJS

        #Devops - GitHub , Travis CI , Terraform , Docker , Serverless

        See more
        Interest over time
        Reviews of AWS Batch and AWS Lambda
        Review ofAWS LambdaAWS Lambda

        I switched my auto chatbot to run in lambda and it was peace !

        How developers use AWS Batch and AWS Lambda
        Avatar of Nathan Heffley
        Nathan Heffley uses AWS LambdaAWS Lambda

        To use Pusher's presence channel each client must be connected through a backend authentication system. While Pointer doesn't actually have any login based authentication it still needed a backend system to connect users to the proper channel.

        A small function was built that only gets called when a user first joins a session. After the user is authenticated they can communicate directly with other clients on the same channel. This made the authentication code the perfect candidate for a serverless function. Using AWS Lambda through Netlify's Functions feature made it a breeze to host.

        Avatar of Simple Merchant
        Simple Merchant uses AWS LambdaAWS Lambda

        We're moving almost the entirety of our backend processes into Lambda. This has given us vast cost savings in terms of pure infrastructure billing - and time worrying about platform and scale. This move has also made our architecture almost entirely event-driven - another huge benefit as our business itself is inherently event-driven.

        Avatar of Volkan Özçelik
        Volkan Özçelik uses AWS LambdaAWS Lambda

        I mostly use AWS Lambda for triggering DevOps-related actions, like triggering an alarm or a deployment, or scheduling a backup.

        I haven’t gone totally “serverless” and I’m not planning to go 100% serverless anytime soon.

        But when I do, AWS Lambda will be an important element in my serverless setup.

        Avatar of Promethean TV
        Promethean TV uses AWS LambdaAWS Lambda

        PrometheanTV uses various Lambda functions to provide back-end capabilities to the platform without the need of deploying servers. Examples include, geo lookup services, and data aggregation services.

        Avatar of Flux Work
        Flux Work uses AWS LambdaAWS Lambda

        Serverless is the future. And AWS Lambda is the most mature FaaS out there. AWS SAM makes it easy to package Lambda as micro-apps.

        How much does AWS Batch cost?
        How much does AWS Lambda cost?
        Pricing unavailable
        News about AWS Batch
        More news