AWS Batch vs AWS Lambda

Overview

AWS Lambda

Stacks26.0K

Followers18.8K

Votes432

AWS Batch

Stacks84

Followers251

Votes6

AWS Batch vs AWS Lambda: What are the differences?

AWS Batch and AWS Lambda are both services offered by Amazon Web Services (AWS) that enable developers to run and manage their applications at scale. However, there are some key differences between the two:

Scaling and Control: AWS Batch provides fine-grained control over the scaling and management of your batch computing workloads. It allows you to set up and manage queues, define job dependencies, and control batch scheduling. On the other hand, AWS Lambda automatically scales your application to handle incoming requests, without requiring manual management or configuration.
Pricing Model: AWS Batch follows a pay-as-you-go pricing model, where you are charged based on the number of vCPUs and memory resources used by your batch compute environments. In contrast, AWS Lambda follows a metered pricing model, where you are billed based on the number of requests and the duration of each request invocation.
Compute Environment: AWS Batch allows you to define and manage your own compute environments, where you have control over the type and capacity of the underlying EC2 instances used for batch processing. On the other hand, AWS Lambda abstracts the compute environment and automatically manages the infrastructure, allowing you to focus on writing code without worrying about the underlying resources.
Execution Limits: AWS Batch allows you to run long-running processes, where each job can run for hours or even days if needed. This is particularly useful for scientific and batch processing workloads. On the other hand, AWS Lambda has a maximum execution duration limit of five minutes, which makes it most suitable for short-lived tasks or event-driven applications.
Integration with Services: AWS Batch can be easily integrated with other AWS services such as Amazon S3, Amazon DynamoDB, and AWS Step Functions. This allows you to build complex workflows and process large amounts of data using batch processing. AWS Lambda also has integrations with various AWS services, but its main strength lies in its ability to respond to events and trigger serverless functions in response to actions from other services.
Dependencies and Language Support: AWS Batch supports running applications written in any programming language as long as it can be containerized. This gives you the flexibility to use your preferred programming language. On the other hand, AWS Lambda supports a wider range of programming languages out of the box, including Node.js, Java, Python, Ruby, C#, PowerShell, and Go. It also allows you to include external libraries and dependencies in your function code.

In summary, AWS Batch provides fine-grained control over batch processing workloads, while AWS Lambda offers automatic scaling and event-based execution for serverless functions.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on AWS Lambda, AWS Batch

Tim

CTO at Checkly Inc.

Sep 18, 2019

Needs adviceon

Heroku

AWS Lambda

When adding a new feature to Checkly rearchitecting some older piece, I tend to pick Heroku for rolling it out. But not always, because sometimes I pick AWS Lambda . The short story:

Developer Experience trumps everything.
AWS Lambda is cheap. Up to a limit though. This impact not only your wallet.
If you need geographic spread, AWS is lonely at the top.

The setup

Recently, I was doing a brainstorm at a startup here in Berlin on the future of their infrastructure. They were ready to move on from their initial, almost 100% Ec2 + Chef based setup. Everything was on the table. But we crossed out a lot quite quickly:

Pure, uncut, self hosted Kubernetes — way too much complexity
Managed Kubernetes in various flavors — still too much complexity
Zeit — Maybe, but no Docker support
Elastic Beanstalk — Maybe, bit old but does the job
Heroku
Lambda

It became clear a mix of PaaS and FaaS was the way to go. What a surprise! That is exactly what I use for Checkly! But when do you pick which model?

I chopped that question up into the following categories:

Developer Experience / DX 🤓
Ops Experience / OX 🐂 (?)
Cost 💵
Lock in 🔐

Read the full post linked below for all details

357k views357k

Comments

Detailed Comparison

AWS Lambda	AWS Batch
AWS Lambda is a compute service that runs your code in response to events and automatically manages the underlying compute resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own back-end services that operate at AWS scale, performance, and security.	It enables developers, scientists, and engineers to easily and efficiently run hundreds of thousands of batch computing jobs on AWS. It dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized instances) based on the volume and specific resource requirements of the batch jobs submitted.
Extend other AWS services with custom logic;Build custom back-end services;Completely Automated Administration;Built-in Fault Tolerance;Automatic Scaling;Integrated Security Model;Bring Your Own Code;Pay Per Use;Flexible Resource Model	-
Statistics
Stacks 26.0K	Stacks 84
Followers 18.8K	Followers 251
Votes 432	Votes 6
Pros & Cons
Pros 129 No infrastructure 83 Cheap 70 Quick 59 Stateless 47 No deploy, no server, great sleep Cons 7 Cant execute ruby or go 3 Compute time limited 1 Can't execute PHP w/o significant effort	Pros 3 Scalable 3 Containerized Cons 3 More overhead than lambda 1 Image management

What are some alternatives to AWS Lambda, AWS Batch?

Azure Functions

Azure Functions is an event driven, compute-on-demand experience that extends the existing Azure application platform with capabilities to implement code triggered by events occurring in virtually any Azure or 3rd party service as well as on-premises systems.

Google Cloud Run

A managed compute platform that enables you to run stateless containers that are invocable via HTTP requests. It's serverless by abstracting away all infrastructure management.

Serverless

Build applications comprised of microservices that run in response to events, auto-scale for you, and only charge you when they run. This lowers the total cost of maintaining your apps, enabling you to build more logic, faster. The Framework uses new event-driven compute services, like AWS Lambda, Google CloudFunctions, and more.

Google Cloud Functions

Construct applications from bite-sized business logic billed to the nearest 100 milliseconds, only while your code is running

Knative

Knative provides a set of middleware components that are essential to build modern, source-centric, and container-based applications that can run anywhere: on premises, in the cloud, or even in a third-party data center

OpenFaaS

Serverless Functions Made Simple for Docker and Kubernetes

Nuclio

nuclio is portable across IoT devices, laptops, on-premises datacenters and cloud deployments, eliminating cloud lock-ins and enabling hybrid solutions.

Apache OpenWhisk

OpenWhisk is an open source serverless platform. It is enterprise grade and accessible to all developers thanks to its superior programming model and tooling. It powers IBM Cloud Functions, Adobe I/O Runtime, Naver, Nimbella among others.

Cloud Functions for Firebase

Cloud Functions for Firebase lets you create functions that are triggered by Firebase products, such as changes to data in the Realtime Database, uploads to Cloud Storage, new user sign ups via Authentication, and conversion events in Analytics.

Fission

Write short-lived functions in any language, and map them to HTTP requests (or other event triggers). Deploy functions instantly with one command. There are no containers to build, and no Docker registries to manage.

Related Comparisons

AWS Batch vs AWS Lambda: What are the differences?

Scaling and Control: AWS Batch provides fine-grained control over the scaling and management of your batch computing workloads. It allows you to set up and manage queues, define job dependencies, and control batch scheduling. On the other hand, AWS Lambda automatically scales your application to handle incoming requests, without requiring manual management or configuration.
Pricing Model: AWS Batch follows a pay-as-you-go pricing model, where you are charged based on the number of vCPUs and memory resources used by your batch compute environments. In contrast, AWS Lambda follows a metered pricing model, where you are billed based on the number of requests and the duration of each request invocation.
Compute Environment: AWS Batch allows you to define and manage your own compute environments, where you have control over the type and capacity of the underlying EC2 instances used for batch processing. On the other hand, AWS Lambda abstracts the compute environment and automatically manages the infrastructure, allowing you to focus on writing code without worrying about the underlying resources.
Execution Limits: AWS Batch allows you to run long-running processes, where each job can run for hours or even days if needed. This is particularly useful for scientific and batch processing workloads. On the other hand, AWS Lambda has a maximum execution duration limit of five minutes, which makes it most suitable for short-lived tasks or event-driven applications.
Integration with Services: AWS Batch can be easily integrated with other AWS services such as Amazon S3, Amazon DynamoDB, and AWS Step Functions. This allows you to build complex workflows and process large amounts of data using batch processing. AWS Lambda also has integrations with various AWS services, but its main strength lies in its ability to respond to events and trigger serverless functions in response to actions from other services.
Dependencies and Language Support: AWS Batch supports running applications written in any programming language as long as it can be containerized. This gives you the flexibility to use your preferred programming language. On the other hand, AWS Lambda supports a wider range of programming languages out of the box, including Node.js, Java, Python, Ruby, C#, PowerShell, and Go. It also allows you to include external libraries and dependencies in your function code.

In summary, AWS Batch provides fine-grained control over batch processing workloads, while AWS Lambda offers automatic scaling and event-based execution for serverless functions.

AWS Batch vs AWS Lambda

Overview