AWS Lambda vs Sidekiq

Overview

Sidekiq

Stacks1.2K

Followers632

Votes408

AWS Lambda

Stacks26.0K

Followers18.8K

Votes432

AWS Lambda vs Sidekiq: What are the differences?

Introduction

In this comparison, we will explore the key differences between AWS Lambda and Sidekiq, two popular technologies used for serverless computing and background job processing respectively.

Language Support: AWS Lambda supports a wide range of programming languages, including Java, Python, Node.js, C#, and more, allowing developers to choose their preferred language for coding functions. On the other hand, Sidekiq is primarily designed for Ruby applications, making it the go-to option for Ruby developers.
Execution Environment: AWS Lambda runs on the AWS cloud infrastructure, meaning developers are relieved from managing servers or infrastructure. It automatically scales to handle incoming requests and provisions resources as needed. In contrast, Sidekiq needs to be set up and managed in the application's infrastructure, requiring additional maintenance and configuration.
Scaling and Cost: AWS Lambda offers auto-scaling, where it manages the concurrency automatically based on the incoming traffic. The pricing is based on the number of invocations, duration of execution, and resource usage. Sidekiq, on the other hand, requires manual scaling by configuring the number of workers and concurrency. Scaling Sidekiq can be costlier as it requires provisioning and managing the infrastructure for handling increased load.
Integration with Ecosystem: AWS Lambda is part of the Amazon Web Services ecosystem, which makes it seamless to integrate with other AWS services such as S3, DynamoDB, and SQS. It also offers native integrations with various AWS monitoring and logging tools. In contrast, Sidekiq integrates well with Ruby frameworks and libraries, making it easier to work with the Ruby ecosystem.
Event-driven vs. Job-based: AWS Lambda follows an event-driven architecture, where functions are triggered by events such as HTTP requests, database updates, or message queues. This makes it suitable for real-time processing and response. Conversely, Sidekiq utilizes a job-based model, where jobs are added to a queue and processed by workers asynchronously. This is advantageous for background processing and handling heavy or long-running tasks.
Deployment and Management: AWS Lambda provides a fully managed service, which means that AWS takes care of infrastructure, deployment, and monitoring. Developers can focus solely on writing functions and leave the operational aspect to AWS. In contrast, Sidekiq requires developers to deploy and manage the application infrastructure themselves, including monitoring, error handling, and scalability.

In summary, AWS Lambda and Sidekiq differ in terms of language support, execution environment, scaling and cost, integration with ecosystem, event-driven vs. job-based architecture, and deployment and management approaches.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on Sidekiq, AWS Lambda

Tim

CTO at Checkly Inc.

Sep 18, 2019

Needs adviceon

Heroku

AWS Lambda

When adding a new feature to Checkly rearchitecting some older piece, I tend to pick Heroku for rolling it out. But not always, because sometimes I pick AWS Lambda . The short story:

Developer Experience trumps everything.
AWS Lambda is cheap. Up to a limit though. This impact not only your wallet.
If you need geographic spread, AWS is lonely at the top.

The setup

Recently, I was doing a brainstorm at a startup here in Berlin on the future of their infrastructure. They were ready to move on from their initial, almost 100% Ec2 + Chef based setup. Everything was on the table. But we crossed out a lot quite quickly:

Pure, uncut, self hosted Kubernetes — way too much complexity
Managed Kubernetes in various flavors — still too much complexity
Zeit — Maybe, but no Docker support
Elastic Beanstalk — Maybe, bit old but does the job
Heroku
Lambda

It became clear a mix of PaaS and FaaS was the way to go. What a surprise! That is exactly what I use for Checkly! But when do you pick which model?

I chopped that question up into the following categories:

Developer Experience / DX 🤓
Ops Experience / OX 🐂 (?)
Cost 💵
Lock in 🔐

Read the full post linked below for all details

357k views357k

Comments

Detailed Comparison

Sidekiq	AWS Lambda
Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple.	AWS Lambda is a compute service that runs your code in response to events and automatically manages the underlying compute resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own back-end services that operate at AWS scale, performance, and security.
-	Extend other AWS services with custom logic;Build custom back-end services;Completely Automated Administration;Built-in Fault Tolerance;Automatic Scaling;Integrated Security Model;Bring Your Own Code;Pay Per Use;Flexible Resource Model
Statistics
Stacks 1.2K	Stacks 26.0K
Followers 632	Followers 18.8K
Votes 408	Votes 432
Pros & Cons
Pros 124 Simple 99 Efficient background processing 60 Scalability 37 Better then resque 26 Great documentation	Pros 129 No infrastructure 83 Cheap 70 Quick 59 Stateless 47 No deploy, no server, great sleep Cons 7 Cant execute ruby or go 3 Compute time limited 1 Can't execute PHP w/o significant effort

What are some alternatives to Sidekiq, AWS Lambda?

Beanstalkd

Beanstalks's interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

Azure Functions

Azure Functions is an event driven, compute-on-demand experience that extends the existing Azure application platform with capabilities to implement code triggered by events occurring in virtually any Azure or 3rd party service as well as on-premises systems.

Google Cloud Run

A managed compute platform that enables you to run stateless containers that are invocable via HTTP requests. It's serverless by abstracting away all infrastructure management.

Serverless

Build applications comprised of microservices that run in response to events, auto-scale for you, and only charge you when they run. This lowers the total cost of maintaining your apps, enabling you to build more logic, faster. The Framework uses new event-driven compute services, like AWS Lambda, Google CloudFunctions, and more.

Google Cloud Functions

Construct applications from bite-sized business logic billed to the nearest 100 milliseconds, only while your code is running

Knative

Knative provides a set of middleware components that are essential to build modern, source-centric, and container-based applications that can run anywhere: on premises, in the cloud, or even in a third-party data center

Hangfire

It is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline. It supports all kind of background tasks – short-running and long-running, CPU intensive and I/O intensive, one shot and recurrent.

OpenFaaS

Serverless Functions Made Simple for Docker and Kubernetes

Nuclio

nuclio is portable across IoT devices, laptops, on-premises datacenters and cloud deployments, eliminating cloud lock-ins and enabling hybrid solutions.

Resque

Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both.

Related Comparisons

AWS Lambda vs Sidekiq: What are the differences?

Introduction

In this comparison, we will explore the key differences between AWS Lambda and Sidekiq, two popular technologies used for serverless computing and background job processing respectively.

Language Support: AWS Lambda supports a wide range of programming languages, including Java, Python, Node.js, C#, and more, allowing developers to choose their preferred language for coding functions. On the other hand, Sidekiq is primarily designed for Ruby applications, making it the go-to option for Ruby developers.
Execution Environment: AWS Lambda runs on the AWS cloud infrastructure, meaning developers are relieved from managing servers or infrastructure. It automatically scales to handle incoming requests and provisions resources as needed. In contrast, Sidekiq needs to be set up and managed in the application's infrastructure, requiring additional maintenance and configuration.
Scaling and Cost: AWS Lambda offers auto-scaling, where it manages the concurrency automatically based on the incoming traffic. The pricing is based on the number of invocations, duration of execution, and resource usage. Sidekiq, on the other hand, requires manual scaling by configuring the number of workers and concurrency. Scaling Sidekiq can be costlier as it requires provisioning and managing the infrastructure for handling increased load.
Integration with Ecosystem: AWS Lambda is part of the Amazon Web Services ecosystem, which makes it seamless to integrate with other AWS services such as S3, DynamoDB, and SQS. It also offers native integrations with various AWS monitoring and logging tools. In contrast, Sidekiq integrates well with Ruby frameworks and libraries, making it easier to work with the Ruby ecosystem.
Event-driven vs. Job-based: AWS Lambda follows an event-driven architecture, where functions are triggered by events such as HTTP requests, database updates, or message queues. This makes it suitable for real-time processing and response. Conversely, Sidekiq utilizes a job-based model, where jobs are added to a queue and processed by workers asynchronously. This is advantageous for background processing and handling heavy or long-running tasks.
Deployment and Management: AWS Lambda provides a fully managed service, which means that AWS takes care of infrastructure, deployment, and monitoring. Developers can focus solely on writing functions and leave the operational aspect to AWS. In contrast, Sidekiq requires developers to deploy and manage the application infrastructure themselves, including monitoring, error handling, and scalability.

AWS Lambda vs Sidekiq

Overview

AWS Lambda vs Sidekiq: What are the differences?