What is Faktory and what are its top alternatives?
Top Alternatives to Faktory
Sidekiq uses threads to handle many jobs at the same time in the same process. It does not require Rails but will integrate tightly with Rails 3/4 to make background processing dead simple. ...
RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received. ...
It is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline. It supports all kind of background tasks – short-running and long-running, CPU intensive and I/O intensive, one shot and recurrent. ...
Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both. ...
Beanstalks's interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously. ...
It is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites. It includes Adaptive process spawning, Advanced process management with graceful stop/start, Emergency restart in case of accidental opcode cache destruction etc. ...
The fastest, most reliable, Redis-based queue for Node. Carefully written for rock solid stability and atomicity. ...
Delayed_job (or DJ) encapsulates the common pattern of asynchronously executing longer tasks in the background. It is a direct extraction from Shopify where the job table is responsible for a multitude of core tasks. ...
Faktory alternatives & related posts
- Efficient background processing99
- Better then resque37
- Great documentation26
- Admin tool15
- Great community14
- Integrates with redis automatically, with zero config8
- Great support7
- Stupidly simple to integrate and run on Rails/Heroku7
- Pro version2
- Dashboard w/live polling1
- Great ecosystem of addons1
related Sidekiq posts
We decided to use AWS Lambda for several serverless tasks such as
- Managing AWS backups
- Processing emails received on Amazon SES and stored to Amazon S3 and notified via Amazon SNS, so as to push a message on our Redis so our Sidekiq Rails workers can process inbound emails
- Pushing some relevant Amazon CloudWatch metrics and alarms to Slack
I'm building a new process management tool. I decided to build with Rails as my backend, using Sidekiq for background jobs. I chose to work with these tools because I've worked with them before and know that they're able to get the job done. They may not be the sexiest tools, but they work and are reliable, which is what I was optimizing for. For data stores, I opted for PostgreSQL and Redis. Because I'm planning on offering dashboards, I wanted a SQL database instead of something like MongoDB that might work early on, but be difficult to use as soon as I want to facilitate aggregate queries.
- It's fast and it works with good metrics/monitoring229
- Ease of configuration79
- I like the admin interface58
- Easy to set-up and start with50
- Intuitive work through python18
- Standard protocols18
- Written primarily in Erlang10
- Simply superb8
- Completeness of messaging patterns6
- Scales to 1 million messages per second3
- Better than most traditional queue based message broker2
- Supports AMQP2
- Inubit Integration1
- Supports MQTT1
- Runs on Open Telecom Platform1
- High performance1
- Clear documentation with different scripting language1
- Great ui1
- Better routing system1
- Delayed messages1
- Too complicated cluster/HA config and management9
- Needs Erlang runtime. Need ops good with Erlang runtime6
- Configuration must be done first, not by your code5
related RabbitMQ posts
As Sentry runs throughout the day, there are about 50 different offline tasks that we execute—anything from “process this event, pretty please” to “send all of these cool people some emails.” There are some that we execute once a day and some that execute thousands per second.
Managing this variety requires a reliably high-throughput message-passing technology. We use Celery's RabbitMQ implementation, and we stumbled upon a great feature called Federation that allows us to partition our task queue across any number of RabbitMQ servers and gives us the confidence that, if any single server gets backlogged, others will pitch in and distribute some of the backlogged tasks to their consumers.
Hi, I am building an enhanced web-conferencing app that will have a voice/video call, live chats, live notifications, live discussions, screen sharing, etc features. Ref: Zoom.
I need advise finalizing the tech stack for this app. I am considering below tech stack:
- Frontend: React
- Backend: Node.js
- Database: MongoDB
- IAAS: #AWS
- Containers & Orchestration: Docker / Kubernetes
- DevOps: GitLab, Terraform
- Brokers: Redis / RabbitMQ
I need advice at the platform level as to what could be considered to support concurrent video streaming seamlessly.
Also, please suggest what could be a better tech stack for my app?
#SAAS #VideoConferencing #WebAndVideoConferencing #zoom #stack
- Integrated UI dashboard6
- In Memory2
related Hangfire posts
- Easy to use on heroku1
related Resque posts
- Does one thing well12
- External admin UI developer friendly3
- Job delay3
- Job prioritization2
- External admin UI2
related Beanstalkd posts
I used Kafka originally because it was mandated as part of the top-level IT requirements at a Fortune 500 client. What I found was that it was orders of magnitude more complex ...and powerful than my daily Beanstalkd , and far more flexible, resilient, and manageable than RabbitMQ.
So for any case where utmost flexibility and resilience are part of the deal, I would use Kafka again. But due to the complexities involved, for any time where this level of scalability is not required, I would probably just use Beanstalkd for its simplicity.
I tend to find RabbitMQ to be in an uncomfortable middle place between these two extremities.
related PHP-FPM posts
- Automatic recovery from process crashes2
- Ease of use1
- Based on Redis1
related Bull posts
- Easy to get started3
- Doesn't require Redis1
related delayed_job posts
delayed_job is a great Rails background job library for new projects, as it only uses what you already have: a relational database. We happily used it during the company’s first two years.
But it started to falter as our web and database transactions significantly grew. Our app interacted with users via SMS texts sent inside background jobs. Because the delayed_job daemon ran every couple seconds, this meant that users often waited several long seconds before getting text replies, which was not acceptable. Moreover, job processing was done inside AWS Elastic Beanstalk web instances, which were already under stress and not meant to handle jobs.
We needed a fast background job system that could process jobs in near real-time and integrate well with AWS. Sidekiq is a fast and popular Ruby background job library, but it does not leverage the Elastic Beanstalk worker architecture, and you have to maintain a Redis instance.
We ended up choosing active-elastic-job, which seamlessly integrates with worker instances and Amazon SQS. SQS is a fast queue and you don’t need to worry about infrastructure or scaling, as AWS handles it for you.
We noticed significant performance gains immediately after making the switch.
We use Sidekiq to process millions of Ruby background jobs a day under normal loads. We sometimes process more than that when running one-off backfill tasks.
With so many jobs, it wouldn't really make sense to use delayed_job, as it would put our main database under unnecessary load, which would make it a bottleneck with most DB queries serving jobs and not end users. I suppose you could create a separate DB just for jobs, but that can be a hassle. Sidekiq uses a separate Redis instance so you don't have this problem. And it is very performant!
I also like that its free version comes "batteries included" with:
- A web monitoring UI that provides some nice stats.
- An API that can come in handy for one-off tasks, like changing the queue of certain already enqueued jobs.
Sidekiq is a pleasure to use. All our engineers love it!