All changes to our resources are published to RabbitMQ, and is the core of our evented architecture. RabbitMQ
Automations are what makes a CRM powerful. With Celery and RabbitMQ we've been able to make powerful automations that truly works for our clients. Such as for example, automatic daily reports, reminders for their activities, important notifications regarding their client activities and actions on the website and more.
We use Celery basically for everything that needs to be scheduled for the future, and using RabbitMQ as our Queue-broker is amazing since it fully integrates with Django and Celery storing on our database results of the tasks done so we can see if anything fails immediately.
We make extensive use of Redis for our caches and use it as a way to save "semi-permanent" stuff like user-submit settings (that get refreshed on each login) or cooldowns that expire very fast. Additionally we also utilize the Pub-Sub capabilities that Redis has to offer.
We decided against using a dedicated Message-Broker/Streaming Platform like RabbitMQ or Kafka, as we already had a packet-based, custom protocol for communication between servers and services, and we only needed some "tiny" Pub-Sub magic to fill in the gaps. An entire additional service just for this oddjob would've been a total overkill.
Which is the best Python framework for microservices?
We are using Nameko for building microservices in Python. The things we really like are dependency injection and the ease with which one can expose endpoints via RPC over RabbitMQ. We are planning to try a tool that helps us write polyglot microservices and nameko is not super compatible with it. Also, we are a bit worried about the not so good community support from nameko and looking for a python alternate to write microservices.
I really like Nameko for it's ease of use, but haven't used it for a while - the GRPC bridge looks pretty cool (https://github.com/nameko/nameko-grpc).
If you're looking for something compact, popular and with a burgeoning community (although without much in the way of gRPC support) I'd recommend checking out Tiangolo's FastAPI
Hey there! We are looking at Datadog, Dynatrace, AppDynamics, and New Relic as options for our web application monitoring.
Current Environment: .NET Core Web app hosted on Microsoft IIS
Future Environment: Web app will be hosted on Microsoft Azure
Tech Stacks: IIS, RabbitMQ, Redis, Microsoft SQL Server
Requirement: Infra Monitoring, APM, Real - User Monitoring (User activity monitoring i.e., time spent on a page, most active page, etc.), Service Tracing, Root Cause Analysis, and Centralized Log Management.
Please advise on the above. Thanks!
Hello there, We're developing a team chat application which would consist of direct (one-to-one) conversations and channel (group) conversations. I'm not the developer (of course), but my team suggested to go with Redis.
I've seen tech stacks of BIG team chat applications like Slack and Flock...but they haven't used RabbitMQ and used Redis instead.
A quick question, what's a good choice to go with for RabbitMQ or Redis for a message queue system in our case?
This is of course determined by the needs of your application. It is important how many of your estimated instant users in your application will be. Also, the features of the application will affect the architecture of the application. For example, if the message data would be processed on the server, I would prefer a distributed server solution such as akka actor with the rabbitmq cluster. I would definitely use Redis. Both technologies are incomparable lanes. Redis is a database and its purpose is to process data from a different memory with the memory used by the code running on the server. Rabbit is a messaging queue system. It contributes to the architecture in a different dimension. Performance and stability are keywords.
I'm curious to find out why many people decide to compare seemingly disparate technologies such Redis and RabbitMQ? Is there some sort of esoteric ninja trickery behind this?
Each tool supports different use cases. RabbitMQ is a middleware peace supporting message driven reqs like you are trying to accomplish and Redis, on the other hand, allows to store data if performance, cache is important. If we are taking about a message queue system approach you could use RabbitMQ, Amazon SNS/SQS or Apache Kafka
We are going to develop a microservices-based application. It consists of AngularJS, ASP.NET Core, and MSSQL.
We have 3 types of microservices. Emailservice, Filemanagementservice, Filevalidationservice
I am a beginner in microservices. But I have read about RabbitMQ, but come to know that there are Redis and Kafka also in the market. So, I want to know which is best.
Kafka is an Enterprise Messaging Framework whereas Redis is an Enterprise Cache Broker, in-memory database and high performance database.Both are having their own advantages, but they are different in usage and implementation. Now if you are creating microservices check the user consumption volumes, its generating logs, scalability, systems to be integrated and so on. I feel for your scenario initially you can go with KAFKA bu as the throughput, consumption and other factors are scaling then gradually you can add Redis accordingly.
I first recommend that you choose Angular over AngularJS if you are starting something new. AngularJs is no longer getting enhancements, but perhaps you meant Angular. Regarding microservices, I recommend considering microservices when you have different development teams for each service that may want to use different programming languages and backend data stores. If it is all the same team, same code language, and same data store I would not use microservices. I might use a message queue, in which case RabbitMQ is a good one. But you may also be able to simply write your own in which you write a record in a table in MSSQL and one of your services reads the record from the table and processes it. The most challenging part of doing it yourself is writing a service that does a good job of reading the queue without reading the same message multiple times or missing a message; and that is where RabbitMQ can help.
Hi, I am building an enhanced web-conferencing app that will have a voice/video call, live chats, live notifications, live discussions, screen sharing, etc features. Ref: Zoom.
I need advise finalizing the tech stack for this app. I am considering below tech stack:
- Frontend: React
- Backend: Node.js
- Database: MongoDB
- IAAS: #AWS
- Containers & Orchestration: Docker / Kubernetes
- DevOps: GitLab, Terraform
- Brokers: Redis / RabbitMQ
I need advice at the platform level as to what could be considered to support concurrent video streaming seamlessly.
Also, please suggest what could be a better tech stack for my app?
#SAAS #VideoConferencing #WebAndVideoConferencing #zoom #stack
You're going to want to look hard at WebRTC. It's what almost every realtime video service uses. The appeal is that it establishes a direct connection between peers so that the massive video bandwidth doesn't need to go through your backend. That aside, actor clusters will be the other technology that handle that sort of traffic well. It was popularized by erlang for telecom backbone, akka is another choice for actor systems.
Infrastructure wise, kubernetes would be a fine choice. Just make sure to look up some benchmarks for Container Network Interface (CNI) implementations that support high bandwidth traffic.
Kubernetes provides Auto-scaling whereas Docker Swarm doesn't support autoscaling. Kubernetes supports up to 5000 nodes whereas Docker Swarm supports more than 2000 nodes. Kubernetes is less extensive and customizable whereas Docker Swarm is more comprehensive and highly customizable. So if your main usecase is autoscaling go for kubernetes else Docker is always a good choice.
Kafka was only introduced to our platform in August 2018 as a means to manage our data pipeline and to replace other messaging systems used to decouple various components in our system. Kafka provides the scale and storage we need to manage data for however many devices we might service. Additionally, Kafka has helped us lay the framework for improved and highly detailed statistics gathering and analysis.
The question for which Message Queue to use mentioned "availability, distributed, scalability, and monitoring". I don't think that this excludes many options already. I does not sound like you would take advantage of Kafka's strengths (replayability, based on an even sourcing architecture). You could pick one of the AMQP options.
I would recommend the RabbitMQ message broker, which not only implements the AMQP standard 0.9.1 (it can support 1.x or other protocols as well) but has also several very useful extensions built in. It ticks the boxes you mentioned and on top you will get a very flexible system, that allows you to build the architecture, pick the options and trade-offs that suite your case best.
For more information about RabbitMQ, please have a look at the linked markdown I assembled. The second half explains many configuration options. It also contains links to managed hosting and to libraries (though it is missing Python's - which should be Puka, I assume).