Kafka vs PubNub

Overview

PubNub

Stacks249

Followers458

Votes238

Kafka

Stacks24.2K

Followers22.3K

Votes607

GitHub Stars31.2K

Forks14.8K

Kafka vs PubNub: What are the differences?

Introduction

Kafka and PubNub are two popular platforms used in real-time data streaming and messaging. While both serve a similar purpose, there are key differences between these two platforms that make them suitable for different use cases.

Scalability:
- Kafka: Kafka is designed to handle high-throughput, distributed messaging scenarios. It provides high scalability and can handle large volumes of data from multiple sources.
- PubNub: PubNub is a real-time messaging platform that specializes in delivering messages instantly to a large number of devices. It offers global scalability with its distributed network of data centers.
Message Persistence:
- Kafka: Kafka stores messages in a distributed commit log, which allows for fault-tolerant message persistence. Messages can be stored for longer periods and can be replayed if needed.
- PubNub: PubNub does not provide built-in message persistence. Messages are delivered in real-time and are not stored for long-term retrieval.
Message Delivery Guarantees:
- Kafka: Kafka offers strong message delivery guarantees with at-least-once semantics. It ensures that messages are not lost and are delivered in the order they were produced.
- PubNub: PubNub provides different levels of delivery guarantees, including at-most-once, at-least-once, and exactly-once. This allows developers to choose the level of reliability required for their application.
Use Cases:
- Kafka: Kafka is commonly used for building real-time streaming data pipelines, event sourcing, and building fault-tolerant messaging systems.
- PubNub: PubNub is often used for real-time messaging in chat applications, Internet of Things (IoT) scenarios, and multiplayer gaming.
Integration Capabilities:
- Kafka: Kafka integrates well with a wide range of systems and frameworks, including Apache Spark, Apache Flink, and Apache Storm. It provides connectors and APIs for easy integration.
- PubNub: PubNub offers SDKs and APIs for integration with various platforms, including JavaScript, iOS, Android, and backend services like AWS Lambda and Google Cloud Functions.
Ease of Use and Management:
- Kafka: Kafka requires more setup and configuration compared to PubNub. It is more suitable for scenarios where fine-grained control over the messaging infrastructure is required.
- PubNub: PubNub is designed to be easy to use and requires minimal setup. It simplifies the messaging infrastructure, making it suitable for applications where ease of use and quick deployment are priorities.

In summary, Kafka is suitable for high-throughput, distributed messaging scenarios with strong message delivery guarantees and extensive integration capabilities. PubNub, on the other hand, is ideal for real-time messaging applications that require global scalability, quick deployment, and ease of use.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on PubNub, Kafka

viradiya

Apr 12, 2020

Needs adviceon

AngularJS

ASP.NET Core

MSSQL

We are going to develop a microservices-based application. It consists of AngularJS, ASP.NET Core, and MSSQL.

We have 3 types of microservices. Emailservice, Filemanagementservice, Filevalidationservice

I am a beginner in microservices. But I have read about RabbitMQ, but come to know that there are Redis and Kafka also in the market. So, I want to know which is best.

933k views933k

Comments

Ishfaq

Feb 28, 2020

Needs advice

Our backend application is sending some external messages to a third party application at the end of each backend (CRUD) API call (from UI) and these external messages take too much extra time (message building, processing, then sent to the third party and log success/failure), UI application has no concern to these extra third party messages.

So currently we are sending these third party messages by creating a new child thread at end of each REST API call so UI application doesn't wait for these extra third party API calls.

I want to integrate Apache Kafka for these extra third party API calls, so I can also retry on failover third party API calls in a queue(currently third party messages are sending from multiple threads at the same time which uses too much processing and resources) and logging, etc.

Question 1: Is this a use case of a message broker?

Question 2: If it is then Kafka vs RabitMQ which is the better?

804k views804k

Comments

Roman

Senior Back-End Developer, Software Architect

Feb 12, 2019

Reviewon

Kafka

I use Kafka because it has almost infinite scaleability in terms of processing events (could be scaled to process hundreds of thousands of events), great monitoring (all sorts of metrics are exposed via JMX).

Downsides of using Kafka are:

you have to deal with Zookeeper
you have to implement advanced routing yourself (compared to RabbitMQ it has no advanced routing)

10.9k views10.9k

Comments

Detailed Comparison

PubNub	Kafka
PubNub makes it easy for you to add real-time capabilities to your apps, without worrying about the infrastructure. Build apps that allow your users to engage in real-time across mobile, browser, desktop and server.	Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
PubNub SDKs support over 50 of the most popular environments, including: iOS, Android, JavaScript, .NET, Java, Ruby, Python, PHP and many more.;Data Push - Establish and maintain persistent socket connections to any device (mobile, browser, desktop and server) and push data to global audiences in less than ¼ of a second;Presence - Automatically detect when users enter or leave your app and whether machines are online;Storage and Playback- Store your real-time data streams for future access, “playback” or re-publish;Mobile - Easily manage the complexity of real-time apps on mobile devices;Analytics - Simulate an audience of millions. See a live view of audience size, engagement and geography;Security - Protect your real-time data with easy to use enterprise-grade security	Written at LinkedIn in Scala;Used by LinkedIn to offload processing of all page and other views;Defaults to using persistence, uses OS disk cache for hot data (has higher throughput then any of the above having persistence enabled);Supports both on-line as off-line processing
Statistics
GitHub Stars -	GitHub Stars 31.2K
GitHub Forks -	GitHub Forks 14.8K
Stacks 249	Stacks 24.2K
Followers 458	Followers 22.3K
Votes 238	Votes 607
Pros & Cons
Pros 36 Massively scalable & easy to use 25 Easy setup 20 Reliable 19 Great support 14 Flexible to integrate to custom applications Cons 1 Costly	Pros 126 High-throughput 119 Distributed 92 Scalable 86 High-Performance 66 Durable Cons 32 Non-Java clients are second-class citizens 29 Needs Zookeeper 9 Operational difficulties 5 Terrible Packaging

What are some alternatives to PubNub, Kafka?

Firebase

Firebase is a cloud service designed to power real-time, collaborative applications. Simply add the Firebase library to your application to gain access to a shared data structure; any changes you make to that data are automatically synchronized with the Firebase cloud and with other clients within milliseconds.

Socket.IO

It enables real-time bidirectional event-based communication. It works on every platform, browser or device, focusing equally on reliability and speed.

RabbitMQ

RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.

Celery

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

Pusher

Pusher is the category leader in delightful APIs for app developers building communication and collaboration features.

Amazon SQS

Transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be always available. With SQS, you can offload the administrative burden of operating and scaling a highly available messaging cluster, while paying a low price for only what you use.

NSQ

NSQ is a realtime distributed messaging platform designed to operate at scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. See features & guarantees.

SignalR

SignalR allows bi-directional communication between server and client. Servers can now push content to connected clients instantly as it becomes available. SignalR supports Web Sockets, and falls back to other compatible techniques for older browsers. SignalR includes APIs for connection management (for instance, connect and disconnect events), grouping connections, and authorization.

Ably

Ably offers WebSockets, stream resume, history, presence, and managed third-party integrations to make it simple to build, extend, and deliver digital realtime experiences at scale.

Syncano

Syncano is a backend platform to build powerful real-time apps more efficiently. Integrate with any API, minimize boilerplate code and control your data - all from one place.

Related Comparisons

Kafka vs PubNub: What are the differences?

Introduction

Scalability:
- Kafka: Kafka is designed to handle high-throughput, distributed messaging scenarios. It provides high scalability and can handle large volumes of data from multiple sources.
- PubNub: PubNub is a real-time messaging platform that specializes in delivering messages instantly to a large number of devices. It offers global scalability with its distributed network of data centers.
Message Persistence:
- Kafka: Kafka stores messages in a distributed commit log, which allows for fault-tolerant message persistence. Messages can be stored for longer periods and can be replayed if needed.
- PubNub: PubNub does not provide built-in message persistence. Messages are delivered in real-time and are not stored for long-term retrieval.
Message Delivery Guarantees:
- Kafka: Kafka offers strong message delivery guarantees with at-least-once semantics. It ensures that messages are not lost and are delivered in the order they were produced.
- PubNub: PubNub provides different levels of delivery guarantees, including at-most-once, at-least-once, and exactly-once. This allows developers to choose the level of reliability required for their application.
Use Cases:
- Kafka: Kafka is commonly used for building real-time streaming data pipelines, event sourcing, and building fault-tolerant messaging systems.
- PubNub: PubNub is often used for real-time messaging in chat applications, Internet of Things (IoT) scenarios, and multiplayer gaming.
Integration Capabilities:
- Kafka: Kafka integrates well with a wide range of systems and frameworks, including Apache Spark, Apache Flink, and Apache Storm. It provides connectors and APIs for easy integration.
- PubNub: PubNub offers SDKs and APIs for integration with various platforms, including JavaScript, iOS, Android, and backend services like AWS Lambda and Google Cloud Functions.
Ease of Use and Management:
- Kafka: Kafka requires more setup and configuration compared to PubNub. It is more suitable for scenarios where fine-grained control over the messaging infrastructure is required.
- PubNub: PubNub is designed to be easy to use and requires minimal setup. It simplifies the messaging infrastructure, making it suitable for applications where ease of use and quick deployment are priorities.

Kafka vs PubNub

Overview