Kafka vs TiDB

Overview

Kafka

Stacks24.2K

Followers22.3K

Votes607

GitHub Stars31.2K

Forks14.8K

TiDB

Stacks76

Followers177

Votes28

GitHub Stars39.3K

Forks6.0K

Kafka vs TiDB: What are the differences?

Kafka: Distributed, fault tolerant, high throughput pub-sub messaging system. Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design; TiDB: A distributed NewSQL database compatible with MySQL protocol. Inspired by the design of Google F1, TiDB supports the best features of both traditional RDBMS and NoSQL.

Kafka belongs to "Message Queue" category of the tech stack, while TiDB can be primarily classified under "Databases".

Some of the features offered by Kafka are:

Written at LinkedIn in Scala
Used by LinkedIn to offload processing of all page and other views
Defaults to using persistence, uses OS disk cache for hot data (has higher throughput then any of the above having persistence enabled)

On the other hand, TiDB provides the following key features:

Horizontal scalability
Asynchronous schema changes
Consistent distributed transactions

Kafka and TiDB are both open source tools. TiDB with 19.6K GitHub stars and 2.85K forks on GitHub appears to be more popular than Kafka with 12.7K GitHub stars and 6.81K GitHub forks.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on Kafka, TiDB

viradiya

Apr 12, 2020

Needs adviceon

AngularJS

ASP.NET Core

MSSQL

We are going to develop a microservices-based application. It consists of AngularJS, ASP.NET Core, and MSSQL.

We have 3 types of microservices. Emailservice, Filemanagementservice, Filevalidationservice

I am a beginner in microservices. But I have read about RabbitMQ, but come to know that there are Redis and Kafka also in the market. So, I want to know which is best.

933k views933k

Comments

Ishfaq

Feb 28, 2020

Needs advice

Our backend application is sending some external messages to a third party application at the end of each backend (CRUD) API call (from UI) and these external messages take too much extra time (message building, processing, then sent to the third party and log success/failure), UI application has no concern to these extra third party messages.

So currently we are sending these third party messages by creating a new child thread at end of each REST API call so UI application doesn't wait for these extra third party API calls.

I want to integrate Apache Kafka for these extra third party API calls, so I can also retry on failover third party API calls in a queue(currently third party messages are sending from multiple threads at the same time which uses too much processing and resources) and logging, etc.

Question 1: Is this a use case of a message broker?

Question 2: If it is then Kafka vs RabitMQ which is the better?

804k views804k

Comments

Roman

Senior Back-End Developer, Software Architect

Feb 12, 2019

Reviewon

Kafka

I use Kafka because it has almost infinite scaleability in terms of processing events (could be scaled to process hundreds of thousands of events), great monitoring (all sorts of metrics are exposed via JMX).

Downsides of using Kafka are:

you have to deal with Zookeeper
you have to implement advanced routing yourself (compared to RabbitMQ it has no advanced routing)

10.9k views10.9k

Comments

Detailed Comparison

Kafka	TiDB
Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.	Inspired by the design of Google F1, TiDB supports the best features of both traditional RDBMS and NoSQL.
Written at LinkedIn in Scala;Used by LinkedIn to offload processing of all page and other views;Defaults to using persistence, uses OS disk cache for hot data (has higher throughput then any of the above having persistence enabled);Supports both on-line as off-line processing	Horizontal scalability;Asynchronous schema changes;Consistent distributed transactions;Compatible with MySQL protocol;Written in Go;NewSQL over TiKV;Multiple storage engine support
Statistics
GitHub Stars 31.2K	GitHub Stars 39.3K
GitHub Forks 14.8K	GitHub Forks 6.0K
Stacks 24.2K	Stacks 76
Followers 22.3K	Followers 177
Votes 607	Votes 28
Pros & Cons
Pros 126 High-throughput 119 Distributed 92 Scalable 86 High-Performance 66 Durable Cons 32 Non-Java clients are second-class citizens 29 Needs Zookeeper 9 Operational difficulties 5 Terrible Packaging	Pros 9 Open source 7 Horizontal scalability 5 Strong ACID 3 HTAP 2 Mysql Compatibility

What are some alternatives to Kafka, TiDB?

MongoDB

MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding.

MySQL

The MySQL software delivers a very fast, multi-threaded, multi-user, and robust SQL (Structured Query Language) database server. MySQL Server is intended for mission-critical, heavy-load production systems as well as for embedding into mass-deployed software.

PostgreSQL

PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions.

RabbitMQ

RabbitMQ gives your applications a common platform to send and receive messages, and your messages a safe place to live until received.

Microsoft SQL Server

Microsoft® SQL Server is a database management and analysis system for e-commerce, line-of-business, and data warehousing solutions.

SQLite

SQLite is an embedded SQL database engine. Unlike most other SQL databases, SQLite does not have a separate server process. SQLite reads and writes directly to ordinary disk files. A complete SQL database with multiple tables, indices, triggers, and views, is contained in a single disk file.

Cassandra

Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL.

Memcached

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

MariaDB

Started by core members of the original MySQL team, MariaDB actively works with outside developers to deliver the most featureful, stable, and sanely licensed open SQL server in the industry. MariaDB is designed as a drop-in replacement of MySQL(R) with more features, new storage engines, fewer bugs, and better performance.

RethinkDB

RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn.

Related Comparisons

Some of the features offered by Kafka are:

Written at LinkedIn in Scala
Used by LinkedIn to offload processing of all page and other views
Defaults to using persistence, uses OS disk cache for hot data (has higher throughput then any of the above having persistence enabled)

On the other hand, TiDB provides the following key features:

Horizontal scalability
Asynchronous schema changes
Consistent distributed transactions

Kafka and TiDB are both open source tools. TiDB with 19.6K GitHub stars and 2.85K forks on GitHub appears to be more popular than Kafka with 12.7K GitHub stars and 6.81K GitHub forks.

Kafka vs TiDB