Elasticsearch vs Pilosa

Overview

Elasticsearch

Stacks35.5K

Followers27.1K

Votes1.6K

Pilosa

Stacks1

Followers10

Votes0

Elasticsearch vs Pilosa: What are the differences?

Data Model: Elasticsearch uses a document-oriented data model where data is stored in JSON format, while Pilosa is a bitmap-indexing data model that stores data as binary values. This difference in data models impacts how data is indexed and queried in each system.
Querying: Elasticsearch provides complex full-text search capabilities, aggregations, and filtering through its query DSL (Domain Specific Language). Pilosa focuses on set-based operations like unions, intersections, and differences which are efficient for analyzing relationships between data points.
Scalability: Elasticsearch is designed for horizontal scalability, with built-in features like sharding and replication that allow it to handle large quantities of data across multiple nodes. Pilosa, on the other hand, is designed for massive scalability by partitioning and distributing bitmaps across a cluster of nodes.
Indexing Speed: Elasticsearch is known for its fast indexing speed, making it suitable for near real-time data indexing and searching. Pilosa, with its bitmap-based indexing, provides rapid response times for set-based queries but may not be as fast for indexing large volumes of data compared to Elasticsearch.
Use Cases: Elasticsearch is commonly used for full-text search, log analytics, and real-time analytics due to its powerful search capabilities and scalability. Pilosa is typically used for analytical workloads that involve complex set calculations, particularly in scenarios where relationships between data points need to be analyzed efficiently.
Consistency Model: Elasticsearch provides eventual consistency by default, allowing for quick data availability but potential for stale results. Pilosa offers strong consistency where updates are immediately reflected across all nodes in the cluster, ensuring real-time accuracy but potentially impacting latency.

In Summary, Key differences between Elasticsearch and Pilosa lie in their data models, querying capabilities, scalability approaches, indexing speed, preferred use cases, and consistency models.

Share your Stack

Help developers discover the tools you use. Get visibility for your team's tech choices and contribute to the community's knowledge.

View Docs

CLI (Node.js)

Manual

Advice on Elasticsearch, Pilosa

Rana Usman

Chief Technology Officer at TechAvanza

Jun 4, 2020

Needs adviceon

Firebase

Elasticsearch

Algolia

Hey everybody! (1) I am developing an android application. I have data of around 3 million record (less than a TB). I want to save that data in the cloud. Which company provides the best cloud database services that would suit my scenario? It should be secured, long term useable, and provide better services. I decided to use Firebase Realtime database. Should I stick with Firebase or are there any other companies that provide a better service?

(2) I have the functionality of searching data in my app. Same data (less than a TB). Which search solution should I use in this case? I found Elasticsearch and Algolia search. It should be secure and fast. If any other company provides better services than these, please feel free to suggest them.

Thank you!

408k views408k

Comments

Detailed Comparison

Elasticsearch	Pilosa
Elasticsearch is a distributed, RESTful search and analytics engine capable of storing data and searching it in near real time. Elasticsearch, Kibana, Beats and Logstash are the Elastic Stack (sometimes called the ELK Stack).	Pilosa is an open source, distributed bitmap index that dramatically accelerates queries across multiple, massive data sets.
Distributed and Highly Available Search Engine;Multi Tenant with Multi Types;Various set of APIs including RESTful;Clients available in many languages including Java, Python, .NET, C#, Groovy, and more;Document oriented;Reliable, Asynchronous Write Behind for long term persistency;(Near) Real Time Search;Built on top of Apache Lucene;Per operation consistency;Inverted indices with finite state transducers for full-text querying;BKD trees for storing numeric and geo data;Column store for analytics;Compatible with Hadoop using the ES-Hadoop connector;Open Source under Apache 2 and Elastic License	-
Statistics
Stacks 35.5K	Stacks 1
Followers 27.1K	Followers 10
Votes 1.6K	Votes 0
Pros & Cons
Pros 329 Powerful api 315 Great search engine 231 Open source 214 Restful 200 Near real-time search Cons 7 Resource hungry 6 Diffecult to get started 5 Expensive 4 Hard to keep stable at large scale	No community feedback yet
Integrations
Kibana Beats Logstash	Golang Java Python

What are some alternatives to Elasticsearch, Pilosa?

Algolia

Our mission is to make you a search expert. Push data to our API to make it searchable in real time. Build your dream front end with one of our web or mobile UI libraries. Tune relevance and get analytics right from your dashboard.

Apache Spark

Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.

Presto

Distributed SQL Query Engine for Big Data

Amazon Athena

Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.

Typesense

It is an open source, typo tolerant search engine that delivers fast and relevant results out-of-the-box. has been built from scratch to offer a delightful, out-of-the-box search experience. From instant search to autosuggest, to faceted search, it has got you covered.

Apache Flink

Apache Flink is an open source system for fast and versatile data analytics in clusters. Flink supports batch and streaming analytics, in one system. Analytical programs can be written in concise and elegant APIs in Java and Scala.

lakeFS

It is an open-source data version control system for data lakes. It provides a “Git for data” platform enabling you to implement best practices from software engineering on your data lake, including branching and merging, CI/CD, and production-like dev/test environments.

Druid

Druid is a distributed, column-oriented, real-time analytics data store that is commonly used to power exploratory dashboards in multi-tenant environments. Druid excels as a data warehousing solution for fast aggregate queries on petabyte sized data sets. Druid supports a variety of flexible filters, exact calculations, approximate algorithms, and other useful calculations.

Amazon CloudSearch

Amazon CloudSearch enables you to search large collections of data such as web pages, document files, forum posts, or product information. With a few clicks in the AWS Management Console, you can create a search domain, upload the data you want to make searchable to Amazon CloudSearch, and the search service automatically provisions the required technology resources and deploys a highly tuned search index.

Amazon Elasticsearch Service

Amazon Elasticsearch Service is a fully managed service that makes it easy for you to deploy, secure, and operate Elasticsearch at scale with zero down time.

Related Comparisons

Elasticsearch vs Pilosa: What are the differences?

Data Model: Elasticsearch uses a document-oriented data model where data is stored in JSON format, while Pilosa is a bitmap-indexing data model that stores data as binary values. This difference in data models impacts how data is indexed and queried in each system.
Querying: Elasticsearch provides complex full-text search capabilities, aggregations, and filtering through its query DSL (Domain Specific Language). Pilosa focuses on set-based operations like unions, intersections, and differences which are efficient for analyzing relationships between data points.
Scalability: Elasticsearch is designed for horizontal scalability, with built-in features like sharding and replication that allow it to handle large quantities of data across multiple nodes. Pilosa, on the other hand, is designed for massive scalability by partitioning and distributing bitmaps across a cluster of nodes.
Indexing Speed: Elasticsearch is known for its fast indexing speed, making it suitable for near real-time data indexing and searching. Pilosa, with its bitmap-based indexing, provides rapid response times for set-based queries but may not be as fast for indexing large volumes of data compared to Elasticsearch.
Use Cases: Elasticsearch is commonly used for full-text search, log analytics, and real-time analytics due to its powerful search capabilities and scalability. Pilosa is typically used for analytical workloads that involve complex set calculations, particularly in scenarios where relationships between data points need to be analyzed efficiently.
Consistency Model: Elasticsearch provides eventual consistency by default, allowing for quick data availability but potential for stale results. Pilosa offers strong consistency where updates are immediately reflected across all nodes in the cluster, ensuring real-time accuracy but potentially impacting latency.

In Summary, Key differences between Elasticsearch and Pilosa lie in their data models, querying capabilities, scalability approaches, indexing speed, preferred use cases, and consistency models.

Elasticsearch vs Pilosa

Overview