Alternatives to Couchbase logo

Alternatives to Couchbase

MongoDB, CouchDB, Cassandra, Redis, and HBase are the most popular alternatives and competitors to Couchbase.
390
493
+ 1
102

What is Couchbase and what are its top alternatives?

Developed as an alternative to traditionally inflexible SQL databases, the Couchbase NoSQL database is built on an open source foundation and architected to help developers solve real-world problems and meet high scalability demands.
Couchbase is a tool in the Databases category of a tech stack.

Top Alternatives to Couchbase

  • MongoDB

    MongoDB

    MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding. ...

  • CouchDB

    CouchDB

    Apache CouchDB is a database that uses JSON for documents, JavaScript for MapReduce indexes, and regular HTTP for its API. CouchDB is a database that completely embraces the web. Store your data with JSON documents. Access your documents and query your indexes with your web browser, via HTTP. Index, combine, and transform your documents with JavaScript. ...

  • Cassandra

    Cassandra

    Partitioning means that Cassandra can distribute your data across multiple machines in an application-transparent matter. Cassandra will automatically repartition as machines are added and removed from the cluster. Row store means that like relational databases, Cassandra organizes data by rows and columns. The Cassandra Query Language (CQL) is a close relative of SQL. ...

  • Redis

    Redis

    Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. ...

  • HBase

    HBase

    Apache HBase is an open-source, distributed, versioned, column-oriented store modeled after Google' Bigtable: A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Apache Hadoop. ...

  • Oracle

    Oracle

    Oracle Database is an RDBMS. An RDBMS that implements object-oriented features such as user-defined types, inheritance, and polymorphism is called an object-relational database management system (ORDBMS). Oracle Database has extended the relational model to an object-relational model, making it possible to store complex business models in a relational database. ...

  • Firebase

    Firebase

    Firebase is a cloud service designed to power real-time, collaborative applications. Simply add the Firebase library to your application to gain access to a shared data structure; any changes you make to that data are automatically synchronized with the Firebase cloud and with other clients within milliseconds. ...

  • Elasticsearch

    Elasticsearch

    Elasticsearch is a distributed, RESTful search and analytics engine capable of storing data and searching it in near real time. Elasticsearch, Kibana, Beats and Logstash are the Elastic Stack (sometimes called the ELK Stack). ...

Couchbase alternatives & related posts

MongoDB logo

MongoDB

64.7K
54K
4.1K
The database for giant ideas
64.7K
54K
+ 1
4.1K
PROS OF MONGODB
  • 824
    Document-oriented storage
  • 591
    No sql
  • 546
    Ease of use
  • 465
    Fast
  • 406
    High performance
  • 256
    Free
  • 215
    Open source
  • 179
    Flexible
  • 142
    Replication & high availability
  • 109
    Easy to maintain
  • 41
    Querying
  • 37
    Easy scalability
  • 36
    Auto-sharding
  • 35
    High availability
  • 31
    Map/reduce
  • 26
    Document database
  • 24
    Easy setup
  • 24
    Full index support
  • 15
    Reliable
  • 14
    Fast in-place updates
  • 13
    Agile programming, flexible, fast
  • 11
    No database migrations
  • 7
    Easy integration with Node.Js
  • 7
    Enterprise
  • 5
    Enterprise Support
  • 4
    Great NoSQL DB
  • 3
    Aggregation Framework
  • 3
    Support for many languages through different drivers
  • 3
    Drivers support is good
  • 2
    Schemaless
  • 2
    Easy to Scale
  • 2
    Fast
  • 2
    Awesome
  • 2
    Managed service
  • 1
    Consistent
CONS OF MONGODB
  • 5
    Very slowly for connected models that require joins
  • 3
    Not acid compliant
  • 1
    Proprietary query language

related MongoDB posts

Jeyabalaji Subramanian

Recently we were looking at a few robust and cost-effective ways of replicating the data that resides in our production MongoDB to a PostgreSQL database for data warehousing and business intelligence.

We set ourselves the following criteria for the optimal tool that would do this job: - The data replication must be near real-time, yet it should NOT impact the production database - The data replication must be horizontally scalable (based on the load), asynchronous & crash-resilient

Based on the above criteria, we selected the following tools to perform the end to end data replication:

We chose MongoDB Stitch for picking up the changes in the source database. It is the serverless platform from MongoDB. One of the services offered by MongoDB Stitch is Stitch Triggers. Using stitch triggers, you can execute a serverless function (in Node.js) in real time in response to changes in the database. When there are a lot of database changes, Stitch automatically "feeds forward" these changes through an asynchronous queue.

We chose Amazon SQS as the pipe / message backbone for communicating the changes from MongoDB to our own replication service. Interestingly enough, MongoDB stitch offers integration with AWS services.

In the Node.js function, we wrote minimal functionality to communicate the database changes (insert / update / delete / replace) to Amazon SQS.

Next we wrote a minimal micro-service in Python to listen to the message events on SQS, pickup the data payload & mirror the DB changes on to the target Data warehouse. We implemented source data to target data translation by modelling target table structures through SQLAlchemy . We deployed this micro-service as AWS Lambda with Zappa. With Zappa, deploying your services as event-driven & horizontally scalable Lambda service is dumb-easy.

In the end, we got to implement a highly scalable near realtime Change Data Replication service that "works" and deployed to production in a matter of few days!

See more
Robert Zuber

We use MongoDB as our primary #datastore. Mongo's approach to replica sets enables some fantastic patterns for operations like maintenance, backups, and #ETL.

As we pull #microservices from our #monolith, we are taking the opportunity to build them with their own datastores using PostgreSQL. We also use Redis to cache data we’d never store permanently, and to rate-limit our requests to partners’ APIs (like GitHub).

When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We handle any side-effects of S3’s eventual consistency model within our own code. This ensures that we deal with user requests correctly while writes are in process.

See more
CouchDB logo

CouchDB

445
499
139
HTTP + JSON document database with Map Reduce views and peer-based replication
445
499
+ 1
139
PROS OF COUCHDB
  • 43
    JSON
  • 30
    Open source
  • 18
    Highly available
  • 12
    Partition tolerant
  • 11
    Eventual consistency
  • 7
    Sync
  • 5
    REST API
  • 4
    Attachments mechanism to docs
  • 4
    Multi master replication
  • 3
    Changes feed
  • 1
    REST interface
  • 1
    js- and erlang-views
CONS OF COUCHDB
    Be the first to leave a con

    related CouchDB posts

    Jonathan Pugh
    Software Engineer / Project Manager / Technical Architect · | 25 upvotes · 1.6M views

    I needed to choose a full stack of tools for cross platform mobile application design & development. After much research and trying different tools, these are what I came up with that work for me today:

    For the client coding I chose Framework7 because of its performance, easy learning curve, and very well designed, beautiful UI widgets. I think it's perfect for solo development or small teams. I didn't like React Native. It felt heavy to me and rigid. Framework7 allows the use of #CSS3, which I think is the best technology to come out of the #WWW movement. No other tech has been able to allow designers and developers to develop such flexible, high performance, customisable user interface elements that are highly responsive and hardware accelerated before. Now #CSS3 includes variables and flexboxes it is truly a powerful language and there is no longer a need for preprocessors such as #SCSS / #Sass / #less. React Native contains a very limited interpretation of #CSS3 which I found very frustrating after using #CSS3 for some years already and knowing its powerful features. The other very nice feature of Framework7 is that you can even build for the browser if you want your app to be available for desktop web browsers. The latest release also includes the ability to build for #Electron so you can have MacOS, Windows and Linux desktop apps. This is not possible with React Native yet.

    Framework7 runs on top of Apache Cordova. Cordova and webviews have been slated as being slow in the past. Having a game developer background I found the tweeks to make it run as smooth as silk. One of those tweeks is to use WKWebView. Another important one was using srcset on images.

    I use #Template7 for the for the templating system which is a no-nonsense mobile-centric #HandleBars style extensible templating system. It's easy to write custom helpers for, is fast and has a small footprint. I'm not forced into a new paradigm or learning some new syntax. It operates with standard JavaScript, HTML5 and CSS 3. It's written by the developer of Framework7 and so dovetails with it as expected.

    I configured TypeScript to work with the latest version of Framework7. I consider TypeScript to be one of the best creations to come out of Microsoft in some time. They must have an amazing team working on it. It's very powerful and flexible. It helps you catch a lot of bugs and also provides code completion in supporting IDEs. So for my IDE I use Visual Studio Code which is a blazingly fast and silky smooth editor that integrates seamlessly with TypeScript for the ultimate type checking setup (both products are produced by Microsoft).

    I use Webpack and Babel to compile the JavaScript. TypeScript can compile to JavaScript directly but Babel offers a few more options and polyfills so you can use the latest (and even prerelease) JavaScript features today and compile to be backwards compatible with virtually any browser. My favorite recent addition is "optional chaining" which greatly simplifies and increases readability of a number of sections of my code dealing with getting and setting data in nested objects.

    I use some Ruby scripts to process images with ImageMagick and pngquant to optimise for size and even auto insert responsive image code into the HTML5. Ruby is the ultimate cross platform scripting language. Even as your scripts become large, Ruby allows you to refactor your code easily and make it Object Oriented if necessary. I find it the quickest and easiest way to maintain certain aspects of my build process.

    For the user interface design and prototyping I use Figma. Figma has an almost identical user interface to #Sketch but has the added advantage of being cross platform (MacOS and Windows). Its real-time collaboration features are outstanding and I use them a often as I work mostly on remote projects. Clients can collaborate in real-time and see changes I make as I make them. The clickable prototyping features in Figma are also very well designed and mean I can send clickable prototypes to clients to try user interface updates as they are made and get immediate feedback. I'm currently also evaluating the latest version of #AdobeXD as an alternative to Figma as it has the very cool auto-animate feature. It doesn't have real-time collaboration yet, but I heard it is proposed for 2019.

    For the UI icons I use Font Awesome Pro. They have the largest selection and best looking icons you can find on the internet with several variations in styles so you can find most of the icons you want for standard projects.

    For the backend I was using the #GraphCool Framework. As I later found out, #GraphQL still has some way to go in order to provide the full power of a mature graph query language so later in my project I ripped out #GraphCool and replaced it with CouchDB and Pouchdb. Primarily so I could provide good offline app support. CouchDB with Pouchdb is very flexible and efficient combination and overcomes some of the restrictions I found in #GraphQL and hence #GraphCool also. The most impressive and important feature of CouchDB is its replication. You can configure it in various ways for backups, fault tolerance, caching or conditional merging of databases. CouchDB and Pouchdb even supports storing, retrieving and serving binary or image data or other mime types. This removes a level of complexity usually present in database implementations where binary or image data is usually referenced through an #HTML5 link. With CouchDB and Pouchdb apps can operate offline and sync later, very efficiently, when the network connection is good.

    I use PhoneGap when testing the app. It auto-reloads your app when its code is changed and you can also install it on Android phones to preview your app instantly. iOS is a bit more tricky cause of Apple's policies so it's not available on the App Store, but you can build it and install it yourself to your device.

    So that's my latest mobile stack. What tools do you use? Have you tried these ones?

    See more
    Gabriel Pa

    We implemented our first large scale EPR application from naologic.com using CouchDB .

    Very fast, replication works great, doesn't consume much RAM, queries are blazing fast but we found a problem: the queries were very hard to write, it took a long time to figure out the API, we had to go and write our own @nodejs library to make it work properly.

    It lost most of its support. Since then, we migrated to Couchbase and the learning curve was steep but all worth it. Memcached indexing out of the box, full text search works great.

    See more
    Cassandra logo

    Cassandra

    3.2K
    3.1K
    492
    A partitioned row store. Rows are organized into tables with a required primary key.
    3.2K
    3.1K
    + 1
    492
    PROS OF CASSANDRA
    • 114
      Distributed
    • 95
      High performance
    • 80
      High availability
    • 74
      Easy scalability
    • 52
      Replication
    • 26
      Multi datacenter deployments
    • 26
      Reliable
    • 8
      OLTP
    • 7
      Open source
    • 7
      Schema optional
    • 2
      Workload separation (via MDC)
    • 1
      Fast
    CONS OF CASSANDRA
    • 2
      Reliability of replication
    • 1
      Updates

    related Cassandra posts

    Thierry Schellenbach
    Shared insights
    on
    RedisRedisCassandraCassandraRocksDBRocksDB
    at

    1.0 of Stream leveraged Cassandra for storing the feed. Cassandra is a common choice for building feeds. Instagram, for instance started, out with Redis but eventually switched to Cassandra to handle their rapid usage growth. Cassandra can handle write heavy workloads very efficiently.

    Cassandra is a great tool that allows you to scale write capacity simply by adding more nodes, though it is also very complex. This complexity made it hard to diagnose performance fluctuations. Even though we had years of experience with running Cassandra, it still felt like a bit of a black box. When building Stream 2.0 we decided to go for a different approach and build Keevo. Keevo is our in-house key-value store built upon RocksDB, gRPC and Raft.

    RocksDB is a highly performant embeddable database library developed and maintained by Facebook’s data engineering team. RocksDB started as a fork of Google’s LevelDB that introduced several performance improvements for SSD. Nowadays RocksDB is a project on its own and is under active development. It is written in C++ and it’s fast. Have a look at how this benchmark handles 7 million QPS. In terms of technology it’s much more simple than Cassandra.

    This translates into reduced maintenance overhead, improved performance and, most importantly, more consistent performance. It’s interesting to note that LinkedIn also uses RocksDB for their feed.

    #InMemoryDatabases #DataStores #Databases

    See more
    Umair Iftikhar
    Technical Architect at Vappar · | 3 upvotes · 139.4K views

    Developing a solution that collects Telemetry Data from different devices, nearly 1000 devices minimum and maximum 12000. Each device is sending 2 packets in 1 second. This is time-series data, and this data definition and different reports are saved on PostgreSQL. Like Building information, maintenance records, etc. I want to know about the best solution. This data is required for Math and ML to run different algorithms. Also, data is raw without definitions and information stored in PostgreSQL. Initially, I went with TimescaleDB due to PostgreSQL support, but to increase in sites, I started facing many issues with timescale DB in terms of flexibility of storing data.

    My major requirement is also the replication of the database for reporting and different purposes. You may also suggest other options other than Druid and Cassandra. But an open source solution is appreciated.

    See more
    Redis logo

    Redis

    43.2K
    32.3K
    3.9K
    An in-memory database that persists on disk
    43.2K
    32.3K
    + 1
    3.9K
    PROS OF REDIS
    • 875
      Performance
    • 535
      Super fast
    • 511
      Ease of use
    • 441
      In-memory cache
    • 321
      Advanced key-value cache
    • 190
      Open source
    • 179
      Easy to deploy
    • 163
      Stable
    • 153
      Free
    • 120
      Fast
    • 40
      High-Performance
    • 39
      High Availability
    • 34
      Data Structures
    • 32
      Very Scalable
    • 23
      Replication
    • 20
      Great community
    • 19
      Pub/Sub
    • 17
      "NoSQL" key-value data store
    • 14
      Hashes
    • 12
      Sets
    • 10
      Sorted Sets
    • 9
      Lists
    • 8
      BSD licensed
    • 8
      NoSQL
    • 7
      Async replication
    • 7
      Integrates super easy with Sidekiq for Rails background
    • 7
      Bitmaps
    • 6
      Open Source
    • 6
      Keys with a limited time-to-live
    • 5
      Strings
    • 5
      Lua scripting
    • 4
      Awesomeness for Free!
    • 4
      Hyperloglogs
    • 3
      outstanding performance
    • 3
      Runs server side LUA
    • 3
      Networked
    • 3
      LRU eviction of keys
    • 3
      Written in ANSI C
    • 3
      Feature Rich
    • 3
      Transactions
    • 2
      Data structure server
    • 2
      Performance & ease of use
    • 1
      Existing Laravel Integration
    • 1
      Automatic failover
    • 1
      Easy to use
    • 1
      Object [key/value] size each 500 MB
    • 1
      Simple
    • 1
      Channels concept
    • 1
      Scalable
    • 1
      Temporarily kept on disk
    • 1
      Dont save data if no subscribers are found
    • 0
      Jk
    CONS OF REDIS
    • 14
      Cannot query objects directly
    • 2
      No secondary indexes for non-numeric data types
    • 1
      No WAL

    related Redis posts

    Robert Zuber

    We use MongoDB as our primary #datastore. Mongo's approach to replica sets enables some fantastic patterns for operations like maintenance, backups, and #ETL.

    As we pull #microservices from our #monolith, we are taking the opportunity to build them with their own datastores using PostgreSQL. We also use Redis to cache data we’d never store permanently, and to rate-limit our requests to partners’ APIs (like GitHub).

    When we’re dealing with large blobs of immutable data (logs, artifacts, and test results), we store them in Amazon S3. We handle any side-effects of S3’s eventual consistency model within our own code. This ensures that we deal with user requests correctly while writes are in process.

    See more

    I'm working as one of the engineering leads in RunaHR. As our platform is a Saas, we thought It'd be good to have an API (We chose Ruby and Rails for this) and a SPA (built with React and Redux ) connected. We started the SPA with Create React App since It's pretty easy to start.

    We use Jest as the testing framework and react-testing-library to test React components. In Rails we make tests using RSpec.

    Our main database is PostgreSQL, but we also use MongoDB to store some type of data. We started to use Redis  for cache and other time sensitive operations.

    We have a couple of extra projects: One is an Employee app built with React Native and the other is an internal back office dashboard built with Next.js for the client and Python in the backend side.

    Since we have different frontend apps we have found useful to have Bit to document visual components and utils in JavaScript.

    See more
    HBase logo

    HBase

    362
    409
    15
    The Hadoop database, a distributed, scalable, big data store
    362
    409
    + 1
    15
    PROS OF HBASE
    • 9
      Performance
    • 5
      OLTP
    • 1
      Fast Point Queries
    CONS OF HBASE
      Be the first to leave a con

      related HBase posts

      Hi, I'm building a machine learning pipelines to store image bytes and image vectors in the backend.

      So, when users query for the random access image data (key), we return the image bytes and perform machine learning model operations on it.

      I'm currently considering going with Amazon S3 (in the future, maybe add Redis caching layer) as the backend system to store the information (s3 buckets with sharded prefixes).

      As the latency of S3 is 100-200ms (get/put) and it has a high throughput of 3500 puts/sec and 5500 gets/sec for a given bucker/prefix. In the future I need to reduce the latency, I can add Redis cache.

      Also, s3 costs are way fewer than HBase (on Amazon EC2 instances with 3x replication factor)

      I have not personally used HBase before, so can someone help me if I'm making the right choice here? I'm not aware of Hbase latencies and I have learned that the MOB feature on Hbase has to be turned on if we have store image bytes on of the column families as the avg image bytes are 240Kb.

      See more
      Oracle logo

      Oracle

      1.6K
      1.3K
      107
      An RDBMS that implements object-oriented features such as user-defined types, inheritance, and polymorphism
      1.6K
      1.3K
      + 1
      107
      PROS OF ORACLE
      • 42
        Reliable
      • 31
        Enterprise
      • 15
        High Availability
      • 5
        Hard to maintain
      • 4
        Expensive
      • 4
        Maintainable
      • 3
        High complexity
      • 3
        Hard to use
      CONS OF ORACLE
      • 13
        Expensive

      related Oracle posts

      Hi. We are planning to develop web, desktop, and mobile app for procurement, logistics, and contracts. Procure to Pay and Source to pay, spend management, supplier management, catalog management. ( similar to SAP Ariba, gap.com, coupa.com, ivalua.com vroozi.com, procurify.com

      We got stuck when deciding which technology stack is good for the future. We look forward to your kind guidance that will help us.

      We want to integrate with multiple databases with seamless bidirectional integration. What APIs and middleware available are best to achieve this? SAP HANA, Oracle, MySQL, MongoDB...

      ASP.NET / Node.js / Laravel. ......?

      Please guide us

      See more
      Firebase logo

      Firebase

      28.2K
      23.8K
      1.9K
      The Realtime App Platform
      28.2K
      23.8K
      + 1
      1.9K
      PROS OF FIREBASE
      • 361
        Realtime backend made easy
      • 264
        Fast and responsive
      • 234
        Easy setup
      • 207
        Real-time
      • 186
        JSON
      • 128
        Free
      • 121
        Backed by google
      • 81
        Angular adaptor
      • 63
        Reliable
      • 36
        Great customer support
      • 26
        Great documentation
      • 23
        Real-time synchronization
      • 20
        Mobile friendly
      • 17
        Rapid prototyping
      • 12
        Great security
      • 11
        Automatic scaling
      • 10
        Freakingly awesome
      • 8
        Chat
      • 8
        Angularfire is an amazing addition!
      • 8
        Super fast development
      • 6
        Awesome next-gen backend
      • 6
        Ios adaptor
      • 5
        Built in user auth/oauth
      • 5
        Firebase hosting
      • 4
        Speed of light
      • 4
        Very easy to use
      • 3
        It's made development super fast
      • 3
        Great
      • 3
        Brilliant for startups
      • 2
        Great all-round functionality
      • 2
        Low battery consumption
      • 2
        I can quickly create static web apps with no backend
      • 2
        The concurrent updates create a great experience
      • 2
        JS Offline and Sync suport
      • 1
        Faster workflow
      • 1
        Large
      • 1
        Serverless
      • 1
        .net
      • 1
        Free SSL
      • 1
        Good Free Limits
      • 1
        Push notification
      • 1
        Easy to use
      • 1
        Easy Reactjs integration
      CONS OF FIREBASE
      • 29
        Can become expensive
      • 15
        No open source, you depend on external company
      • 15
        Scalability is not infinite
      • 9
        Not Flexible Enough
      • 5
        Cant filter queries
      • 3
        Very unstable server
      • 2
        No Relational Data
      • 2
        Too many errors
      • 1
        No offline sync

      related Firebase posts

      Stephen Gheysens
      Senior Solutions Engineer at Twilio · | 14 upvotes · 361.9K views

      Hi Otensia! I'd definitely recommend using the skills you've already got and building with JavaScript is a smart way to go these days. Most platform services have JavaScript/Node SDKs or NPM packages, many serverless platforms support Node in case you need to write any backend logic, and JavaScript is incredibly popular - meaning it will be easy to hire for, should you ever need to.

      My advice would be "don't reinvent the wheel". If you already have a skill set that will work well to solve the problem at hand, and you don't need it for any other projects, don't spend the time jumping into a new language. If you're looking for an excuse to learn something new, it would be better to invest that time in learning a new platform/tool that compliments your knowledge of JavaScript. For this project, I might recommend using Netlify, Vercel, or Google Firebase to quickly and easily deploy your web app. If you need to add user authentication, there are great examples out there for Firebase Authentication, Auth0, or even Magic (a newcomer on the Auth scene, but very user friendly). All of these services work very well with a JavaScript-based application.

      See more
      Tassanai Singprom

      This is my stack in Application & Data

      JavaScript PHP HTML5 jQuery Redis Amazon EC2 Ubuntu Sass Vue.js Firebase Laravel Lumen Amazon RDS GraphQL MariaDB

      My Utilities Tools

      Google Analytics Postman Elasticsearch

      My Devops Tools

      Git GitHub GitLab npm Visual Studio Code Kibana Sentry BrowserStack

      My Business Tools

      Slack

      See more
      Elasticsearch logo

      Elasticsearch

      26.1K
      19.8K
      1.6K
      Open Source, Distributed, RESTful Search Engine
      26.1K
      19.8K
      + 1
      1.6K
      PROS OF ELASTICSEARCH
      • 321
        Powerful api
      • 311
        Great search engine
      • 231
        Open source
      • 213
        Restful
      • 200
        Near real-time search
      • 96
        Free
      • 83
        Search everything
      • 54
        Easy to get started
      • 45
        Analytics
      • 26
        Distributed
      • 6
        Fast search
      • 5
        More than a search engine
      • 3
        Great docs
      • 3
        Awesome, great tool
      • 3
        Easy to scale
      • 2
        Document Store
      • 2
        Nosql DB
      • 2
        Great piece of software
      • 2
        Great customer support
      • 2
        Intuitive API
      • 2
        Fast
      • 2
        Easy setup
      • 2
        Highly Available
      • 1
        Not stable
      • 1
        Scalability
      • 1
        Open
      • 1
        Reliable
      • 1
        Github
      • 1
        Elaticsearch
      • 1
        Actively developing
      • 1
        Responsive maintainers on GitHub
      • 1
        Ecosystem
      • 1
        Easy to get hot data
      • 1
        Potato
      • 0
        Community
      CONS OF ELASTICSEARCH
      • 6
        Resource hungry
      • 6
        Diffecult to get started
      • 5
        Expensive
      • 3
        Hard to keep stable at large scale

      related Elasticsearch posts

      Tim Abbott

      We've been using PostgreSQL since the very early days of Zulip, but we actually didn't use it from the beginning. Zulip started out as a MySQL project back in 2012, because we'd heard it was a good choice for a startup with a wide community. However, we found that even though we were using the Django ORM for most of our database access, we spent a lot of time fighting with MySQL. Issues ranged from bad collation defaults, to bad query plans which required a lot of manual query tweaks.

      We ended up getting so frustrated that we tried out PostgresQL, and the results were fantastic. We didn't have to do any real customization (just some tuning settings for how big a server we had), and all of our most important queries were faster out of the box. As a result, we were able to delete a bunch of custom queries escaping the ORM that we'd written to make the MySQL query planner happy (because postgres just did the right thing automatically).

      And then after that, we've just gotten a ton of value out of postgres. We use its excellent built-in full-text search, which has helped us avoid needing to bring in a tool like Elasticsearch, and we've really enjoyed features like its partial indexes, which saved us a lot of work adding unnecessary extra tables to get good performance for things like our "unread messages" and "starred messages" indexes.

      I can't recommend it highly enough.

      See more
      Tymoteusz Paul
      Devops guy at X20X Development LTD · | 23 upvotes · 4.6M views

      Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).

      It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up or vagrant reload we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.

      I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.

      We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.

      If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.

      The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).

      Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.

      See more