Need advice about which tool to choose?Ask the StackShare community!
Amazon S3 vs Couchbase: What are the differences?
Introduction:
Amazon S3 and Couchbase are two popular data storage solutions used by businesses for different purposes. While Amazon S3 is a cloud-based object storage service provided by Amazon Web Services (AWS), Couchbase is a NoSQL database designed for high-performance applications.
Key Differences between Amazon S3 and Couchbase:
Data Structure: Amazon S3 stores data as objects in a flat structure, where each object is assigned a unique key. On the other hand, Couchbase stores data in a JSON-like document structure, allowing for more flexible and hierarchical data modeling.
Data Access: Amazon S3 primarily provides REST-based APIs for accessing data, making it suitable for static file storage and retrieval. In contrast, Couchbase provides a rich set of APIs (including key-value, document, and query APIs) for efficient and flexible data access, enabling real-time application development.
Scalability: Amazon S3 offers virtually unlimited scale, allowing businesses to store and retrieve massive amounts of data without worrying about capacity constraints. Couchbase also provides horizontal scalability through its distributed architecture, enabling seamless scaling of read and write operations as the data grows.
Data Consistency: Amazon S3 guarantees eventual consistency, meaning that changes made to the data might not be immediately reflected in all replicas. In contrast, Couchbase supports strong consistency, ensuring that updates are immediately consistent across all replicas, making it suitable for use cases with strict consistency requirements.
Data Querying and Indexing: While Amazon S3 focuses primarily on storage, Couchbase offers powerful querying and indexing capabilities. It allows developers to create and optimize flexible indexes on various data attributes, enabling efficient data retrieval and complex query processing.
Data Replication and High Availability: Amazon S3 automatically replicates data across multiple availability zones within a region, providing high availability and durability. Couchbase also offers built-in data replication, allowing businesses to replicate data across multiple Couchbase clusters for better fault tolerance and disaster recovery.
In Summary, Amazon S3 is an object storage service with a flat structure, suitable for static file storage, while Couchbase is a NoSQL database with flexible data modeling, powerful querying capabilities, and strong consistency support. Both offer scalability and data replication features, but Couchbase provides richer APIs and indexing options for real-time application development.
Hello! I have a mobile app with nearly 100k MAU, and I want to add a cloud file storage service to my app.
My app will allow users to store their image, video, and audio files and retrieve them to their device when necessary.
I have already decided to use PHP & Laravel as my backend, and I use Contabo VPS. Now, I need an object storage service for my app, and my options are:
Amazon S3 : It sounds to me like the best option but the most expensive. Closest to my users (MENA Region) for other services, I will have to go to Europe. Not sure how important this is?
DigitalOcean Spaces : Seems like my best option for price/service, but I am still not sure
Wasabi: the best price (6 USD/MONTH/TB) and free bandwidth, but I am not sure if it fits my needs as I want to allow my users to preview audio and video files. They don't recommend their service for streaming videos.
Backblaze B2 Cloud Storage: Good price but not sure about them.
There is also the self-hosted s3 compatible option, but I am not sure about that.
Any thoughts will be helpful. Also, if you think I should post in a different sub, please tell me.
If pricing is the issue i'd suggest you use digital ocean, but if its not use amazon was digital oceans API is s3 compatible
Hello Mohammad, I am using : Cloudways >> AWS >> Bahrain for last 2 years. This is best I consider out of my 10 year research on Laravel hosting.
We Have thousands of .pdf docs generated from the same form but with lots of variability. We need to extract data from open text and more important - from tables inside the docs. The output of Couchbase/Mongo will be one row per document for backend processing. ADOBE renders the tables in an unusable form.
I prefer MongoDB due to own experience with migration of old archive of pdf and meta-data to a new “archive”. The biggest advantage is speed of filters output - a new archive is way faster and reliable then the old one - but also the the easy programming of MongoDB with many code snippets and examples available. I have no personal experience so far with Couchbase. From the architecture point of view both options are OK - go for the one you like.
I would like to suggest MongoDB or ArangoDB (can't choose both, so ArangoDB). MongoDB is more mature, but ArangoDB is more interesting if you will need to bring graph database ideas to solution. For example if some data or some documents are interlinked, then probably ArangoDB is a best solution.
To process tables we used Abbyy software stack. It's great on table extraction.
If you can select text with mouse drag in PDF. Use pdftotext it is fast! You can install it on server with command "apt-get install poppler-utils". Use it like "pdftotext -layout /path-to-your-file". In same folder it will make text file with line by line content. There is few classes on git stacks that you can use, also.
After using couchbase for over 4 years, we migrated to MongoDB and that was the best decision ever! I'm very disappointed with Couchbase's technical performance. Even though we received enterprise support and were a listed Couchbase Partner, the experience was horrible. With every contact, the sales team was trying to get me on a $7k+ license for access to features all other open source NoSQL databases get for free.
Here's why you should not use Couchbase
Full-text search Queries The full-text search often returns a different number of results if you run the same query multiple types
N1QL queries Configuring the indexes correctly is next to impossible. It's poorly documented and nobody seems to know what to do, even the Couchbase support engineers have no clue what they are doing.
Community support I posted several problems on the forum and I never once received a useful answer
Enterprise support It's very expensive. $7k+. The team constantly tried to get me to buy even though the community edition wasn't working great
Autonomous Operator It's actually just a poorly configured Kubernetes role that no matter what I did, I couldn't get it to work. The support team was useless. Same lack of documentation. If you do get it to work, you need 6 servers at least to meet their minimum requirements.
Couchbase cloud Typical for Couchbase, the user experience is awful and I could never get it to work.
Minimum requirements
The minimum requirements in production are 6 servers. On AWS the calculated monthly cost would be ~$600
. We achieved better performance using a $16
MongoDB instance on the Mongo Atlas Cloud
writing queries is a nightmare While N1QL is similar to SQL and it's easier to write because of the familiarity, that isn't entirely true. The "smart index" that Couchbase advertises is not smart at all. Creating an index with 5 fields, and only using 4 of them won't result in Couchbase using the same index, so you have to create a new one.
Couchbase UI
The UI that comes with every database deployment is full of bugs, barely functional and the developer experience is poor. When I asked Couchbase about it, they basically said they don't care because real developers use SQL directly from code
Consumes too much RAM
Couchbase is shipped with a smaller Memcached instance to handle the in-memory cache. Memcached ends up using 8 GB of RAM for 5000 documents
! I'm not kidding! We had less than 5000 docs on a Couchbase instance and less than 20 indexes and RAM consumption was always over 8 GB
Memory allocations are useless I asked the Couchbase team a question: If a bucket has 1 GB allocated, what happens when I have more than 1GB stored? Does it overflow? Does it cache somewhere? Do I get an error? I always received the same answer: If you buy the Couchbase enterprise then we can guide you.
Minio is a free and open source object storage system. It can be self-hosted and is S3 compatible. During the early stage it would save cost and allow us to move to a different object storage when we scale up. It is also fast and easy to set up. This is very useful during development since it can be run on localhost.
We implemented our first large scale EPR application from naologic.com using CouchDB .
Very fast, replication works great, doesn't consume much RAM, queries are blazing fast but we found a problem: the queries were very hard to write, it took a long time to figure out the API, we had to go and write our own @nodejs library to make it work properly.
It lost most of its support. Since then, we migrated to Couchbase and the learning curve was steep but all worth it. Memcached indexing out of the box, full text search works great.
We offer our customer HIPAA compliant storage. After analyzing the market, we decided to go with Google Storage. The Nodejs API is ok, still not ES6 and can be very confusing to use. For each new customer, we created a different bucket so they can have individual data and not have to worry about data loss. After 1000+ customers we started seeing many problems with the creation of new buckets, with saving or retrieving a new file. Many false positive: the Promise returned ok, but in reality, it failed.
That's why we switched to S3 that just works.
Pros of Amazon S3
- Reliable590
- Scalable492
- Cheap456
- Simple & easy329
- Many sdks83
- Logical30
- Easy Setup13
- REST API11
- 1000+ POPs11
- Secure6
- Easy4
- Plug and play4
- Web UI for uploading files3
- Faster on response2
- Flexible2
- GDPR ready2
- Easy to use1
- Plug-gable1
- Easy integration with CloudFront1
Pros of Couchbase
- High performance18
- Flexible data model, easy scalability, extremely fast18
- Mobile app support9
- You can query it with Ansi-92 SQL7
- All nodes can be read/write6
- Equal nodes in cluster, allowing fast, flexible changes5
- Both a key-value store and document (JSON) db5
- Open source, community and enterprise editions5
- Automatic configuration of sharding4
- Local cache capability4
- Easy setup3
- Linearly scalable, useful to large number of tps3
- Easy cluster administration3
- Cross data center replication3
- SDKs in popular programming languages3
- Elasticsearch connector3
- Web based management, query and monitoring panel3
- Map reduce views2
- DBaaS available2
- NoSQL2
- Buckets, Scopes, Collections & Documents1
- FTS + SQL together1
Sign up to add or upvote prosMake informed product decisions
Cons of Amazon S3
- Permissions take some time to get right7
- Requires a credit card6
- Takes time/work to organize buckets & folders properly6
- Complex to set up3
Cons of Couchbase
- Terrible query language3