What is Azure Storage and what are its top alternatives?
Top Alternatives to Azure Storage
- Azure Redis Cache
It perfectly complements Azure database services such as Cosmos DB. It provides a cost-effective solution to scale read and write throughput of your data tier. Store and share database query results, session states, static contents, and more using a common cache-aside pattern. ...
- Amazon S3
Amazon Simple Storage Service provides a fully redundant data storage infrastructure for storing and retrieving any amount of data, at any time, from anywhere on the web ...
- Azure Cosmos DB
Azure DocumentDB is a fully managed NoSQL database service built for fast and predictable performance, high availability, elastic scaling, global distribution, and ease of development. ...
Outlook.com is a free, personal email service from Microsoft. Keep your inbox clutter-free with powerful organizational tools, and collaborate easily with OneDrive and Office Online integration. ...
Harness the power of Dropbox. Connect to an account, upload, download, search, and more. ...
- Google Cloud Storage
Google Cloud Storage allows world-wide storing and retrieval of any amount of data and at any time. It provides a simple programming interface which enables developers to take advantage of Google's own reliable and fast networking infrastructure to perform data operations in a secure and cost effective manner. If expansion needs arise, developers can benefit from the scalability provided by Google's infrastructure. ...
- Amazon EBS
Amazon EBS volumes are network-attached, and persist independently from the life of an instance. Amazon EBS provides highly available, highly reliable, predictable storage volumes that can be attached to a running Amazon EC2 instance and exposed as a device within the instance. Amazon EBS is particularly suited for applications that require a database, file system, or access to raw block level storage. ...
Minio is an object storage server compatible with Amazon S3 and licensed under Apache 2.0 License ...
Azure Storage alternatives & related posts
related Azure Redis Cache posts
- Simple & easy329
- Many sdks83
- Easy Setup13
- REST API11
- 1000+ POPs11
- Plug and play4
- Web UI for uploading files3
- GDPR ready2
- Faster on response2
- Easy integration with CloudFront1
- Easy to use1
- Permissions take some time to get right7
- Takes time/work to organize buckets & folders properly6
- Requires a credit card5
- Complex to set up3
related Amazon S3 posts
To provide employees with the critical need of interactive querying, we’ve worked with Presto, an open-source distributed SQL query engine, over the years. Operating Presto at Pinterest’s scale has involved resolving quite a few challenges like, supporting deeply nested and huge thrift schemas, slow/ bad worker detection and remediation, auto-scaling cluster, graceful cluster shutdown and impersonation support for ldap authenticator.
Our infrastructure is built on top of Amazon EC2 and we leverage Amazon S3 for storing our data. This separates compute and storage layers, and allows multiple compute clusters to share the S3 data.
We have hundreds of petabytes of data and tens of thousands of Apache Hive tables. Our Presto clusters are comprised of a fleet of 450 r4.8xl EC2 instances. Presto clusters together have over 100 TBs of memory and 14K vcpu cores. Within Pinterest, we have close to more than 1,000 monthly active users (out of total 1,600+ Pinterest employees) using Presto, who run about 400K queries on these clusters per month.
Each query submitted to Presto cluster is logged to a Kafka topic via Singer. Singer is a logging agent built at Pinterest and we talked about it in a previous post. Each query is logged when it is submitted and when it finishes. When a Presto cluster crashes, we will have query submitted events without corresponding query finished events. These events enable us to capture the effect of cluster crashes over time.
Each Presto cluster at Pinterest has workers on a mix of dedicated AWS EC2 instances and Kubernetes pods. Kubernetes platform provides us with the capability to add and remove workers from a Presto cluster very quickly. The best-case latency on bringing up a new worker on Kubernetes is less than a minute. However, when the Kubernetes cluster itself is out of resources and needs to scale up, it can take up to ten minutes. Some other advantages of deploying on Kubernetes platform is that our Presto deployment becomes agnostic of cloud vendor, instance types, OS, etc.
#BigData #AWS #DataScience #DataEngineering
Our whole DevOps stack consists of the following tools:
- GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
- Respectively Git as revision control system
- SourceTree as Git GUI
- Visual Studio Code as IDE
- CircleCI for continuous integration (automatize development process)
- Prettier / TSLint / ESLint as code linter
- SonarQube as quality gate
- Docker as container management (incl. Docker Compose for multi-container application management)
- VirtualBox for operating system simulation tests
- Kubernetes as cluster management for docker containers
- Heroku for deploying in test environments
- nginx as web server (preferably used as facade server in production environment)
- SSLMate (using OpenSSL) for certificate management
- Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
- PostgreSQL as preferred database system
- Redis as preferred in-memory database/store (great for caching)
The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:
- Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
- Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
- Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
- Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
- Scalability: All-in-one framework for distributed systems.
- Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
- Best-of-breed NoSQL features28
- High scalability22
- Globally distributed15
- Automatic indexing over flexible json data model14
- Tunable consistency10
- Always on with 99.99% availability sla10
- Predictable performance6
- High performance5
- Analytics Store5
- Rapid Development2
- No Sql2
- Auto Indexing2
- Ease of use2
- Poor No SQL query support4
related Azure Cosmos DB posts
We have an in-house build experiment management system. We produce samples as input to the next step, which then could produce 1 sample(1-1) and many samples (1 - many). There are many steps like this. So far, we are tracking genealogy (limited tracking) in the MySQL database, which is becoming hard to trace back to the original material or sample(I can give more details if required). So, we are considering a Graph database. I am requesting advice from the experts.
- Is a graph database the right choice, or can we manage with RDBMS?
- If RDBMS, which RDMS, which feature, or which approach could make this manageable or sustainable
- If Graph database(Neo4j, OrientDB, Azure Cosmos DB, Amazon Neptune, ArangoDB), which one is good, and what are the best practices?
I am sorry that this might be a loaded question.
Hi Mohamad, out of these two options, I'd recommend starting with MongoDB (on MongoDB Atlas) for a few reasons:
• Open Source & Portability - With MongoDB being open source, you have transparency into how your system will work. Not only can you see how it works, but you later have the option to migrate to self-hosted versions of the platform (decreasing costs and avoiding vendor lock-in) or move to a Mongo-compatible hosted database like Amazon DocumentDB or Azure Cosmos DB.
• Querying & Aggregation - MongoDB has been around a few years longer than Firebase, and in my opinion, that is evident from the great design and flexibility of APIs you have for querying and aggregating data.
• Tooling - MongoDB Atlas monitoring tools and the Compass GUI are great for understanding and interacting with the data in your database as you're growing your platform.
I hope this helps!
- Back up1
- Stable service1
related OneDrive posts
- Easy to work with432
- Shared file hosting175
- 'just works'168
- No brainer100
- Integration with external services79
- Good api49
- Least cost (free) for the basic needs case38
- It just works10
- Accessible from all of my devices6
- Command Line client5
- Can even be used by your grandma4
- Mac app3
- Sync API3
- Synchronizing laptop and desktop - work anywhere3
- Cross platform app2
- Extended version history2
- Beautiful UI2
- Delta synchronization2
- Ability to pay monthly without losing your files2
- Easy/no setup1
- YC Company1
- Everybody needs to share and synchronize files reliably1
- Backups, local and cloud1
- The more the merrier1
- For when client needs file without opening firewall1
- Easy to work with1
- Everybody needs to share and synchronize files reliabl1
- Easy to use1
- So easy1
- What a beautiful app1
- The more the merrier0
- Personal vs company account is confusing3
- Replication kills CPU and battery1
related Dropbox posts
Anyone recommend a good connector like Kloudless for connecting a SaaS app to Dropbox/Box etc? Cheers
- More praticlal and easy1
related Google Cloud Storage posts
In #Aliadoc, we're exploring the crowdfunding option to get traction before launch. We are building a SaaS platform for website design customization.
For the Admin UI and website editor we use React and we're currently transitioning from a Create React App setup to a custom one because our needs have become more specific. We use CloudFlare as much as possible, it's a great service.
For routing dynamic resources and proxy tasks to feed websites to the editor we leverage CloudFlare Workers for improved responsiveness. We use Firebase for our hosting needs and user authentication while also using several Cloud Functions for Firebase to interact with other services along with Google App Engine and Google Cloud Storage, but also the Real Time Database is on the radar for collaborative website editing.
We generally hate configuration but honestly because of the stage of our project we lack resources for doing heavy sysops work. So we are basically just relying on Serverless technologies as much as we can to do all server side processing.
Visual Studio Code definitively makes programming a much easier and enjoyable task, we just love it. We combine it with Bitbucket for our source code control needs.
- Point-in-time snapshots36
- Data reliability27
- Configurable i/o performance19
related Amazon EBS posts
We are looking for a centralised monitoring solution for our application deployed on Amazon EKS. We would like to monitor using metrics from Kubernetes, AWS services (NeptuneDB, AWS Elastic Load Balancing (ELB), Amazon EBS, Amazon S3, etc) and application microservice's custom metrics.
We are expected to use around 80 microservices (not replicas). I think a total of 200-250 microservices will be there in the system with 10-12 slave nodes.
We tried Prometheus but it looks like maintenance is a big issue. We need to manage scaling, maintaining the storage, and dealing with multiple exporters and Grafana. I felt this itself needs few dedicated resources (at least 2-3 people) to manage. Not sure if I am thinking in the correct direction. Please confirm.
You mentioned Datadog and Sysdig charges per host. Does it charge per slave node?
- Store and Serve Resumes & Job Description PDF, Backups9
- S3 Compatible6
- Open Source4
- Encryption and Tamper-Proof3
- Private Cloud Storage2
- Lambda Compute2
- Data Protection2
- Highly Available2
- Pluggable Storage Backend2
- Deletion of huge buckets is not possible2