Alternatives to pgAdmin

Which tools are preferred if I choose to work on more data side? Which one is good if I decide to work on web development? I'm using DBeaver and am now considering a move to AzureDataStudio to break the monotony while working. I would like to hear your opinion. Which one are you using, and what are the things you are missing in dbeaver or data studio.

Our whole DevOps stack consists of the following tools:

• GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
• Respectively Git as revision control system
• SourceTree as Git GUI
• Visual Studio Code as IDE
• CircleCI for continuous integration (automatize development process)
• Prettier / TSLint / ESLint as code linter
• SonarQube as quality gate
• Docker as container management (incl. Docker Compose for multi-container application management)
• VirtualBox for operating system simulation tests
• Kubernetes as cluster management for docker containers
• Heroku for deploying in test environments
• nginx as web server (preferably used as facade server in production environment)
• SSLMate (using OpenSSL) for certificate management
• Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
• PostgreSQL as preferred database system
• Redis as preferred in-memory database/store (great for caching)

The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:

• Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
• Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
• Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
• Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
• Scalability: All-in-one framework for distributed systems.
• Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.

Recently we were looking at a few robust and cost-effective ways of replicating the data that resides in our production MongoDB to a PostgreSQL database for data warehousing and business intelligence.

We set ourselves the following criteria for the optimal tool that would do this job: - The data replication must be near real-time, yet it should NOT impact the production database - The data replication must be horizontally scalable (based on the load), asynchronous & crash-resilient

Based on the above criteria, we selected the following tools to perform the end to end data replication:

We chose MongoDB Stitch for picking up the changes in the source database. It is the serverless platform from MongoDB. One of the services offered by MongoDB Stitch is Stitch Triggers. Using stitch triggers, you can execute a serverless function (in Node.js) in real time in response to changes in the database. When there are a lot of database changes, Stitch automatically "feeds forward" these changes through an asynchronous queue.

We chose Amazon SQS as the pipe / message backbone for communicating the changes from MongoDB to our own replication service. Interestingly enough, MongoDB stitch offers integration with AWS services.

In the Node.js function, we wrote minimal functionality to communicate the database changes (insert / update / delete / replace) to Amazon SQS.

Next we wrote a minimal micro-service in Python to listen to the message events on SQS, pickup the data payload & mirror the DB changes on to the target Data warehouse. We implemented source data to target data translation by modelling target table structures through SQLAlchemy . We deployed this micro-service as AWS Lambda with Zappa. With Zappa, deploying your services as event-driven & horizontally scalable Lambda service is dumb-easy.

In the end, we got to implement a highly scalable near realtime Change Data Replication service that "works" and deployed to production in a matter of few days!