Ivan Begtin

I don't think that OpenRefine and Apache Hive are compatible for such tasks. If you need to cleanup and process huge amount of data (big data) I would recommend to use Clickhouse instead and to do data processing tasks using SQL queries, not manually.

OpenRefine is a great tool with the great limitations. It doesn't handle big datasets, it doesn't scale, it doesn't handle JSON documents with sub-documents.

I am a big fan of MongoDB and It's great for document storage but I am not really sure that it's the best engine for log storage. If data that you store is "flat" and well-defined than log storage based on engines like Clickhouse or Elasticsearch stach could be much more efficient. Also it's quite important how you reuse collected logs. Do you calculate aggregated metrics? Do you need full search ? And so on.

If logs are really simple and full text search needed than Logstash + Elasticsearch. If you need to calculate a lot of metrics and logs are not just text, but include numbers/values needed for aggregation than Clickhouse.

Both Asana and Trello support Kanban style project tracking. Trello is Kanban-only project management, knowledge management, actually card-management tools. Asana is much more complex, supports different project management approaches, well integrated and helpful for any style/type project.

We choose Asana finally, but still some projects kept in Trello

Used it for a long time ago, it's just common sense choice for mail delivery solutions. In Infoculture we use it for weekly newsletter and it works excellent for any amount of newsletters and subscribers.

Initially we had choice over email tools, but finally Mailchimp was right choice.

It's extremely configurable, ready for self-hosted solution and easy scalable. We use it with Scaleway cloud hosting as preconfigured image.

I would recommend Gitlab as helpful alternative to Github, good for teams that have limitations on using Github.

If you have a lot of realtime operations and not-so-high variability of data so Postgres is the best choice. MongoDB is less suitable for high-performance projects but is good for semi-structured data and not-realtime projects. For example we use MongoDB as DB for analytics but we use Postgres for any high-load projects

I would like to suggest MongoDB or ArangoDB (can't choose both, so ArangoDB). MongoDB is more mature, but ArangoDB is more interesting if you will need to bring graph database ideas to solution. For example if some data or some documents are interlinked, then probably ArangoDB is a best solution.

To process tables we used Abbyy software stack. It's great on table extraction.