What is Google Cloud Dataflow?
Who uses Google Cloud Dataflow?
Why developers like Google Cloud Dataflow?
Here are some stack decisions, common use cases and reviews by companies and developers who chose Google Cloud Dataflow in their tech stack.
We really drank the Google Kool-Aid on analytics. So, everything's going into Google BigQuery and almost everything is going straight into Google Cloud Pub/Sub and then doing some processing in Google Cloud Dataflow before ending up in BigQuery. We still do too much processing and augmentation on the front end before it goes into Pub/Sub. And that's using some kind of stuff we pulled together using Amazon DynamoDB and so on. And it's very brittle, actually. Actually, Dynamo throttling is one of our biggest headaches. So, I want all of that to go away and do all our augmentation in BigQuery after the data's been collected. And having it just go straight into Pub/Sub. So, we're working on that. And it'll happen, some time. #Analytics #AnalyticsPipeline
I use Google Cloud Dataflow because it has great templates for plug and play action.
I haven't invested in the apache beam framework because you need to know Java to take full advantage of it. The Python API is a second class citizen.
Google Cloud Dataflow's Features
- Fully managed
- Combines batch and streaming with a single API
- High performance with automatic workload rebalancing Open source SDK