Review
BigQuery is not a good choice for this due to the columnar implementation under the hood and the lack of customizable indexing as you already discovered. It's a great tool to process very high amount of data cheaply in little time, but you will not get to milliseconds of latency in any case. BigTable is a better suited alternative for this, DynamoDB as well if AWS products are of interest. Depending on how big your dataset is and how fast you append data to it, a relational database like Postgres might be a good option too, it can definitely handle the scale!
0 views0