Fast and reliable large-scale data processing engine