redis needs memory to work well - is great as an ephemeral INDEX populated from a map-reduced kafka queue maybe..
kafka is simple, fast, durable, replicatable, specializable (kafka of kafkas). You can, for example have a load balanced front end shard based on message event details into independent kafka queues, or just RAID-10 on SSD for performance scaling (5GB/sec should be doable on 1 node - with enough money). kafka doesn't really take CPU, and it's linear disk access patterns.
however, kafka doesn't really provide transactions, so depending on what you want to do on the other end of collecting the event data, this could be an issue.. However, you could still use kafka as a durable "inbox" then stream it into a redis or RabbitMQ or JMS or SQS (or even EMR hadoop style reporting) or whatever for event-handling. I would personally steer away from a transactional Queue for inbox, because you'll drop in-bound messages when you saturate IoPs (lets say during a RAID rebuild when you're only at 10% IOP capacity). As an example, We've lost SQS events in amazon; that's just not a problem with kafka (because there's no state management to speak of)
Thanks for detailed review. I took time to digest all things. And Please One other request, suggest any tool or way for MQTT to Kafka, build it custom or any library available?