Michael Maraist

redis needs memory to work well - is great as an ephemeral INDEX populated from a map-reduced kafka queue maybe..

kafka is simple, fast, durable, replicatable, specializable (kafka of kafkas). You can, for example have a load balanced front end shard based on message event details into independent kafka queues, or just RAID-10 on SSD for performance scaling (5GB/sec should be doable on 1 node - with enough money). kafka doesn't really take CPU, and it's linear disk access patterns.

however, kafka doesn't really provide transactions, so depending on what you want to do on the other end of collecting the event data, this could be an issue.. However, you could still use kafka as a durable "inbox" then stream it into a redis or RabbitMQ or JMS or SQS (or even EMR hadoop style reporting) or whatever for event-handling. I would personally steer away from a transactional Queue for inbox, because you'll drop in-bound messages when you saturate IoPs (lets say during a RAID rebuild when you're only at 10% IOP capacity). As an example, We've lost SQS events in amazon; that's just not a problem with kafka (because there's no state management to speak of)

My advice to anyone learning to program is to not obsess on the language.. You SHOULD learn all languages.. Same as learning human spoken languages - the more you learn, the more ways your mind can interpret a new problem set. Learning them at the same time isn't a big deal (just like growing up in a bilingual home). Your language and your software stack are guaranteed to change 3 times in your career. Don't assume you're going to choose the "right one". And you wont waste any time learning one you never wind up using.

As a person who works on linux and OSX desktops, I have a personal bias against working for companies and software stacks that require C# or Visual Studio. But this is not due to their technical merits, but instead the OS as a platform condusive to efficient CLI toolchains and container management. But aside from that, I can use vi/IntelliJ-suite to write most languages, so language isn't a real concern. If you're windows bound, pretty much everything is available to you (through VMs and docker).

Ideally you do at least SOME full stack development learning. This means you'll need javascript, and thus nodejs would be a good stack to learn. If you ultimately like gaming or 3D, you might need C# and certainly python.

Any AI (which is a hot employment topic for the forseeable future) would like python skillz.

I personally love Java (and Android's Kotlin varient) for it's very very well designed multi-threaded libraries. go and rust are newer and thus do a slightly better job at this, but due to the open-source nature of java and editors that auto-reveal function call source code, it's very easy to learn how vendors implemented MT code and various other algorithms. Python should be equally "open" to 3rd party library review (and thus again how they solve complex problems), though a lot of times, I see python resorting to compiled C-libraries (and thus the cost to crack open the code and or trace through the code becomes too burdensome to bother).