You can use ReductStore to keep a history of MQTT messages by using its Client SDKs. This can be useful if you use a binary format for your data and it can be recorded in a classical TSDB. You can set a FIFO quota for a bucket in your ReductStore instance so that the database removes old MQTT messages when the limit is reached.
Hi All,
I'm looking to develop an ESP32-based local (that is the hub is on the same network) and small (usually no more than 10 devices per hub) IoT network over wifi.
For Most of the runtime, the devices report readings with max 1s frequency to the hub, and MQTT is great for this. But I'm looking for future needs such as FOTA, and maybe some RPC on the devices. MQTT can be used to execute functions (with the result reported on another topic) on the edge devices but it feels hacky and wrong long term.
I've looked at various IoT frameworks, but many are paid which is a non-starter or seems like overkill (like a cloud backend for millions of devices). I'm looking for the best tools for a small device network, running on a local resource-constrained Linux.
I'm hoping for open-ended feedback or some library recommendations for further reading, Many thanks in advance!
I haven't yet used NATS for IoT but based on the NATS documentation:
"In greenfield IoT deployments, when possible, we prefer NATS extended out to endpoints and devices for a few reasons. There are significant advantages with security and observability when using a single technology end to end. Compared to MQTT, NATS is nearly as lightweight in terms of protocol bandwidth and maintainer supported clients efficiently utilize resources so we consider NATS to be a good choice to use end to end, including use on resource constrained devices."
https://docs.nats.io/nats-server/configuration/mqtt
My personal view is that NATS is simply fantastic to use with your core systems to power an event-driven architecture due to its amazing performance and easy of use. As such if you are using NATS allover the place then why add a second system to manage and to transfer information in and out from. You can just use NATS instead to cover all your needs.
NATS is amazing for IoT. Since integrating JetStream into its core it now supports various modes so you can choose one that works for you the best.
Kindly suggest the best tool for generating 10Mn+ concurrent user load. The tool must support MQTT traffic, REST API, support to interfaces such as Kafka, websockets, persistence HTTP connection, auth type support to assess the support /coverage.
The tool can be integrated into CI pipelines like Azure Pipelines, GitHub, and Jenkins.
JMeter is best suited for generating user load with built-in integrations. To generate that type of load, you’ll need to choose a cloud-based solution that runs JMeter, such as BlazeMeter or RedLine13. They support JMeter for testing RESTful APIs and there is a plugin specifically designed for MQTT. You’ll want to look at subscription options and costs to run at that great a load.
There are JMeter plugins you can configure for all sorts of profiling including persistence of connections. And third-party plugins that you can add as JAR files to your load test, such as this one specifically for Kafka. JMeter also has good support for configuring authorization headers.
With regard to CI/CD integration, both support Jenkins. RedLine13 has a custom Jenkins plugin and allows for resources for tests to be sourced from GitHub and other places like AWS S3.
If using AWS then use Database Migration Service to connect to these sources. You might need DBAs support for making the logs replicable in the source and write some transformations to convert the incoming record compatible to progreSQL.
