Anouncement: Amqtt_db relational persistance layer for the amqtt MQTT broker
The Amqtt_db relational persistance layer enables the Amqtt MQTT broker to effectively store MQTT data in typed DB columns.Amqtt_db can can utilize all DB-Systems supported by SQLAlchemy and the next release targets timescaleDB for really high performance applications.
The aim of Amqtt_DB is to overcome performance problems in large IoT systems where the fast analysis of the data is crucial.
Most IoT systems use object stores as backends, which is quite good for high throughput of messages but on the other hand stores the individual measures in terms of blobs. So in many cases additional indexing mechanisms like elastic search are used to accelaerate the access to the backend data - which introduces unwanted complexity.
Also most IoT-Systems which persist data are missusing a MQTT-Client which registers itself on all messages on the broker. This is a quite flexible way but not optimal since the MQTT broker and/or the MQTT Client became bottlenecks.
Now the revival of the hbmqtt MQTT Broker (now called amqtt) comes to rescue. Amqtt is written in python utilizing asyncio for fast switching of IO workloads. Amqtt_db is a native plugin for the amqtt plugin architecture leverage the asyncio nature and therefore scales quite well.
The most crucial part of persisting MQTT messages in general is that there are no standards for the MQTT payloads. With Homie a first efford was made to standarize MQTT payloads. So in the future amqtt_db will be conplient with Homie. But not all MQTT-Sender will use Homie. So amqtt_db inludes a small really extensible framework for Payload decoding, wich will be factored out into a seperate Python packet in the next release.
Amqtt_db is fully controlled via yaml config files.
Please test amqtt and amqtt_db