Publish multiple readings with MQTT

Your new topic does not fit any of the above??? Check first. Then post here. Thanks.

Moderator: igrr

 Reply

User avatar

Publish multiple readings with MQTT

By Sunny Jones
- Tue May 09, 2017 6:30 am #65758 I have a project were I want to take multiple readings from 1-4 current sensors and publish them to a MQTT broker. Right now I´m not sure about sampling frequency that I need for these sensors and if I could use oversampling to decrease the number of values to publish. These readings will occur up to 20 times per hour when the current is above a threshold value and will last for 2-15 seconds.
But let´s say that I want to read 4 sensors at 100Hz for 15 seconds and then publish them to an MQTT. So it would be up to 6000 values that I need to publish to the broker. It is not important to publish these values exactly in real time as I suspect that could be a problem. But directly after the readings goes below the threshold value they need to be published.
Most of the MQTT-examples that I´ve found only publish couple of readings at a time.

Should I store these readings in int arrays and then publish the whole array or is it possible to publish each separate value in this array?
If it´s best to publish the array as one value do I need to convert these values to a char array or is there a way that are more efficient?

Maybe someone here have experience to deal with this type of problem

The connection to the MQTT server could be quite slow with GPRS up to 4G
User avatar

Re: Publish multiple readings with MQTT

By piersfinlayson
- Fri May 12, 2017 3:24 am #65856 With that sort of quantity of data you're going to struggle to send all of it in one go. Let's assume that your ints are each 4 bytes in size, and you need a byte to separate the ints from each other in the message you are sending. That's 5 bytes per value, with 6,000 values, that comes to 30KB. You'd require pretty much all of the RAM on the ESP8266 to send these in one go. (That's ignoring the fact that you've stored them off an in array which has used a similar amount of data ... so you may not be able to store all 6,000 values in one go either unless you have smaller ints).

In addition to the above if you're using tuanpmt's mqtt stack you'll find that it has its own circular buffer to store messages in while sending. I forget the default size of this, but you want to keep it fairly low for the rest of your code to have some RAM to play with. I keep this at 4KB in my code.

Therefore you're going to have to pace your sending of this sort of quantity of data - send some, wait until the send has completed (may need to add some sort of callback to the MQTT library if it doesn't already have one - or could use a timer which is a bit risky given you might be on GPRS or 4G so will take differing amounts of time). I tend to limit myself to sending around 256 bytes at a time.

On the other hand you definitely don't want to send each value one at a time. This would be pretty inefficient (I don't know how efficient MQTT is - probably quite - but at the very least you're duplicating the topic each time, and there will be some message/TCP/IP overhead as well).
 Reply
Return to General Discussions