Mosquitto is a lightweight MQTT broker that is suitable for embedded use. It also provides built-in support to persist and store messages which can be used to implement reliable communication even when the network is not stable. This post describes how to set up the persistent storage, what are the main parameters to tune it and how to troubleshoot the setup.
Cleansession and persistent storage
Normally when a MQTT broker receives a message it will deliver it immediately to all clients with matching subscriptions. If there are no subscribers, the message will not be delivered anywhere. Yet, in some situations it would make sense to keep the message in the broker and deliver it when a client comes back online. This can be solved by using a cleansession flag when connecting.
If cleansession flag is set to false when a client connects, the broker will create a persistent session. When the client is online, all messages are delivered normally. But if a client with a persistent session disconnects, the broker will keep the information about its subscriptions and will queue quality of service 1 and 2 messages until the client connects again, and then deliver the queued messages. Mosquitto stores these queues in memory. So what will happen if the broker is shut down? With Mosquitto, the queued messages are lost unless a persistent storage has been configured.
For instance, an embedded device could use Mosquitto as a local MQTT broker to have a single outgoing connection from the device using a MQTT bridge, while having multiple communicating applications locally. This can be useful for instance with Azure IoT-HUB which only allows one connection per client ID. In such architecture, it is often required that all message are delivered even though the outward connection from the device is not be stable, and the device may also be shut down or lose power. In this scenario, the persistence will also simplify the application design since individual applications can send the messages using a reliable localhost connection and the broker will handle the storage and delivery of the messages to cloud.
Configuring persistent storage
The persistent storage in Mosquitto can be enabled simply by setting the persistence option to true in the Mosquitto configuration.
persistence [ true | false ]
If true, connection, subscription and message data will be written to the disk in mosquitto.db at the location dictated by persistence_location. When mosquitto is restarted, it will reload the information stored in mosquitto.db. The data will be written to disk when mosquitto closes and also at periodic intervals as defined by autosave_interval. Writing of the persistence database may also be forced by sending mosquitto the SIGUSR1 signal. If false, the data will be stored in memory only. Defaults to false.
As documented in the excerpt above, other options that control storage behavior are persistence_location which defines the path where the database is stored and autosave_interval that defines how often (defined in seconds) the database is written to disk. If autosave_on_changes is enabled, the autosave_interval is interpreted as changes in retained, received and queued messages instead of seconds. The default database file name can be changed with persistence_file.
By default the clients are persisted indefinitely. This can lead to situations that the database file starts to accumulate if clients with cleansession=false never reconnect. This could happen for instance if the clients use randomly generated client IDs. In Mosquitto, client expiration can be set using persistent_client_expiration option. It is also good to note that by default only QoS1 and QoS2 messages are persisted. QoS0 messages can be persisted by enabling queue_qos0_messages option
There are also limits in place for how many messages and bytes are queued. By default the number of queued messages per client is 100 and the size is not limited. This functionality can be tuned using max_queued_bytes and max_queued_messages parameters. If both of these parameters are set, the messages will be queued until the either limit is reached.
Detailed description for all of the configuration options mentioned above can be found from the Mosquitto manual pages.
Retained messages
In addition to queued messages in persistent sessions, another type of persistence is retained messages. Retained message is a normal MQTT message with retain flag set to true. For these messages, the broker will keep the last message and corresponding QoS level stored for the topic. When a client subscribes, the broker will immediately deliver the last retained message of that topic. Retained messages are useful to deliver status information to newly connected clients.
In Mosquitto, the retained messages are also stored in memory unless the persistence option is set. When persistence is enabled, the retained messages are written to the Mosquitto database file.
Bridges and persistence
Bridge is a normal MQTT connection but instead of ordinary client connecting to a broker, it is a connection between two brokers. Therefore the cleansession flag is also available for bridge connections. When the connecting broker sets the cleansession to false it will cause the remote broker to keep subscriptions and queued message in case the network connection drops. This is usually the desired functionality when using a local broker in an embedded device, so the remote cloud broker will queue and then deliver messages when the device comes back online.
In Mosquitto, the cleansession can be set for each bridge connection separately.
Debugging Mosquitto persistence
It might be necessary to get status information about the current persisted messages and queue sizes, or to troubleshoot the details of the persistent messages. Basic status information can be obtained using $SYS topics.
For instance the number of persisted messages and bytes can be read from the following topics respectively.
$SYS/broker/store/messages/count
$SYS/broker/store/messages/bytes
The SYS messages are sent when notifications option is enabled. By default notifications are sent to both local and remote clients which can be problematic if the remote broker does not accept these messages. It is also possible to only send the status information locally by enabling notifications_local_only.
These status messages give an overview of the persistent state. However, sometimes it is necessary to view the persisted messages more closely. The Mosquitto project provides a dump tool to view the contents of the mosquitto.db file.
Page Fault Blog 