Thing Minder

Once upon a time, IBM developed the lightweight, TCP-based protocol MQTT. The name was an acronym for "Message Queuing Telemetry Transport"; however, the long form of the name disappeared with the ratification of MQTT as an official OASIS standard at the end of 2014.

MQTT is a pub/sub protocol. Clients connect to a server known as a broker and, if they are authorized, they can issue messages or wait for messages to arrive. By default, MQTT uses TCP port 1883 and TLS-TCP port 8883.

Unlike HTTP, MQTT does not require polling by the client; instead, the broker informs waiting subscribers as soon as a message arrives and communicates the message to them. This approach saves bandwidth and allows communication with lower latency.

The broker doesn't know what the clients do with the messages; the broker's job is over and done with as soon as the message is delivered. MQTT enables a decoupled 1-to-n or 1-to-1 relationship between different devices or programs that do not specifically need to know each other in order to communicate. The entities just need to understand each other's data.

MQTT clients, both subscribers and publishers, can come and go at any time. New clients can connect to the server at any time. However, an MQTT broker usually has access control lists (ACLs) that govern access to certain topics.

Topic, a Name String

Each message is associated with a topic. Topics are hierarchically structured name strings in UTF-8 with a length up to 64KB, and they resemble Unix paths. Unlike path names, topics do not start with a leading forward slash. For example – staying with the Internet of Things (IoT) theme – a lamp could publish its state via a lamp/kitchen/state topic, while the lamp/living_room/switch topic switches a lamp in the living room on or off. On the same broker, a fan can publish its state via air_conditioning/fan/number/1.

If certain clients want to find out the state of the lamp in the living room, they can request messages using lamp/living_room/state. If they use lamp/+/state instead, they will find out as soon as someone switches any lamp on or off. Clients can receive all activities for all lamps using lamp/#.

The special character + stands for a topic level and can occur multiple times (lamp/+/+). The # character, however, can only occur once, and always at the end of the topic; it means "this level and all underlying topic levels." Simply specifying # will show a client everything. Clients don't actually know whether they really see all the messages that arrive at the broker; the broker ACLS decide this.

Messages transport payloads; payload data can also be binary and be up to 256MB in size. This restriction is more theoretical because microcontrollers, which only have a few kilobytes of memory, also often speak MQTT and it isn't possible for them to handle such a volume of data. Payloads are often simple strings (23, to indicate a temperature) or data packed in JSON ({"celsius": 23.0 }).

Last Will and Testament

If a message is published with a topic, the broker distributes it to all subscribers following the topic. If nobody is currently following that topic, the message is lost – it isn't cached because there is no queue. To prevent this behavior, users can set a retained flag when publishing. This flag prompts the broker to store the message and to offer it to future subscribers. If, for example, the outside temperature is published with temp/terrace and a client subscribes to this topic hours later, this client is immediately sent this recently published temperature value (last-known value) if the retained flag is set.

What happens if a client unexpectedly fails? LWT (Last Will and Testament) is an important feature of MQTT. A client tells the broker during the connection setup that: "In case of my death, please send the following message with this topic …" If the broker then detects that this client is no longer alive, it does as it was told and publishes the message with the desired topic. MQTT thus has monitoring installed: It's easy to write to clients that respond to the passing of other clients, such as by sending alerts.

LWT messages are often used together with the retained flag in order to permanently describe the state of a client: When starting, a client will, for example, publish a status message with a retained flag with the clients/id/status topic. If this client deceases, the broker would distribute the dead message with a retained flag and the same topic on its behalf. A query on clients/id/status thus always reveals whether the client is online or offline.

Think of an MQTT broker as a large pot into which messages fall. Anyone can help themselves from this pot at any time. It then processes the data and publishes it again under certain circumstances.

You could, for example, imagine that a temperature sensor periodically publishes readings in Fahrenheit with the temp/fahrenheit topic. A trivial program subscribes to this topic, reads the values, converts them into degrees Celsius and publishes them again (throws them back into the pot), this time with the temp/celsius topic. The program is then simultaneously a subscriber and publisher.

Quality Control

MQTT is based on TCP and benefits from its high transmission reliability. Nevertheless, MQTT uses three service qualities to make sure messages are transmitted. These guarantees range from no assurance (Level 0) to assurance that the message at least arrive once (Level 1), to the promise that a message will arrive exactly once (Level 2).

As you can see in Figure 1, the three Quality of Service (QoS) levels are increasingly communicative: QoS Level 2 requires a fourway handshake between the client and broker. QoS 0 is usually very reliable in LAN. QoS 0 is usually enough if telemetry data is transmitted. It's not so bad if the outside temperature published every minute fails now and then – it'll be sent again a minute later anyway (QoS 0 is fine). Users might want to transmit the position of a mobile device with a bit more reliability (QoS 1 or 2), although the network traffic and costs rise, too. The higher the specified QoS level, the more network traffic you create.

Figure 1: Quality control levels for MQTT.