Eclipse Mosquitto™ is an open source (EPL/EDL licensed) message broker that implements the MQTT protocol versions 3.1 and 3.1.1. MQTT provides a lightweight method of carrying out messaging using a publish/subscribe model. This makes it suitable for "Internet of Things" messaging such as with low power sensors or mobile devices such as phones, embedded computers or microcontrollers like the Arduino.

Mosquitto is an iot.eclipse.org project

Version 1.4.14 released

This is a bugfix release.

Version 1.4.13 contained a regression that meant persistence data was only being saved after client information had been freed. This release fixes that.

If you use persistence then it is strongly recommended to avoid 1.4.13 so you do not suffer data loss.

Version 1.4.13 released

This is a bugfix and security release.

Security

  • Fix CVE-2017-9868. The persistence file was readable by all local users,
    potentially allowing sensitive information to be leaked.
    This can also be fixed administratively, by restricting access to the
    directory in which the persistence file is stored.

Broker

  • Fix for poor websockets performance.
  • Fix lazy bridges not timing out for idle_timeout. Closes #417.
  • Fix problems with large retained messages over websockets. Closes #427.
  • Set persistence file to only be readable by owner, except on Windows. Closes
    #468.
  • Fix CONNECT check for reserved=0, as per MQTT v3.1.1 check MQTT-3.1.2-3.
  • When the broker stop, wills for any connected clients are now “sent”. Closes
    #477.
  • Auth plugins can be configured to disable the check for +# in usernames/client ids with the auth_plugin_deny_special_chars option. Partially closes #462.
  • Restrictions for CVE-2017-7650 have been relaxed – ‘/’ is allowed in usernames/client ids. Remainder of fix for #462.

Clients

  • Don’t use / in auto-generated client ids.

A vulnerability exists in Mosquitto versions 0.15 to 1.4.12 inclusive known as CVE-2017-9868.

If persistence is enabled, then the persistence file is created world readable, which has the potential to make sensitive information available to any local user.

Patches are available to fix this for Unix like operating systems (i.e. not Windows): https://mosquitto.org/files/cve/2017-9868/

This will be fixed in version 1.4.13, due to be released shortly.

This can also be fixed administratively by removing world read permissions for the directory that the persistence file is stored in. In many systems this can be achieved with:

chmod 700 /var/lib/mosquitto

A short paper has been published on Mosquitto in The Journal of Open Source Software. If you use Mosquitto in your academic work, please now use this paper as your citation.

R. A. Light, “Mosquitto: server and client implementation of the MQTT protocol,” The Journal of Open Source Software, vol. 2, no. 13, May 2017, DOI: 10.21105/joss.00265

The paper link is http://dx.doi.org/10.21105/joss.00265

A bibtex entry is available here.

A vulnerability exists in Mosquitto versions 0.15 to 1.4.11 inclusive known as CVE-2017-7650.

Pattern based ACLs can be bypassed by clients that set their username/client id to ‘#’ or ‘+’. This allows locally or remotely connected clients to access MQTT topics that they do have the rights to. The same issue may be present in third party authentication/access control plugins for Mosquitto.

The vulnerability only comes into effect where pattern based ACLs are in use, or potentially where third party plugins are in use.

The issue is fixed in Mosquitto 1.4.12, which has just been released. Patches for older versions are available at https://mosquitto.org/files/cve/2017-7650/

The fix addresses the problem by restricting access for clients with a ‘#’, ‘+’, or ‘/’ in their username or client id. ‘/’ has been included in the list of characters disallowed because it also has a special meaning in a topic and may represent an additional risk. The restriction placed on clients is that they may not receive or send messages that are subject to a pattern based ACL check, nor any message that is subject to a plugin check.

Thanks to Artem Zinenko from HackerDom CTF team for finding this vulnerability and responsibly reporting it.

Complete list of fixes addressed in version 1.4.12:

Broker

  • Fix mosquitto.db from becoming corrupted due to client messages being persisted with no stored message. Closes #424.
  • Fix bridge not restarting properly. Closes #428.
  • Fix unitialized memory in gets_quiet on Windows. Closes #426.
  • Fix building with WITH_ADNS=no for systems that don’t use glibc. Closes #415.
  • Fixes to readme.md.
  • Fix deprecation warning for OpenSSL 1.1. PR #416.
  • Don’t segfault on duplicate bridge names. Closes #446.
  • Fix CVE-2017-7650

For the final time…

This guy arrived on Tuesday, two weeks early and weighing 9lb 6oz / 4.26kg. Apologies if I’m a bit out of touch for a while.

Version 1.4.11 released

This is a bugfix release.

Broker

  • Fix crash when “lazy” type bridge attempts to reconnect. Closes #259.
  • maximum_connections now applies to websockets listeners. Closes #271.
  • Allow bridges to use TLS with IPv6.
  • Don’t error on zero length persistence files. Closes #316.
  • For http only websockets clients, close files served over http in all cases when the client disconnects. Closes #354.
  • Fix error message when websockets http_dir directory does not exist.
  • Improve password utility error message. Closes #379.
  • Bridges can use asynchronous DNS lookups on systems with glibc. This can be enabled at compile time using “WITH_ADNS=yes”.

Clients

  • Use of –ciphers no longer requires you to also pass –tls-version. Closes #380.

Client library

  • Clients can now use TLS with IPv6.
  • Fix potential socket leakage when reconnecting. Closes #304.
  • Fix potential negative timeout being passed to pselect. Closes #329.

Pre Christmas Update

I have taken a bit of a break from Mosquitto for the past few months, partly because I needed a break but also to work on another unrelated project. I’m now back and working on Mosquitto again, primarily implementing support for the upcoming MQTT v5 spec which has added even more features since I mentioned last wrote about it. Once that is in a state that is reasonably compliant if incomplete, I will be looking for testers.

There are a few fixes in the repository waiting for release, I anticipate releasing 1.4.11 before the end of the year.

There have been some changes to test.mosquitto.org. On its original host I was seeing lots of bandwidth being used by lots of clients, but in particular lots and lots of tiny connections being made which not showing up on my bandwidth monitoring, but were consuming bandwidth and causing problems at my provider. My provider got in touch to say that at times approximately half of the network flows for their network were related to test.mosquitto.org, and could would I please have a chat with the transit provider to discuss how best to manage this service. In the face of that and the risk of exceeding 2TB bandwidth usage per month, test.mosquitto.org has been moved to a lower spec host with smaller pipes and “automatic DDOS protection”. This means I now get half a dozen emails per day to say that test.mosquitto.org is under attack. If you find you can’t connect to test.mosquitto.org, it might be because you have been blocked by this DDOS protection – if so, maybe think about how you are using the service.

The final thought for this post – if you are part of a company that uses mosquitto and it adds value to your company, please consider making a donation to the project that reflects that value. If it is difficult for your company to make donations but you would still like to contribute back, please get in touch and maybe we can arrange something.

Version 1.4.10 released

This is a bugfix release.

Broker

  • Fix TLS operation with websockets listeners and libwebsockets 2.x. Closes
    #186.
  • Don’t disconnect client on HUP before reading the pending data. Closes #7.
  • Fix some $SYS messages being incorrectly persisted. Closes #191.
  • Support OpenSSL 1.1.0.
  • Call fsync after persisting data to ensure it is correctly written. Closes
    #189.
  • Fix persistence saving of subscription QoS on big-endian machines.
  • Fix will retained flag handling on Windows. Closes #222.
  • Broker now displays an error if it is unable to open the log file. Closes
    #234.

Client library

  • Support OpenSSL 1.1.0.
  • Fixed the C++ library not allowing SOCKS support to be used. Closes #198.
  • Fix memory leak when verifying a server certificate with a subjectAltName
    section. Closes #237.

Build

  • Don’t attempt to install docs when WITH_DOCS=no. Closes #184.

MQTT v5 draft features

The MQTT Technical Committee at OASIS continue to work on improvements to MQTT. The next version looks set to be MQTT version 5 and has reached the “working draft” stage. This post lists some of the changes that are in the working draft 02 and so gives at least a flavour of the improvements coming up. Take this with a pinch of salt, I may have missed some changes and there is no commitment that any of these features will remain in the final specification as they are described here.

Session management

In MQTT v3.1.1 and earlier, a client could control how the server treats its session with the clean session flag. If set to 1, the server would delete any existing session for that client and would not persist the session after disconnecting. If set to 0, the server would restore any existing session for a client when it reconnected, and persist the session when the client disconnected.

A session here means the subscriptions for a client and any queued messages.

The new spec changes this behaviour. The clean session flag has been renamed to clean start (this was actually the name of the flag in the old MQTT v3 spec) and now only affects how the broker handles a client session when the client connects. If set to 1, the server discards any previous session information, otherwise session information is kept.

To deal with removing of sessions at any other time, a new identifier/value pair has been introduced. These identifier/value pairs are an addition to the variable header part of some MQTT packets and allow configuring of different behaviour. In the case of the CONNECT packet, a Session Expiry interval can be specified which is a 4 byte integer that gives the number of seconds after a client has disconnected that the server should remove session information for that client. If the Session Expiry interval is absent from the CONNECT packet, then the session will never expire. If it is set to 0, then the session is removed as soon as the client disconnects.

The new clean start flag and session expiry interval allow the existing clean session behaviour to be duplicated but also allow client sessions to be expired based on time.

Updated Connect Return codes

The return codes passed to the client in a CONNACK packet have been expanded to include:

  • 6: Connection Refused, reason unspecified
  • 7: Connection Refused, implementation specific
  • 8: Connection Refused, CONNECT packet was malformed

Repeated topics when publishing

When publishing data to a single topic, a new feature will help reduce bandwidth use. A client or server can set the topic in a PUBLISH message to be a zero length string. This tells the client/server being published to, to use the previous topic instead. This goes some way to reducing the current overhead associated with publishing – a shame it isn’t quite as good as the registered topics available in MQTT-SN.

Payload Format Indicator

Another identifier/value pair is available for use when sending a PUBLISH message. This is the Payload Format indicator. If present and set to 1, this indicates that the PUBLISH payload is UTF-8 encoded data. If set to 0, or if the indicator is not present then the payload is an unspecified byte format, exactly as with MQTT v3.1.1.

Publication Expiry interval

This is an identifier/value pair for use when publishing. If present, this value is a 4 byte integer which gives the number of seconds for which the server will attempt to deliver this message to a subscriber. This means that an offline client with messages being queued may not receive all of the messages when it reconnects, due to some of them expiring. Interestingly, when the server does deliver a message that had a Publication Expiry set, it sets the Publication Expiry on the outgoing message to the client but with the amount of time that there is left until the message expires. This means that the true time to expiry will propagate through bridges or similar.

Publish Return Codes

The PUBACK and PUBREC packets have a new entry in their variable header which is the Publish Return Code. This can be used to tell the client a message has been refused for various reasons, accepted, or accepted with no matching subscribers.  For the PUBREC packet, if the message is refused or accepted with no matching subscribers then there is no expectation for the PUBREL/PUBCOMP messages to be sent for that message.

The PUBCOMP packet also has a similar entry which has the same set of return codes and an additional one for the case when a message had expired. This is for the case when a client reconnects with clean start set to 0 and it has a QoS 2 message part way through its handshake, but the server has already expired the message.

There is still no way to tell a client that its QoS 0 message was refused.

Disconnect notification

In MQTT v3.1.1 and before, only the client sends a DISCONNECT packet. In the draft spec, either the client or the server can send DISCONNECT and it is used to indicate a reason for disconnection. The disconnect return codes are:

  • 0: Connection disconnected by application (sent by client)
  • 1: Server temporarily unavailable (server)
  • 2: Server unavailable (server)
  • 3: Malformed UNSUBSCRIBE packet received (server)
  • 4: Session taken over (server – for when another client connects with the same ID)
  • 5: Malformed packet received

It is clear that there is some duplication there, so I think this is a likely place that changes will be made.

Disconnect expiry notification

The DISCONNECT packet can also include a Session Expiry interval value, as with CONNECT. This allows a client to clean up when it disconnects, or to set a long expiry time, even if these were not specified at the initial connect.