Mosquitto

It’s been a while since there has been an update here, so in lieu of one here are some interesting links I’ve come across recently. Add a comment to the post if you’ve done something cool not mentioned here! Work progresses on mosquitto 1.2.

Initial release of an MQTT-S gateway, written in ruby:

• https://github.com/njh/ruby-em-mqtts

And some MQTT-S tools:

• https://github.com/njh/mqtts-tools

A Pinoccio/MQTT/sensor powered Theramin:

• http://projectable.me/post/48408382189/scout6050-a-combination-of-pinoccio-mpu6050

Voice controlled MQTT LED:

• https://github.com/emmano/voicerecog4pi

An MQTT notification plugin for Jenkins/Hudson:

• https://github.com/gdubya/mqtt-notification-plugin

This is a minor bugfix release that addresses some problems identified during Debian packaging.

Broker

• mosquitto_passwd utility now uses tmpfile() to generate its temporary data storage file. It also creates a backup file that can be used to recover data if an errors occur.

Other

• Build script fixes to help packaging on Debian.

This is a bugfix release.

Broker

• Fix crash on reload if using acl patterns.

Client library

• Fix static C++ functions not being exported on Windows. Fixes bug #1098256.

Binaries should be available shortly.

This is a feature and bugfix release.

Broker

• Add $SYS/broker/messages/dropped
• Add $SYS/broker/clients/expired
• Replace $SYS/broker/+/per second/+ with moving average versions published at $SYS/broker/load/#
• Add $SYS/broker/load/sockets/+ and $SYS/broker/load/connections/+
• Documentation on password file format has been fixed.
• Disable SSL compression. This reduces memory usage significantly and removes the possibility of CRIME type attacks.
• Enable SSL_MODE_RELEASE_BUFFERS mode to reduce SSL memory usage further.
• Add allow_duplicate_messages option.
• ACL files can now have comment lines with # as the first character.
• Display message on startup about which config is being loaded.
• Fix max_inflight_messages and max_queued_messages not being applied.
• Fix documentation error in mosquitto.conf.
• Ensure that QoS 2 queued messages are sent out in a timely manner.
• Local bridges now act on clean_session correctly.
• Local bridges with clean_session==false now remove unused subscriptions on broker restart.
• The $SYS/broker/heap/# messages now no longer include “bytes” as part of the string for ease of use.

Client library

• Free memory used by OpenSSL in mosquitto_lib_cleanup() where possible.
• Change WebSocket subprotocol name to mqttv3.1 to make future changes easier and for compatibility with other implementations.
• mosquitto_loop_read() and mosquitto_loop_write() now handle errors themselves rather than having mosquitto_loop() handle their errors. This makes using them in a separate event loop more straightforward.
• Add mosquitto_loop_forever() / loop_forever() function call to make simple clients easier.
• Disable SSL compression. This reduces memory usage significantly and removes the possibility of CRIME type attacks.
• Enable SSL_MODE_RELEASE_BUFFERS mode to reduce SSL memory usage further.
• mosquitto_tls_set() will now return an error or raise an exception immediately if the CA certificate or client certificate/key cannot be accessed.
• Fix potential memory leaks on connection failures.
• Don’t produce return error from mosquitto_loop() if a system call is interrupted. This prevents disconnects/reconnects in threaded mode and simplifies non-threaded client handling.
• Ignore SIGPIPE to prevent unnecessary client quits in threaded mode.
• Fix document error for mosquitto_message_retry_set().
• Fix mosquitto_topic_matches_sub() for subscriptions with + as the final character. Fixes bug #1085797.
• Rename all “obj” parameters to “userdata” for consistency with other libraries.
• Reset errno before network read/write to ensure EAGAIN isn’t mistakenly returned.
• The message queue length is now tracked and used to determine the maximum number of packets to process at once. This removes the need for the max_packets parameter which is now unused.
• Fix incorrect error value in Python error_string() function. Fixes bug #1086777.
• Reset last message in/out timer in Python module when we send a PINGREQ. Fixes too-early disconnects.

Clients

• Clients now display their own version number and library version number in their help messages.
• Fix “mosquitto_pub -l -q 2″ disconnecting before all messages were transmitted.
• Fix potential out-of-bounds array access with client ids. Fixes bug #1083182.

Other

• mosquitto_passwd can now convert password files with plain text files to hashed versions.

This is a bugfix release.

Changes

Broker

• Fix loading of psk files.
• Don’t return an error when reloading config if an ACL file isn’t defined.  This was preventing psk files being reloaded.
• Clarify meaning of $SYS/broker/clients/total in mosquitto(8) man page.
• Clarify meaning of $SYS/broker/messages/stored in mosquitto(8) man page.
• Fix non-retained message delivery when subscribing to #.
• Fix retained message delivery for subs to foo/# with retained messages at foo.
• Include the filename in password/acl file loading errors.

Library

• Fix possible AttributeError when self._sock == None in Python module.
• Fix reconnecting after a timeout in Python module.
• Fix reconnecting when there were outgoing packets in the queue in the Python module.
• Fix problem with mutex initialisation causing crashes on some Windows installations.

Downloads

Source is available on the download page, the binary packages for Windows are available now and Linux builds will be available as soon as the various build servers complete their tasks.

Mosquitto 1.0 introduced the use of password files with hashed passwords but had no way to convert from the old plain text password files. This feature will be available in version 1.1 but if it is important to you then you can already get the updated code for the mosquitto_passwd utility at https://bitbucket.org/oojah/mosquitto/src/3b8ef11cf687

Version 0.16 of the mosquitto client libraries will have some binary incompatible changes to their APIs. This means that it is a good time to make other changes that are incompatible. If you think any part of the interface (see http://mosquitto.org/api/ ) is crazy or could be improved in any way, please get in touch or add a comment below.

This is a feature and bugfix release.

• Implement “once” and “lazy” bridge start types.
• Add support for $SYS/broker/clients/maximum and $SYS/broker/clients/active topics.
• Add support for $SYS messages/byte per second received/sent topics.
• Updated mosquitto man page – $SYS hierarchy and signal support were out of date.
• Auto generated pub/sub client ids now include the hostname.
• Tool for dumping persistent DB contents is available in src/db_dump. It isn’t installed by default.
• Enforce topic length checks in client library.
• Add new return type MOSQ_ERR_ERRNO to indicate that the errno variable should be checked for the real error code.
• Add support for connection_messages config option.
• mosquitto_sub will now refuse to run if the -c option (disable clean session) is given and no client id is provided.
• mosquitto_pub now gives more useful error messages on invalid input or other error conditions.
• Fix Python will_set() true/True typo.
• Fix messages to topic “a/b” incorrectly matching on a subscription “a” if another subscription “a/#” exists.

The web is currently buzzing with the announcement yesterday that IBM and Eurotech are donating MQTT to the Eclipse Foundation. One part of this is the new Machine to Machine Working Group, again part of Eclipse. Another much more significant part has been released as part of a new Eclipse open source project “Paho“, which is in the proposal stage.

The exciting part is the “Initial contributions” section which states “The initial code contribution to Paho will include  Java and C client-side implementations the MQTT protocol, contributed by IBM. ”

It looks like the code will be licensed under the EPL (Eclipse Public License). This is particularly exciting because there is currently no solid freely available and usable implementation of MQTT in Java.

Well done everyone at IBM for making this happen. Roll on the end of November!

Update:

This post on mqtt.org explains what the various announcements mean more clearly: http://mqtt.org/2011/11/eclipse-paho-open-source-and-other-news

Update 2:

Andy Piper’s blog post covers things even better, along with clearing up some of the confusions from the news releases: http://andypiper.co.uk/2011/11/04/mqtt-goes-free-a-personal-qa/

Something else that has happened recently is the announcement by Facebook that they’re using MQTT in their new Facebook Messenger app. They’ve posted some details in a facebook-engineering blogpost and cite the low bandwidth and battery usage as important considerations.

This is very exciting as an application that is potentially huge and very user oriented (rather than “internet of things” oriented), but the really exciting bit is if you use an iPhone under Settings and Licenses (apparently it’s quite hard to find):

Thanks to Michael Rowe for getting me the screenshot and Andy Piper for pestering Michael on my behalf.

You should note that if you’re in the UK, the Facebook Messenger app isn’t currently available.