Streamsheets now open source

The Mosquitto project is sponsored by Cedalo. This week, Cedalo open sourced its core project, Streamsheets, which has Mosquitto as a key part. Streamsheets are real-time web based spreadsheets that can take input from a variety of data sources, process the data, and provide an output.

They are designed to allow non-programmers to build applications in IoT, industrial, or hobbiest environments, reading data from machines, sensors, APIs, and more.

Connectivity includes MQTT, OPC-UA, REST, file, and others.

If you need a front end for Mosquitto or other communication protocol, it would be worth looking at.

streamsheets screenshot

Some examples:

  • Create dynamic charts and dashboards to visualize your processes
  • Build real-time apps for condition monitoring and alerting
  • Merge data streams from different sources to calculate KPIs and enrich your data
  • Map protocols and change data structures to connect your resources
  • Model simple as well as complex control logic to orchestrate your processes

Download: https://cedalo.com/en/download/

Github: https://github.com/cedalo/streamsheets

Forum: https://forum.streamsheets.com/


Version 1.6.7 released.

Mosquitto 1.6.7 has been released, this is a bugfix release.

Broker

  • Add workaround for working with libwebsockets 3.2.0.
  • Fix potential crash when reloading config. Closes #1424, #1425.

Client library

  • Don't use / in autogenerated client ids, to avoid confusing with topics.
  • Fix mosquitto_max_inflight_messages_set() and mosquitto_int_option(..., MOSQ_OPT_*_MAX, ...) behaviour. Closes #1417.
  • Fix regression on use of mosquitto_connect_async() not working. Closes #1415 and #1422.

Clients

  • mosquitto_sub: Fix -E incorrectly not working unless -d was also specified. [Closes #1418].
  • Updated documentation around automatic client ids.

Version 1.6.5 released

This is a bugfix release.

Compatibility

  • The most recent version of libwebsockets (3.2.0) changed its behaviour and is not compatible with Mosquitto. This has been fixed for the next libwebsockets release. The 1.6.5 release refuses to compile with libwebsockets 3.2.0. All previous versions of Mosquitto that use websockets are affected by the change in behaviour.

Broker

  • Fix v5 DISCONNECT packets with remaining length == 2 being treated as a protocol error. Closes #1367.
  • Fix support for libwebsockets 3.x (excluding 3.2.0)
  • Fix slow websockets performance when sending large messages. Closes #1390.
  • Fix bridges potentially not connecting on Windows. Closes #478.
  • Fix clients authorised using use_identity_as_username or use_subject_as_username being disconnected on SIGHUP. Closes #1402.
  • Improve error messages in some situations when clients disconnect. Reduces the number of "Socket error on client X, disconnecting" messages.
  • Fix Will for v5 clients not being sent if will delay interval was greater than the session expiry interval. Closes #1401.
  • Fix CRL file not being reloaded on HUP. Closes #35.
  • Fix repeated "Error in poll" messages on Windows when only websockets listeners are defined. Closes #1391.

Client library

  • Fix reconnect backoff for the situation where connections are dropped rather than refused. Closes #737.
  • Fix missing locks on mosq->state. Closes #1374.

Documentation

  • Improve details on global/per listener options in the mosquitto.conf man page. Closes #274.
  • Clarify behaviour when clients exceed the message_size_limit. Closes #448.
  • Improve documentation for max_inflight_bytes, max_inflight_messages, and max_queued_messages.

Build

  • Fix missing function warnings on NetBSD.
  • Fix WITH_STATIC_LIBRARIES using CMake on Windows. Closes #1369.
  • Guard ssize_t definition on Windows. Closes #522.

Version 1.6.4 released

This is a bugfix release.

Broker

  • Fix persistent clients being incorrectly expired on Raspberry Pis. Closes #1272.
  • Windows: Allow other applications access to the log file when running. Closes #515.
  • Fix incoming QoS 2 messages being blocked when max_inflight_messages was set to 1. Closes #1332.
  • Fix incoming messages not being removed for a client if the topic being published to does not have any subscribers. Closes #1322.

Client library

  • Fix MQTT v5 subscription options being incorrectly set for MQTT v3 subscriptions. Closes #1353.
  • Make behaviour of mosquitto_connect_async() consistent with mosquitto_connect() when connecting to a non-existent server. Closes #1345.
  • mosquitto_string_option(mosq, MOSQ_OPT_TLS_KEYFORM, ...) was incorrectly returning MOSQ_ERR_INVAL with valid input. This has been fixed. Closes #1360.
  • on_connect callback is now called with the correct v5 reason code if a v5 client connects to a v3.x broker and is sent a CONNACK with the "unacceptable protocol version" connack reason code.
  • Fix memory leak when setting v5 properties in mosquitto_connect_v5().
  • Fix properties not being sent on QoS>0 PUBLISH messages.

Clients

  • mosquitto_pub: fix error codes not being returned when mosquitto_pub exits. Closes #1354.
  • All clients: improve error messages when connecting to a v3.x broker when in v5 mode. Closes #1344.

Other

  • Various documentation fixes.

Version 1.6.3 released

This is a bugfix release.

Broker

  • Fix detection of incoming v3.1/v3.1.1 bridges. Closes #1263.
  • Fix default max_topic_alias listener config not being copied to the in-use listener when compiled without TLS support.
  • Fix random number generation if compiling using WITH_TLS=no and on Linux with glibc >= 2.25. Without this fix, no random numbers would be generated for e.g. on broker client id generation, and so clients connecting expecting this feature would be unable to connect.
  • Fix compilation problem related to getrandom() on non-glibc systems.
  • Fix Will message for a persistent client incorrectly being sent when the client reconnects after a clean disconnect. Closes #1273.
  • Fix Will message for a persistent client not being sent on disconnect. Closes #1273.
  • Improve documentation around the upgrading of persistence files. Closes #1276.
  • Add 'extern "C"' on mosquitto_broker.h and mosquitto_plugin.h for C++ plugin writing. Closes #1290.
  • Fix persistent Websockets clients not receiving messages after they reconnect, having sent DISCONNECT on a previous session. Closes #1227.
  • Disable TLS renegotiation. Client initiated renegotiation is considered to be a potential attack vector against servers. Closes #1257.
  • Fix incorrect shared subscription topic '$shared'.
  • Fix zero length client ids being rejected for MQTT v5 clients with clean start set to true.
  • Fix MQTT v5 overlapping subscription behaviour. Clients now receive message from all matching subscriptions rather than the first one encountered, which ensures the maximum QoS requirement is met.
  • Fix incoming/outgoing quota problems for QoS>0.
  • Remove obsolete store_clean_interval from documentation.

Client library

  • Fix typo causing build error on Windows when building without TLS support. Closes #1264.

Clients

  • Fix -L url parsing when /topic part is missing.
  • Stop some error messages being printed even when --quiet was used. Closes #1284.
  • Fix mosquitto_pub exiting with error code 0 when an error occurred. Closes #1285.
  • Fix mosquitto_pub not using the -c option. Closes #1273.
  • Fix MQTT v5 clients not being able to specify a password without a username. Closes #1274.
  • Fix mosquitto_pub -l not handling network failures. Closes #1152.
  • Fix mosquitto_pub -l not handling zero length input. Closes #1302.
  • Fix double free on exit in mosquitto_pub. Closes #1280.

Documentation:

  • Remove references to Python binding and C++ wrapper in libmosquitto man page. Closes #1266.

Build

  • CLIENT_LDFLAGS now uses LDFLAGS. Closes #1294.

Version 1.6.2 released

This is a security and bugfix release.

Security

If a client connects using MQTT v5, will a Will message that has MQTT v5 properties attached, and the very first Will property is one of content-type, correlation-data, payload-format-indicator, or response-topic, then at the point the client disconnects, the broker will attempt to read from freed memory, resulting in a possible crash.

Broker

  • Fix memory access after free, leading to possible crash, when v5 client with Will message disconnects, where the Will message has as its first property one of content-type, correlation-data, payload-format-indicator, or response-topic. Closes #1244.
  • Fix build for WITH_TLS=no. Closes #1250.
  • Fix Will message not allowing user-property properties.
  • Fix broker originated messages (e.g. $SYS/broker/version) not being published when check_retain_source set to true. Closes #1245.
  • Fix $SYS/broker/version being incorrectly expired after 60 seconds. Closes #1245.

Library

  • Fix crash after client has been unable to connect to a broker. This occurs when the client is exiting and is part of the final library cleanup routine. Closes #1246.

Clients

  • Fix -L url parsing. Closes [#1248].

Version 1.6.1 released

This is a minor service release. The fixes are only related to documentation and the build process, and so is primarily of interest for people building Mosquitto.

Broker

  • Document memory_limit option.

Clients

  • Fix compilation on non glibc systems due to missing sys/time.h header.

Build:

  • Add make check target and document testing procedure. Closes #1230.
  • Document bundled dependencies and how to disable. Closes #1231.
  • Split CFLAGS and CPPFLAGS, and LDFLAGS and LDADD/LIBADD.
  • test/unit now respects CPPFLAGS and LDFLAGS. Closes #1232.
  • Don't call ldconfig in CMake scripts. Closes #1048.
  • Use CMAKE_INSTALL_* variables when installing in CMake. Closes #1049.

Version 1.6 released

This is a feature release and represents a substantial amount of change in the project. Since version 1.5, the overall code line count for the broker, library and clients has increased by 37% to 28k. Testing has been an important focus for this release. The number of tests has increased from 102 to 412. The test coverage, whilst still needing further improvement, has increased from 56% to 61%.

A summary of the notable features is given below.

MQTT v5 support

The big addition for this release is support for MQTT v5. This covers the broker, client library and client, and gives full support for the new specification, although not all features are accessible as they will be.

You can quickly test out a v5 client by using -V 5 and adding properties with the -D option, for example:

mosquitto_sub -V 5 -D connect receive-maximum 3 -D subscribe subscription-identifier 1 ...

The authentication plugin interface has been extended to allow use of the v5 extended authentication feature.

Performance improvements

A number of performance improvements have been implemented in the broker, including the message routing logic, topic matching, and persistence file reading/writing.

More improvements are planned for the next release.

New client - mosquitto_rr

mosquitto_rr is the "request-response" client, intended for the situation where you want to publish a request message and await a response. It works best with the MQTT v5 request-response features, but can be used with v3.1.1 servers if the client it is talking to knows how to respond. This tool is almost certainly not going to see as much use as mosquitto_sub or mosquitto_pub, but is a useful utility to have available.

Contributed features

Some notable features have been contributed by the community.

On the TLS front, support for ALPN allows bridges and clients to connect to servers that have multiple protocols on a single port. The new OCSP stapling support allows the status of TLS certificates to be validated. Finally, TLS Engine support has been added.

Away from TLS, support for Automotive DLT logging has been added, disabled by default.

Deprecations

The C++ wrapper library, libmosquittopp is now deprecated and will be removed in version 2.0. It remains largely unchanged since v1.5.

The C library, libmosquitto, is having its interface changed for version 2.0, so any current function should be considered at risk. The rationale for this is to consolidate the changes introduced since version 1.0, in particular the large number of functions required to support MQTT v5, but that otherwise closely match existing functions.

Support for TLS v1.0 has been dropped. Support for TLS v1.1 will be dropped in version 2.0.

Changelog

The more detailed changelog is below, but does not include many of the fixes and improvements that have been made:

Broker features

  • Add support for MQTT v5
  • Add support for OCSP stapling.
  • Add support for ALPN on bridge TLS connections. Closes #924.
  • Add support for Automotive DLT logging.
  • Add TLS Engine support.
  • Persistence file read/write performance improvements.
  • General performance improvements.
  • Add max_keepalive option, to allow a maximum keepalive value to be set for MQTT v5 clients only.
  • Add bind_interface option which allows a listener to be bound to a specific network interface, in a similar fashion to the bind_address option. Linux only.
  • Add improved bridge restart interval based on Decorrelated Jitter.
  • Add dhparamfile option, to allow DH parameters to be loaded for Ephemeral DH support
  • Disallow writing to $ topics where appropriate.
  • Add explicit support for TLS v1.3.
  • Drop support for TLS v1.0.
  • Improved general support for broker generated client ids. Removed libuuid dependency.
  • auto_id_prefix now defaults to 'auto-'.
  • QoS 1 and 2 flow control improvements.

Client library features

  • Add support for MQTT v5
  • Add mosquitto_subscribe_multiple() for sending subscriptions to multiple topics in one command.
  • Add TLS Engine support.
  • Add explicit support for TLS v1.3.
  • Drop support for TLS v1.0.
  • QoS 1 and 2 flow control improvements.

Client features

  • Add support for MQTT v5
  • Add mosquitto_rr client, which can be used for "request-response" messaging, by sending a request message and awaiting a response.
  • Add TLS Engine support.
  • Add support for ALPN on TLS connections. Closes #924.
  • Add -D option for all clients to specify MQTT v5 properties.
  • Add -E to mosquitto_sub, which causes it to exit immediately after having its subscriptions acknowledged. Use with -c to create a durable client session without requiring a message to be received.
  • Add --remove-retained to mosquitto_sub, which can be used to clear retained messages on a broker.
  • Add --repeat and --repeat-delay to mosquitto_pub, which can be used to repeat single message publishes at a regular interval.
  • -V now accepts 5, 311, 31, as well as mqttv5 etc.
  • Add explicit support for TLS v1.3.
  • Drop support for TLS v1.0.

Broker fixes

  • Improve error reporting when creating listeners.
  • Fix mosquitto_passwd crashing on corrupt password file. Closes [#1207].
  • Fix build on SmartOS due to missing IPV6_V6ONLY. Closes #1212.

Client library fixes

  • Add missing mosquitto_userdata() function.

Client fixes

  • mosquitto_pub wouldn't always publish all messages when using -l and QoS>0. This has been fixed.
  • mosquitto_sub was incorrectly encoding special characters when using %j output format. Closes #1220.

Version 1.5.8 released

This is a bugfix release.

Broker

  • Fix clients being disconnected when ACLs are in use. This only affects the case where a client connects using a username, and the anonymous ACL list is defined but specific user ACLs are not defined. Closes #1162.
  • Make error messages for missing config file clearer.
  • Fix some Coverity Scan reported errors that could occur when the broker was already failing to start.
  • Fix broken mosquitto_passwd on FreeBSD. Closes #1032.
  • Fix delayed bridge local subscriptions causing missing messages. Closes #1174.

Library

  • Use higher resolution timer for random initialisation of client id generation. Closes #1177.
  • Fix some Coverity Scan reported errors that could occur when the library was already quitting.

MQTT 5 Test Release

The work on MQTT v5 support in Mosquitto has reached a point where it may be of interest to a range of people. It is not yet complete, but wider testing and feedback would be appreciated.

The source is available at mosquitto-mqtt5-test1.tar.gz and can be compiled as normal.

Supported features

  • Session expiry
  • Message expiry
  • Reason code on all ACKs (not all reason codes are used)
  • Reason string on all ACKs (no reason strings are provided by the broker however)
  • Payload format and content type
  • Request / response pattern
  • Subscription ID
  • Topic alias
  • Flow control
  • User properties
  • Optional server feature availability
  • Subscription options
  • Server keep alive
  • Assigned Client ID

Unsupported features

  • Shared subscriptions
  • Extended authentication
  • Server reference
  • Not all reason codes are used by the broker
  • Bridges do not use MQTT v5
  • On disk persistence does not include MQTT 5 property support
  • The broker will not create topic aliases