Pachube (now Cosm) has recently announced beta support for publishing and receiving data to their service using MQTT. This is great news and something I know that a lot of people have been hoping for. Well done Pachube!
Their documentation is at https://xively.com/dev/docs/api/communicating/mqtts/ and provides enough information to get going if you’re already familiar with MQTT.
If you aren’t familiar with MQTT, here’s a few examples of how you can use the new service.
First off, I’m going to use the command line MQTT clients I’ve created to publish and receive data. You can get these clients as part of the mosquitto download.
mosquitto_pub -h api.xively.com -u <your xively api-key> -t /v2/feeds/504.csv -m "0,29"
In this example we’re connecting to host api.xively.com, using our xively api-key as the username, publishing to feed /v2/feeds/504 using the csv format and are updating datastream 0 with the value 29. Another way to achieve the same thing would be to do:
mosquitto_pub -h api.xively.com -u <your xively api-key> -t /v2/feeds/504/datastreams/0.csv -m 29
mosquitto_pub can read data from stdin and publish it, so on Unix type systems the following arrangement is possible:
sensor_read | mosquitto_pub -h api.xively.com -u <api-key> -t /v2/feeds/504/datastreams/0.csv -l
The “-l” option reads messages from stdin, sending a separate message for each line. This means that our imaginary executable sensor_read that is reading data from a sensor must be printing each reading as a text line.
In the MQTT world, retrieving data is done through subscriptions:
mosquitto_sub -h api.xively.com -u <api-key> -t /v2/feeds/504/datastreams/0.csv
In this example, mosquitto_sub will print a text line containing the csv data for datastream 0 of feed 504 every time it is updated.
The last will and testament or just “will” is a very nice feature of MQTT. When your client connects to the MQTT broker/server, it can give the broker this will, which consists of a topic and a message. If the client is disconnected from the broker unexpectedly, that is to say without sending a disconnect message, then the broker publishes the will message on the will topic.
This provides a very simple mechanism for client connection monitoring. When your client connects it could publish a message “1” to a topic. If it also set a will to send a message “0” to the same topic on unexpected disconnect, then it would be possible to determine whether that client was connected by monitoring the topic.
In the context of Xively, the same approach is possible, but using a trigger to indicate that the client had disconnected.
The mosquitto_sub client provides support for wills as shown in the example below:
mosquitto_sub -h api.xively.com -u <api-key> -t /v2/feeds/504/datastreams/0.csv --will-topic /v2/feeds/12345/datastreams/0.csv -will-payload "0"
In this example, the Xively broker would publish the value “0” to datastream 0 of feed 12345 if mosquitto_sub disconnects unexpectedly. This isn’t the most useful example because of the limitations of what mosquitto_sub provides.
In practice, to get the full benefit of the advantages that MQTT provides you will probably want to write your own MQTT client to connect to Xively for your specific application. The mqtt.org software page lists client implementations for lots of different programming languages including the mosquitto client libraries in C/C++, libraries in Java, Python and also device specific implementations for Arduino and other low power devices.
The Xively offering is a slightly restricted MQTT offering. “Full” MQTT offers a bit more scope for doing fun things using topic wildcards for example, something that wouldn’t really make sense for Xively.
If you’d like to play on an MQTT broker, try looking at http://test.mosquitto.org/.
If you want some help there are mailing lists and irc channels listed on http://mqtt.org/get-involved
I saw this in the nanode irc channel:
I’ve never seen any real world projects with MQTT… it looks good though.
So I’m looking for real world projects that use MQTT. If you’ve got a project it’d be great if you could mention it in the comments. A short sentence on what it does and how many clients you run on it – really anything you can say. If it’s a secret please still comment if you can, just be very very vague. If you’ve got a blog post describing it, link that instead. I’m interested in everything from a single temperature sensor reporting to a computer up to millions of mobile users.
The arduino, the open source microcontroller board, has had MQTT support for a long time in the form of Nick O’Leary’s arduino client. It does however require networking support which has traditionally provided by an add on shield, which increases the cost of the system.
The nanode is an arduino compatible board which includes network support and can be built for approximately the same cost as a normal arduino board. It’s still a work in progress, but is definitely worth a look if you want to use low power MQTT capable sensors/controllers.
Mark Hindess has written a blog post titled Home Automation Protocols: MQTT, where he asks for suggestions on how to go forward making “a specification for topic usage and semantics”. I think this kind of work is really valuable to make it easy to have different MQTT systems that can interoperate. If you’ve got any suggestions you can make, please go and leave a comment there.
This is a follow up to my previous post on using Google Powermeter, but this time I’m going to give a step by step guide to getting your data uploaded. The only assumptions are that you have a CurrentCost monitor (note that CurrentCost monitors are often rebadged by electricity suppliers such as EON in the UK so check yours) and have already connected it to your computer, want to use MQTT and that you’re using Linux, or another Unix operating system.
In the Bang Goes the Theory episode “The Human Power Station”, a family of four people had their electricity supplied by a large group of cyclists on cycles hooked up to generators, the point being to highlight the amount of energy that is used and wasted on a daily basis. There’s a video at http://www.youtube.com/watch?v=C93cL_zDVIM.
One example that seemed to waste a lot of energy was cooking roast dinner. The oven was turned on to preheat (this is often the first instruction in a recipe), but it wasn’t actually used until a significant time later, wasting a lot of energy. An example which may be more common is turning the oven on to preheat before cooking frozen food (which requires no preparation), then forgetting to check to see if it has preheated.
This used to happen to me, but I’ve solved the problem by using my electricity monitor (I have an electric oven), mqtt and asterisk.
A new wiki has been created, devoted to MQTT. If you want to share what you’re doing with MQTT and how to do it, or want to find any of that out, head over to http://mqtt.org/wiki.
If you want to use MQTT for push in Android apps, you’ll probably want to head over to Anton L’s blog post How to Implement Push Notifications for Android. He has a sample Android app that uses the IBM Java library to implement push notifications using MQTT, as well as a web page solution to demo pushing notifications to your phone.
A popular use for mqtt brokers seems to be coupling them with a CurrentCost (or similar) energy monitor to then log energy data and produce pretty (and useful!) graphs.
Google recently opened up their PowerMeter API which looks to provide very nice graphing of energy data. They are working with utility companies directly with in home monitors, but it’s also possible to use it as an individual.
Toby Evans got to the bottom of registering a device (see his explanatory blog post) which just leaves getting data to Google.
If you’re already logging energy data to an MQTT broker, it’s as simple as adding another subscriber to send the data to Google. You could use the mosquitto_sub client and a script I wrote for posting to google, google_powermeter_update_mqtt.pl like so:
mosquitto_sub -t sensors/cc128/ch1 | google_powermeter_update_mqtt.pl
This assumes that the data appearing on the sensors/cc128/ch1 topic is in the format “<unix timestamp> <power reading in Watts>”.
If you’re not logging your energy data to a broker, you should probably consider doing so :) There is another script google_powermeter_update.pl which may be more suitable and can be used as:
google_powermeter_update.pl <unix timestamp> <power in Watts>
Both of the scripts need your user details adding and should be easy to modify to match your own particular need. They also assume you’re using a single cumulative variable with your device and will need modifying if you’re using more than one variable or aren’t using cumulative variables.
For reference, I use the script cc128.pl to read data from my CurrentCost CC128 (Envi).
Google has a limit of 6 API requests per hour, so the above description will only work for a short while (the 6 request limit doesn’t appear to be a hard limit when you first exceed it, but becomes increasingly stricter). I’m now logging my CC128 data to a mysql database and sending batch updates every 15 minutes with google_powermeter_update_mysql.pl