Exploring soil moisture sensors

Overview

• In this lesson we are going to connect to a soil moisture sensor.
• The soil moisture sensor is connected to an Arduino MKR 1300 microcontroller (a small computer).
• The data will be transmitted to the LoRa network.
• LoRa is short for Long Range radio communication.
• The data is then sent to The Things Network, which is a community run server that collects data from sensors and makes the data available for download.
• We will download the data to our Raspberry Pi, and in future lessons display that data in a Dashboard using Node-RED.
• All these skills are very important in helping to address Climate Change and general Environmental Sustainability issues the world faces.

Learning Objectives

• Learn how to connect to The Things Network using MQTT using Node-Red
• Learn how to extract specific data from JSON formatted data packets

Connecting to the Things network using Node-RED

• In this lesson we are going to use Node-RED as our main programming tool.
• Node-RED is a block or node based programming language.
• Each block has a discrete function and some blocks can contain Javascript code or be configured in different ways.
• Blocks are connected together using wires.
• Node-RED and the programs on the Raspberry Pi are always being updated, so it’s normally good practise to update our Raspberry Pi on a regular basis.

Updating and Upgrading the Raspberry Pi

• Open the Raspberry Pi Terminal.
• Enter the command sudo apt-get update

• This command will update all the software libraries on the Raspberry Pi, so that if we need to install software the Pi will install the latest software packages.
• Enter the command sudo apt-get full-upgrade -y
• This command will upgrade the entire operating system on the Raspberry Pi.
• The -y option tells the computer to answer Yes to any questions.
• The upgrade will take between 5 and 30 minutes to complete.
• It is always to do this once every month.

Running Node-RED on the Raspberry Pi

• Open the Terminal.
• To start Node-RED enter the command node-red

• The display on your Raspberry Pi will be different, but similar enough.
• When using Node-RED it is important to stop Node-RED when you are shutting down your computer.
• To close Node-RED open a new Terminal window and enter the command node-red-stop
• To re-start Node-RED enter either node-red or node-red-start

• It takes Node-RED approximately 20 seconds to boot up.
• To see the Node-RED interface we need to use a Web Browser.
• Open the default Web Browser on the Raspberry Pi.
• In the Search Bar enter localhost:1880 (or http://localhost:1880)
• The number 1880 represents a Port. Your computer listens and talks on different ports to make communication between devices and systems easier.
• Localhost is an abbreviation for 127.0.0.1 which is a address reserved for the host computer (the Raspberry Pi)

• Node-RED can be installed on Windows, Mac or other Linux machines.
• In a few moments the Node-RED programming environment will be displayed.
• One the left is the Node-RED Pallet containing an assortment of Nodes.
• In the middle is the Node-RED Flow Editor
• On the right are different debugging and system related tools.

Connecting to The Things Network using MQTT

• To download the data generated by our sensor we can use MQTT.
• MQTT is short for Message Queuing Telemetry Transport.
• Using this protocol we can subscribe to newly published data. As soon as new data arrived it is delivered to our device, just like a daily news feed or other subscription.
• To subscribe to our sensor data we need to use a mqtt in node
• Search for the mqtt in node in the pallet on the left and drag it into the Editor Window.

• If the mqtt in node is difficult to find you can enter mqtt in the search bar at the top of the pallet.
• In this example we can see that two results are returned, one for mqtt in and one for mqtt out.

• Note that the mqtt in node has two coloured shapes attached to the node:
  • the red triangle means that the node still needs to be configured
  • the blue circle means that the node need to be Deployed before it will work
• To configure the node, double click on it.
• This will open a Sidebar that will pop out from the right hand side.
• For the Server parameter, we will select Add new mqtt-broker and then click on the Edit tool, which looks like a pencil icon.

• The Edit tool will open up another Sidebar listing the Properties for the mqtt-broker node.
• Enter the following details under the Connection tab:
  • Name – soil-7
  • Server – au1.cloud.thethings.network
  • Port – 1883
  • Protocol – MQTT V3.1.1 (if you don’t see the Protocol parameter you don’t need to change any other settings)
• Now click on the Security tab.

• Enter the following details in the Security tab:
  • Username – whi-soil-7-moisture-mkr-1300@ttn (the username is simply a unique name given to the soil moisture sensor on The Things Network. The name is short for Whittlesea soil moisture sensor #7 running on an Arduino MKR 1300 microcontroller)
  • Password - NNSXS.WL5EA6KF6TWUZCALAIIEWNX7YED4ADI5QQOTK7A.XYRLCUHY6WGY5Z5O6TS7A7UFAACBNY72WIZPAITY6K3LTYLOVZXA (Enter this passwork exactly as shown. This key is generated on The Things Network Console. We will demonstrate this in an upcoming lesson)
• Click on Update to commit these changes.

• Once the Server has been configured there are some other minor configurations in the mqtt in node:
  • Topic - # (the hash symbol # indicates that we are subscribing to all topics)
  • QoS (Quality of Service) – 0 (only zero is supported by Node-RED. Normally higher numbers reflect better error detection)
  • Output – a parsed JSON object (all the data sent from the mqtt in node will be in JSON format. JSON format is similar to a dictionary. There is always an attribute and value pair. For example {colour:blue}. JSON objects are enclosed in curly brackets).
  • Name – soil-7 (this node will be subscribing to data from soil moisture sensor #7)
• Click on Done when finished.

• The mqtt-in node has the name soil-7 displayed.
• Note that the red triangle has disappeared because the node is now correctly configured.
• There is still a blue circle because we haven’t Deployed our code.
• Click on the Red Deploy button to start the program.

• If the mqtt in node is working correctly it should display a connected message supported with a green square.
• Note that if the program is running the Deploy button will be shaded out.
• As soon as you make any change to your program the Deploy button will be active again.
• Remember that while editing your program, the old program will still be running in the background on the Node-RED server.

5. Viewing MQTT data using the Node-RED Debug Node

• The mqtt in node has a small connecter on the right-hand side of the node.
• This allows us to connect the output data flow to other nodes using wires.
• From the Node-RED pallet find the debug node and drag into into the Editor Window.
• Then using a mouse connect the mqtt in node to the debug node, as shown below.

• To activate the program click on the Deploy button.
• To see the output of the debug node, click on the debug tab in the far-right window. It is the button that looks like an insect with legs.
• Once selected the word debug should appear in the window.

• You may receive an error message when the Deploy button is clicked.
• Just ignore this so long as the mqtt in node is still showing the connected status.

• The soil moisture sensor data sends data every 10 minutes.
• So after a few minutes you should see your first JSON object payload come through.

• By clicking on the drop down arrows you can expand or collapse different sections of the JSON object.
• In a section of the JSON object named decoded_payload there is data from the sensor:
  • bat – battery voltage data
  • humid – humidity data from within the sensor housing
  • soil – soil moisture sensor data (from under my fig tree)
  • temp – temperature data from within the sensor housing
• Note that the full path is msg.payload.uplink_message.decoded_payload.bat (.bat for battery data, .humid for humidity data, etc)
• Using this path we can start to extract specific data from the JSON object message.

7. Extracting data using the Function node

• Now that we are receiving all the JSON formatted data we can start to extract specific data sets.
• In this example we will extract the battery data.
• Find the function node in the pallet and drag it into the Editor window.
• Also add a debug node and connect them up as shown.

• Double click on the function node. We need to add some code so that the battery data can be extracted.
• Enter the following code:
  • var bat
  • bat=msg.payload.upload_message.decoded_payload.bat (puts the battery data in the variable bat)
  • msg.payload = bat (this then assigns the bat data to the msg.payload. Messages (msg) also contact other attributes such as msg.topic. Here we are only interested in the msg.payload attribute.
  • return msg (this sends the msg to the connecting node to the right. In this case it is the debug node which will display the output in the Debug window on the far right.

• Note – if you can’t see the underscore symbol (_) it may be just a bug.
• The underscore symbol is still there, just invisible.
• Another work around is to create the underscore symbol in a text editor and use the right-mouse button copy and paste function.
• Click on Done when finished.
• Deploy the code.

• To make the Debug window easier to view you can click on the Rubbish bin icon to clear the Debug window.
• Now you just need to wait for a new mqtt message to arrive.

• When the new message arrives, you will see two JSON outputs in the Debug Window.
  • One shows the output from the Debug node connected directly to the mqtt in node (soil-7)
  • The second will show the output from the Debug node connected to the output of the function node (battery). In this case we can see the numerical output 563, which is the battery voltage measurement.

• Congratulations, now you know how to extract JSON data using Node-RED.
• In our next lesson we will extract some other useful data and also create a Dashboard using Node-RED.