Node-RED data processing II

Overview

• This lesson will demonstrate how Node-Red can be used to process data from a sensor.
• In this lesson we will learn how to process the output from a python file using the function node in Node-RED.

Learning Objectives

• Learn how to run a Python program using the Terminal.
• Learn how to run a Python program using the Execute node in Node-RED.
• Learn how to extract data using the Function node in Node-RED.

learning to process data using Node-RED

• Node-RED makes it very easy to display data using a Dashboard, but first we need to get the data we need.
• Luckily, we have experience retrieving data from sensors using Python.
• Reading and processing data using Node-RED is not very different.

Open the file peter_hopper_get_data5.py

• From the main Raspberry Pi menu navigate to Programming and open Python3 IDLE.
• Navigate to the /home/pi/peter-hopper-t3 directory. (t3 is short for term 3).
• Open up the file peter_hopper_get_data5.py and review to make sure it is the same as the code below.

• Run the file and check that it can retrieve sensor data and send the standard output to the Python Shell.

Running python program from the Terminal

• Python programs can also be run from the Terminal.
• Open a new Terminal window.
• Navigate to the peter-hopper-t3 directory with either of these commands.
  • cd /home/pi/peter-hopper-t3 – Absolute path reference includes the full pathname
  • cd ~/peter-hopper-t3 – Absolute path reference but uses the ~ (tilde) symbol in place of /home/pi
  • cd peter-hopper-t3 - Relative path reference – assumes you are in /home/pi

• To run the peter_hopper_get_data5.py file enter the command python3 peter_hopper_get_data5.py
• The output will be like the output from the Python Shell.
• The output is a String.

Executing a Python program in Node-RED

• Start Node-RED in the Terminal with the command Node-RED
• Open a web Browser and enter the URL localhost:1880
• In Node-RED arrange the following nodes in the arrangement shown below.
  • inject node – displayed as a timestamp
  • exec node – execute node can run commands such as python3 peter_hopper_get_data5.py
  • debug node – displaying as msg.payload
  • json node – to convert String outputs to a JSON object

• Double click on the exec (execute) node and enter the following parameters:
• Command: Two commands on the one line will be executed:
  • cd /home/pi/peter-hopper-t3 – navigate to the directory peter-hopper-t3 using Absolute path notation
  • && - the AND Operator. The AND Operator will execute the second command only if the preceding command succeeds.
  • python3 peter_hopper_get_data5.py – second command
• Note – Do not Append the msg.payload. Append should be de-selected (as shown).

• Click on the Deploy button.
• Ensure that the Debug Window (on the right) is selected and click on the Rubbish bin icon to clear Debug messages.
• Run the program by clicking on the button to the right of the inject (timestamp) node.

• There will be two outputs sent in the Debug window (from the Debug nodes)
  • Top msg.payload – output appears in red. Output is a String.
  • Bottom msg.payload – Output appears in purple/red. Output is a JSON Object

• You can expand each of the outputs in the Debug window by clicking on the small triangle in front of the data.

Processing JSON data using the Function Node

• Add a function node to the flow as shown below.

• Double click on the function node and enter the following code:
  • var bat – creates an empty variable named bat.
  • bat = msg.payload.battery –the battery voltage data is retrieved from msg.payload.battery. The battery voltage is then assigned to variable bat.
  • msg.payload = bat – we repackage the payload
  • return msg – send the payload to the next node

• Click on Deploy.
• Clear the Debug window.
• Run the program by clicking on the timestamp button.
• The top debug node will output the unprocessed msg.payload as a String
• The bottom debug node will output the raw battery data (4.1025) – after processing through the function node.
• Formatting the data using JSON has made it easier for the function node to extract the relevant data.

• Double click on the function node and make the following changes so that we can add some more human readable text to the output.
  • msg.payload = “The battery voltage is “ + bat;

• Deploy and run the program.
• Now the output from the bottom debug node will be easier to understand.

Saving Node-RED flows

• Get into the habit of always Saving and Backing up your work.
• Click on the hamburger (three horizontal lines).
• Click on Export.

• In the Export nodes window click on:
  • current flow – the single flow window your are working on
  • JSON – so that all the node configuration data is exported in JSON format
  • formatted – at the bottom of the window, so that the JSON code is formatted more neatly

• If your Node-RED looks different to the one displayed here execute the following commands in new Terminal Window at the end of this lesson to upgrade your Pi.
  • sudo apt-get update
  • sudo apt-get full-upgrade -y

• When you save your flow, it will saved to the Downloads folder.
• In this instance the flow was saved as flows(9).json
• Open the File Manager right mouse button click on the file and select Rename File.
• Add more descriptors to the file name and a date. Then click OK.
• You can now insert a USB memory stick into the Raspberry Pi computer and copy your file to it.