Reading data from an atmospheric sensor

Overview

• In this lesson we are going to write code that will allow us to download data from an atmospheric sensor.
• The sensor will give us data on temperature, atmospheric pressure, battery voltage and signal strength.
• We will use the python language to get this data.
• Once we have data we can do other amazing things such as save the data and even graph the data.
• Environmental data is very important because it allows us to monitor the environment we live in more closely.

Learning Objectives

• Learn how to use the requests library in python to get data from a URL
• Learn how to interpret data presented in the JSON format
• Learn how to process data in python so that it can be saved in variables

Requesting new data from a sensor URL in Python

• Sensors are now commonly connected to the internet.
• This has given rise to the term – The Internet of Things.
• If a sensor is connected to the internet then it will often have a URL (Universal Resource Locator) to allow us to interact with the sensor.
• One common interaction is to be able to see or download data collected by the sensor.
• To get data from a sensor we need to import the requests library in python.
• In this lesson we will download some data from an atmospheric sensor.
• The sensor data includes temperature, air pressure, sensor battery voltage and signal strength.
• In this example we write a python3 program to download sensor data using the requests library.
• The data will be presented in the python Shell.

Open IDLE3 Python Editor

• From the Raspberry Pi main menu select Programming > Python3 (IDLE)
• If IDLE is not installed open the Terminal and enter the commands:
  • sudo apt-get update
  • sudo apt-get install idle3

Create a folder named water_level_sensor using the File Manager

• Click on the File Manager
• Select File and New Folder

• Enter the new folder name water_level_sensor
• This is where we will store our files related to the water sensor.

Create a folder named water_level_sensor using the Terminal

• Alternatively, you can make a folder using the Terminal.
• Open the Terminal and enter the command ls to list the contents in the default user home directory (/home/pi/)

• To create a new folder enter the command mkdir water_level_sensor

Create a new file in python named water_level_sensor.py

• Go back to the IDLE3 program
• From the File drop down menu select New File.
• This will open up a new file Editor Window.

• To save the new file, select File > Save As
• Name the file water_level_sensor.py

• Navigate to your working directory and then save the file as atm_sensor_get.py
• In Linux we never use names that include spaces.
• Replace spaces with either the _ (underline) symbol or the – (hyphen) symbol.

Request data from sensor using python

• The API for WHI-ATC01 - Atmospheric temperature and pressure sensor has changed.
• Please use the following APIs as they have recently changed.
  • temperature (channel 1 - degC) - https://app.alphax.cloud/conduitv6?id=70B3D54993D429DF&token=5ecf2e35d99a404a81408792&limit=1&ch=1
  • pressure (channel 3 - kPa) - https://app.alphax.cloud/conduitv6?id=70B3D54993D429DF&token=5ecf2e35d99a404a81408792&limit=1&ch=3
• Enter the following lines of code to request data from the atmospheric sensor.

• The key items in the code are described:
  • import requests – this statement imports a python library so that the python program can get data from the internet.
  • r = requests.get(URL) – this statement gets the data from a specified URL and stores the result in the variable object named r. URL is short for Uniform Resource Location, also known as a web address.
  • print(r) – this statement prints a code that determines if the request was successful or not.
• Save and the Run the program.

• The response in the Python Shell should be [200].
• A value of [200] means that the request was successful.
• Other numbers or errors will indicate that there is a problem either with the Python code or with communication with the sensor.

View sensor data in JSON format

• We can now add some more code so that we can look at the actual data.
• Add the statement print(r.json()) and run the code.
• The method json() means that the data will be presented in JSON format.
• JSON stands for JavaScript Object Notation.
• It is an open standard to help format data for data exchange between computers and it uses human-readable text.
• The data is stored in attribute-value pairs' similar to the format of a dictionary.

• You can see the data output in the python Shell Window.

• Unfortunately, presenting the data this way is not very readable.
• In the next section of this lesson we will learn how to view and interpret data stored in the JSON format.

Interpreting data in json format

• As previously mentioned, JSON is an open standard that allows data to be exchanged more easily between computers.
• JSON data is stored as text, and this makes it human-readable however you need a special application to see this data more clearly.
• In this lesson we will use the web browser Firefox to visualise the data so we can see the arrangement of attribute-value pairs.

Copying, editing and saving JSON formatted data

• One way to see the data more easily is to view it in the web browser Firefox.
• To do this we need to Copy the JSON data output from the Raspberry Pi and Paste it in a file on your PC.
• We also need to replace all ‘ characters with “ otherwise Firefox wont show the JSON data correctly.
• Copy the data above from the Python Shell.
• To copy, right-mouse button click and select Copy.
• Paste it into a text editor on your PC.
• To paste, right-mouse button click and select Paste.
• Replace all ‘ characters with “ using the in-build Find/Replace function in your Text Editor.
• Then Save the data as a file with the file extension .json on your PC.
• In this example the file name is atm_data.json

Download and install Firefox web browser

• Rather than Firefox, you can now use https://jsonviewer.com/
• On your PC, download and install the Firefox Browser.
• You can use the following link: https://www.mozilla.org/en-US/firefox/new/

Visualising JSON formatted data in Firefox

• In the Firefox Browser Open the new file you created named atm_data.json
• You can do this by selecting File > Open File... from the top Drop Down Menu.
• You should see something like the following displayed. See image below.
• Some key points:
  • Sensor data is divided into four sections labelled [0, 1, 2, 3]
  • Each section is for different data set:
    • Temperature
    • Air Pressure
    • Battery Voltage
    • Signal Strength (RSSI)

• The attribute-value pair we are interested in (for each data set) is the attribute named “val_calibrated”.
• For each data set [0, 1, 2, 3] the “val-calibrated” attribute will match with either [temperature, pressure, battery voltage, signal strength]
• For example, for data set [0], the “val_calibrated” value is 9.04 – which is the Temperature.

Learning how to extract Temperature data from JSON formatted data

• To extract the relevant data from our JSON data we enter the following code print(r.json[0][“val_calibrated”])
  • [0] – represents the first data set (containing Temperature data)
  • [“val_calibrated”] – is the attribute in the attribute-value pair that pairs with the temperature value
  • print() – will print out the temperature data to the Python Shell

• The result can be seen on the last line of the Python Shell.
• In this case the temperature is 5.61 (degrees Centigrade).

Assigning temperature data into a variable

• Now that we know how to find and extract temperature data we can also pass on this value to a variable.
• Putting the temperature data in a variable allows us to save or manipulate the data more easily, just like in algebra.
• Add the following code:
  • temp = r.json()[0][“val_calibrated”] – assign temperature data to variable named temp
  • print(temp) – display the value held in temp in the Python Shell
• Save and then Run this program.

• The output of the program is shown below in the Python Shell window.
• The very last line of output (5.61) shows the output of the print(temp) statement.

• The output of the program has a lot of information we no longer need.
• To reduce the clutter from the output we can comment out some of the statements in our program using the # (hash) symbol.
• Note that when we comment out statement the text will change to a red colour.
• If we comment out statements the computer will no longer execute (run) them.
• Follow the example below and comment out three of the print() statements.
• Save and Run the program.

• You can see that the output of the program has been simplified.
• We can always uncomment these print() statements in the future.

Collecting more data from the atmospheric sensor

• The atmospheric sensor collects other data in addition to temperature.
• The key data we are interested in is:
  • temperature – measured in degrees Celsius
  • air pressure – measured in kilopascals. 101kPa normal. 105kPa high pressure (fine weather).
  • battery voltage – charge level of battery powering the sensor. Max voltage 4.20V. Min voltage 3.60V
  • signal strength (RSSI) – a record of how powerful this signal is at the receiving station (LoRa Gateway)

• In this next example, we will capture data on the air pressure and the battery voltage.

Extracting Air Pressure from JSON formatted data

• If we refer to the data presented in the Firefox web browser, we can see the data is divided into sections.
• Temperature data was in a section prefaced by [0].
• Air Pressure data is a section prefaced by [1].
• Using this information, we can extract the Air Pressure data using the following code.
  • print(r.json()[1][“val_calibrated”])
• Add this additional statement, then Save and Run the program.

• This is the output in the python Shell Window.
• Both Temperature (18.78) and Air Pressure (101.05) data are now displayed.

• As for the temperature data, we can store the Air Pressure data in a variable.
• For Air Pressure we have created a new variable named pres
• Enter the following code, and also comment out the preceding print() statement.
  • pres = r.json()[1][“val_calibrated”]
  • print(pres)

Extracting Battery Voltage from JSON formatted data

• In this last example we will also request Battery Voltage.
• Add the following code to extract the Battery Voltage data:
  • print(r.json()[2][“val_calibrated”]
• Save and Run the program.

• The output in the python Shell Window will now show all three data values including Battery Voltage.
• If the battery voltage drops below 3.70 Volts, then the battery will need to be recharged.

• As a final step we will store the Battery Voltage data in a variable named voltage.
• Enter the following code and remember to comment out the preceding print() statement.
  • voltage = r.json()[2][“val_calibrated”]
  • print(voltage)
• Save and Run the program.

• Congratulations on completing this lesson.
• In the next lesson we will learn to save all this data in a file and also add a date/time stamp so that we have a record of when the data was collected.