Bureau of Meteorology weather data: Difference between revisions

From Sensors in Schools
Jump to navigation Jump to search
 
(4 intermediate revisions by the same user not shown)
Line 16: Line 16:


== Obtaining BOM rain forecast data ==
== Obtaining BOM rain forecast data ==
* Teh Bureau of Meteorology provides accurate weather data that we can download to our computer using an  
* This lesson will demonstrate how to extract data from the Bureau of Meterology using an API (Application Programming Interface).


=== Find Bureau of Meteorology API for daily weather forecast data ===
=== Find Bureau of Meteorology API for daily weather forecast data ===
* Open a Web Browser.
* Open a Web Browser.
* Enter the search term weather.bom.gov.au
* Enter the search term '''weather.bom.gov.au'''
 
[[File:Screen Shot 2022-01-01 at 8.39.18 am.png]]
 
* Select '''Melbourne VIC''' as the weather location.
 
[[File:Screen Shot 2022-01-01 at 8.40.09 am.png]]
 
* Ensure that '''Development Tools''' is selected under '''Preferences''' for your Browser settings.
* Under the '''Develop''' menu select '''Connect Web Inspector'''.
* This will open up an addition window to show some of the inner-workings of the web page.
 
[[File:Screen Shot 2022-01-01 at 8.41.27 am.png]]
 
* In the Development Tools window click on the '''Network Tab'''.
* '''Refresh''' the web browser.
* In the window on the left search for a file named '''daily'''
* Daily holds data related to current and future weather forecasts including '''7 day rain predictions'''.
* You can scroll in the '''Previous''' window to see all the retrieved data presented in JSON format.
 
[[File:Screen Shot 2022-01-01 at 8.43.00 am.png]]
 
* If the mouse hovers over daily the '''link to the API''' will appear.
* In this instance it is - '''https://api.weather.bom.gov.au/v1/locations/r1r0fs/forecasts/daily'''
* You can copy this link and enter it into a Web Browser to test that it works.
 
[[File:Screen Shot 2022-01-01 at 8.48.01 am.png]]
 
=== Retrieve BOM data using Node-Red HTTP-In node ===
* Create a flow in Node-Red using the nodes:
** '''Inject''' – timestamp
** '''http request''' – this will request data using the daily API
** '''debug''' – to display the output of the API in the debug window
 
[[File:Screen Shot 2022-01-01 at 8.51.44 am.png]]
 
* Double click on the '''HTTP request''' node and enter the following details.
* To copy the API request from the Web Browser use the '''right mouse button Copy''' function.
* Paste the API into notepad or a text editor to ensure that it has copied correctly.
* You may need to paste the API into a text editor on the Raspberry Pi.
* Failing this, enter the API directly.
* Ensure that the '''Return is a parsed JSON Object'''
* Give the '''Name - BOM daily'''
 
[[File:Screen Shot 2022-01-01 at 8.53.24 am.png]]
 
* '''Deploy''' the program then click on the input to the '''inject node''' to run the program.
* The output should appear in the '''debug''' window on the right.
* Click on the small expansion '''arrows''' to show more of the data content.
 
[[File:Screen Shot 2022-01-01 at 8.54.41 am.png]]
 
=== Collecting Day 0 rain data ===
* Add a function node and debug node and arrange the nodes as shown below.
 
[[File:Screen Shot 2022-01-01 at 8.55.37 am.png]]
 
* Double click on the function node and enter the following code.
* The code will extract '''day 0 (zero) max rainfall''' and place it in the variable rain.
* '''Name''' the function node '''Day 0 (zero) rain'''
 
[[File:Screen Shot 2022-01-01 at 8.56.42 am.png]]
 
* '''Deploy the code'''.
* Clear the Debug window and then click on the inject node to run the program.
* Your output will be different because it will be based on the daily weather forecast.
* This result shows that today ('''day 0''') there will be a maximum of '''2.2 mm of rain'''.
 
[[File:Screen Shot 2022-01-01 at 8.58.00 am.png]]
 
=== Collecting Day 1 rain data ===
* Copy the '''Day 0 rain''' function node by entering the shortcut '''Ctrl+C'''
* Enter '''Ctrl-V''' to paste a copy of the node.
* Then '''double click''' on the new function node to edit it.
 
[[File:Screen Shot 2022-01-01 at 8.59.24 am.png]]
 
* Change the '''Name''' property to '''Day 1 rain'''
* Modify the code to the example shown below.
* Notice that some '''descriptive text''' will be added to the msg.payload
 
[[File:Screen Shot 2022-01-01 at 9.13.17 am.png]]
 
* Add another '''Debug''' node and '''connect the nodes''' as shown.
* Some rearrangement of the nodes will help make the wires and flow clearer.
 
[[File:Screen Shot 2022-01-01 at 9.14.12 am.png]]
 
* Deploy the flow.
* Clear the Debug window. Click on the inject node to start the program.
* Two outputs are presented.
* One for '''Day 0''' rain and the second for '''Day 1''' rain.
 
[[File:Screen Shot 2022-01-01 at 9.15.22 am.png]]
 
=== Downloading all the rain data using a for loop ===
* Modify the node-red flow to the following.
 
[[File:Screen Shot 2022-01-01 at 9.16.11 am.png]]
 
* Double click on the function node.
* Change the Name to 7 day rain
* Enter the following code:
** '''rainmax = []''' – create a single variable called an array that will hold all the rain data. Think of it as rainmax[0] for the first data item, rainmax[1] for the second data item, etc
** '''for(let i=0; i<8; i++) {}''' – this for loop initilises a counting variable i to zero (0). It tests to see if the counting variable is less than (<) 8. Each time the for loop goes around the counting variable i is increased by one (i++)
** '''rainmax[i] = msg.payload.data[i]''' – in this statement think of the counting variable i being replaced with 0, 1,2, etc with every loop.
** '''msg.payload = [{}] '''– the msg.payload needs to have data formatted correctly before it can be delivered to the chart node.
 
[[File:Screen Shot 2022-01-01 at 9.18.00 am.png]]
 
* Double click on the '''Chart node'''.
* From the '''Group''' select Add '''new ui_group'''
* Then click on the '''Edit pen''' on the right.
 
[[File:Screen Shot 2022-01-01 at 9.19.09 am.png]]
 
* In the Edit chart node > '''Add new dashboard group config node'''
* In '''Tab''' select '''Add new ui_tab'''
* Then click on the '''Edit pen'''
 
[[File:Screen Shot 2022-01-01 at 9.20.14 am.png]]
 
* In the '''Add new dashboard tab config node'''
* Enter the Name '''BOM rain''' and click '''Add'''
 
[[File:Screen Shot 2022-01-01 at 9.21.09 am.png]]
 
* In the Add new dashboard group config node.
* Ensure the Name and Tab are '''BOM rain''' and click '''Add'''
 
[[File:Screen Shot 2022-01-01 at 9.45.42 am.png]]
 
* Back in the parent '''Edit chart''' node window.
* Change from Line chart to '''Bar chart'''.
 
[[File:Screen Shot 2022-01-01 at 9.46.43 am.png]]
 
* In the '''Edit chart node''' window change the Y-axis to '''min 0 and max 100'''
 
[[File:Screen Shot 2022-01-01 at 9.47.31 am.png]]
 
* In the main editing window in Node-Red click on '''Deploy'''.
* Click on the '''Inject node''' to generate data for the Chart.
* In the web browser open a new tab and enter the URL '''localhost:1880/ui'''
* Navigate to '''BOM rain'''
* You should see the following chart.
 
[[File:Screen Shot 2022-01-01 at 9.48.45 am.png]]
 
=== Final improvement ===
* Sometimes the BOM daily data set returns an array that is '''7 or 8 elements long''', depending on the time of day.
* If the for loop asks to elements that don’t exist our program will generate an error.
* One way to fix this is to find out how many elements are in the array using the built-in function '''length'''
 
[[File:Screen Shot 2022-01-01 at 9.50.05 am.png]]

Latest revision as of 22:50, 31 December 2021

Overview

  • The aim of this lesson is to learn how to retrieve weather data from the Bureau of Meteorology (BOM) using an API (Application Programming Interface).
  • An API is simply a URL that will automatically request data from a server.
  • In this example we will extract rain data from the daily weather forecast.
  • The daily weather forecast includes rain forecast data up to 7 days in advance.
  • We will use this data for our smart rainwater tank.
  • Smart rainwater tanks need to know rain events up to 7 days in advance so that they can discharge tank water before a storm arrives.

Learning Objectives

  • Learn how to use the Developer tools in a Web Browser to find an API
  • Learn how to use an API with HTTP Request node in Node-Red
  • Learn how to extract data from a larger data sets using the function node
  • Learn how to prepare data for a chart node

Obtaining BOM rain forecast data

  • This lesson will demonstrate how to extract data from the Bureau of Meterology using an API (Application Programming Interface).

Find Bureau of Meteorology API for daily weather forecast data

  • Open a Web Browser.
  • Enter the search term weather.bom.gov.au

  • Select Melbourne VIC as the weather location.

  • Ensure that Development Tools is selected under Preferences for your Browser settings.
  • Under the Develop menu select Connect Web Inspector.
  • This will open up an addition window to show some of the inner-workings of the web page.

  • In the Development Tools window click on the Network Tab.
  • Refresh the web browser.
  • In the window on the left search for a file named daily
  • Daily holds data related to current and future weather forecasts including 7 day rain predictions.
  • You can scroll in the Previous window to see all the retrieved data presented in JSON format.

Retrieve BOM data using Node-Red HTTP-In node

  • Create a flow in Node-Red using the nodes:
    • Inject – timestamp
    • http request – this will request data using the daily API
    • debug – to display the output of the API in the debug window

  • Double click on the HTTP request node and enter the following details.
  • To copy the API request from the Web Browser use the right mouse button Copy function.
  • Paste the API into notepad or a text editor to ensure that it has copied correctly.
  • You may need to paste the API into a text editor on the Raspberry Pi.
  • Failing this, enter the API directly.
  • Ensure that the Return is a parsed JSON Object
  • Give the Name - BOM daily

  • Deploy the program then click on the input to the inject node to run the program.
  • The output should appear in the debug window on the right.
  • Click on the small expansion arrows to show more of the data content.

Collecting Day 0 rain data

  • Add a function node and debug node and arrange the nodes as shown below.

  • Double click on the function node and enter the following code.
  • The code will extract day 0 (zero) max rainfall and place it in the variable rain.
  • Name the function node Day 0 (zero) rain

  • Deploy the code.
  • Clear the Debug window and then click on the inject node to run the program.
  • Your output will be different because it will be based on the daily weather forecast.
  • This result shows that today (day 0) there will be a maximum of 2.2 mm of rain.

Collecting Day 1 rain data

  • Copy the Day 0 rain function node by entering the shortcut Ctrl+C
  • Enter Ctrl-V to paste a copy of the node.
  • Then double click on the new function node to edit it.

  • Change the Name property to Day 1 rain
  • Modify the code to the example shown below.
  • Notice that some descriptive text will be added to the msg.payload

  • Add another Debug node and connect the nodes as shown.
  • Some rearrangement of the nodes will help make the wires and flow clearer.

  • Deploy the flow.
  • Clear the Debug window. Click on the inject node to start the program.
  • Two outputs are presented.
  • One for Day 0 rain and the second for Day 1 rain.

Downloading all the rain data using a for loop

  • Modify the node-red flow to the following.

  • Double click on the function node.
  • Change the Name to 7 day rain
  • Enter the following code:
    • rainmax = [] – create a single variable called an array that will hold all the rain data. Think of it as rainmax[0] for the first data item, rainmax[1] for the second data item, etc
    • for(let i=0; i<8; i++) {} – this for loop initilises a counting variable i to zero (0). It tests to see if the counting variable is less than (<) 8. Each time the for loop goes around the counting variable i is increased by one (i++)
    • rainmax[i] = msg.payload.data[i] – in this statement think of the counting variable i being replaced with 0, 1,2, etc with every loop.
    • msg.payload = [{}] – the msg.payload needs to have data formatted correctly before it can be delivered to the chart node.

  • Double click on the Chart node.
  • From the Group select Add new ui_group
  • Then click on the Edit pen on the right.

  • In the Edit chart node > Add new dashboard group config node
  • In Tab select Add new ui_tab
  • Then click on the Edit pen

  • In the Add new dashboard tab config node
  • Enter the Name BOM rain and click Add

  • In the Add new dashboard group config node.
  • Ensure the Name and Tab are BOM rain and click Add

  • Back in the parent Edit chart node window.
  • Change from Line chart to Bar chart.

  • In the Edit chart node window change the Y-axis to min 0 and max 100

  • In the main editing window in Node-Red click on Deploy.
  • Click on the Inject node to generate data for the Chart.
  • In the web browser open a new tab and enter the URL localhost:1880/ui
  • Navigate to BOM rain
  • You should see the following chart.

Final improvement

  • Sometimes the BOM daily data set returns an array that is 7 or 8 elements long, depending on the time of day.
  • If the for loop asks to elements that don’t exist our program will generate an error.
  • One way to fix this is to find out how many elements are in the array using the built-in function length

Retrieved from "http://www.waterwaysinwhittlesea.org/index.php?title=Bureau_of_Meteorology_weather_data&oldid=952"