BirdNET-Pi to Monitor Bird Calls
Jump to navigation
Jump to search
BirdNET-Pi Lesson
| Lesson number | Step by step instructions | Dropbox Video link | Teacher resources |
|---|---|---|---|
| Lesson 2a | #The Importance of Using BirdNET-Pi to Monitor Bird Calls | 
|
|
| Lesson 2b | #BirdNET-Pi microSD Card Imaging | 
|
GitHub BirdNET-Pi Installation Guide |
| Lesson 2c | #BirdNET-Pi Software Installation | 
|
BirdNET-Pi Software Installation using SSH |
| Lesson 2d | #BirdNET-Pi Finalising the Installation | 
|
BirdNET-Pi Finalising the Installation |
| Lesson 2e | #Settings Configuration in BirdNET-Pi Web Portal |
Authors
For more information contact Adam Simankowicz or Edmond Lascaris
Overview
- BirdNET-Pi uses a USB microphone connected to a Raspberry Pi to record and identify bird calls.
- The system runs 24/7 and can upload data in real time to https://app.birdweather.com
- Data can also be downloaded via an API so that you can do some additional data processing using Node-RED.
Hardware requirements
- Raspberry Pi 4 or Pi 3 B+.
- USB Microphone - (e.g. Gyvazla USB Microphone Lavalier Clip-on Omnidirectional Condenser Microphone for Computer) [1]
- Heat sink and fan for Raspberry Pi. Optional, but highly recommended because it will keep the processor temperature less than 40degC to prevent throttling and prolong the life of your Raspberry Pi.
Learning Objectives
- Learn about the important role of vegetation in supporting small birds
- Learn how to install BirdNET-Pi on a Raspberry Pi 4
- Learn how to interpret data from a local installation of BirdNET-Pi
- Learn how to use an API to retrieve data from the BirdNET-Pi installed in Bundoora
Small Birds
- Small birds need protection from thick scrubby vegetation so they can build nests and also hide from larger birds.
- Examples of small birds include:
- White-browed Scrubwrens
- Brown Thornbills
- Spotted Pardalotes
- Willy Wagtails
- Yellow Robins
- Golden Whistlers
- Eastern Spinebills
- Grey Fantails
- There is also a documentary you can watch on ABC iView called The Secret Lives of our Urban Birds. Start at the 41 minute mark to see the section on Noisy Minors [2]
- Friends of Darebin Creek - Small Woodland Birds [3]
BirdNET-Pi microSD Card Imaging
- Download and install the Raspberry Pi Imaging software.
- The software is available from https://www.raspberrypi.com/software
- Insert a microSD card in the computer with the Raspberry Pi Imaging software.
- The size of the microSD card needs to be 16 GB.
- Open the Raspberry Pi Imager and click CHOOSE OS to select the operating system for BirdNET-Pi.
- Click Raspberry Pi OS (other) Other Raspberry Pi OS based images
- Scroll down the list and select "Raspberry Pi OS Lite (64-bit)"
- Next, click "CHOOSE STORAGE" to select the SD card the BirdNET-Pi will use
- Choose the SD card onto which we will write the RaspiOS-Lite (64-bit) operating system
- Now, click the gear icon in the bottom right to open the "Advanced options" menu
- Answer Yes - to fill in WiFi details
- Set the hostname you would like this BirdNET-Pi to use to be reached.
- For example, to have multiple BirdNET-Pis on the same network, you might want to give them each a different hostname.
- In this example, the installation will be reachable at "http://birdnetpi1.local", since that is the hostname that is set during this step.
- You should also "Enable SSH" and select to "Use password authentication"
- When you scroll down, you will set the option to "Set username and password."
- This example creates a new user called pi.
- You will also configure the system's connection to your WiFi by entering the network's name (SSID) and the password used to connect to that WiFi.
- Adjust the locale settings for your Time zone and Keyboard layout.
- Save the settings
- Click "WRITE" to write the image to the SD card
- Select "YES" and enter to allow your host system (the computer you're on now) to write the SD card.
- Wait while the image is written to the card and the checksums verified
- You may need to give permission for your computer to make changes to the SD card.
- Enter your password (if requested) and click OK.
- The writing and verification process takes approx 5-10 minutes.
- When the SD card is ready, you'll be notified that is ok to remove it from your computer
- At this point, you will put the SD card into the Raspberry Pi and will allow it a few minutes to boot up.
- After a few minutes, you can move on to the next step.
BirdNET-Pi Software Installation
- At this point, you will put the SD card into the Raspberry Pi and will allow it a few minutes to boot up.
- After a few minutes, you can move on to the next step.
- Using a terminal emulator of your choosing (macOS and Linux users can use the terminals that come with the OS -- Windows users can use PowerShell or PuTTY).
- Use the username and hostname from step 8 to SSH into the Raspberry Pi.
- In this example, we created the birder user on the birdnetpi1.local host, so the command would be ssh pi@birdnetpi1.local
- Alternatively, find the local IP address of the birdNET-Pi using the command ifconfig
- Then use this IP address in the SSH statement from the remote computer. For example, ssh pi@192.168.1.147
- Then enter the password for the pi user on the BirdNET-Pi.
- When warned that you are connecting to a new machine, type yes to proceed
- You will be prompted for the password that you set for user pi
- After entering the password for your user, you will be able to begin the installation of the BirdNET-Pi software.
- Copy and paste the command below into the terminal and press "Enter"
- Note: Ctrl+Shift+V will often paste in Terminals -- in PuTTY, right-click to paste
curl -s https://raw.githubusercontent.com/mcguirepr89/BirdNET-Pi/main/newinstaller.sh | bash
- The installation process will take approximately 5 to 10 minutes.
- When the installation is finished, the system will automatically reboot. When it is booted back up,
you will be able to reach your BirdNET-Pi using the hostname you set in step 8. In this installation example, it is http://birdnetpi1.local. The default (if you did not set a specific hostname in step 8) will be http://birdnetpi.local. It can also be reached at the Pi's IP address.
BirdNET-Pi Finalising the Installation
- Open a Browser on a computer on the local network.
- Enter the following address in the search bar http://birdnetpi1.local
- BirdNET-Pi Dashboard will open up.
Settings Configuration in BirdNET-Pi Web Portal
- The BirdNET-Pi web portal has additional configuration options.
- Click on the Tools dropdown menu.
- A Log into birdnetpi1.local:80 window will appear.
- The username is birdnet
- There is no password.
- Click on Log in
- A new series of administration buttons will appear under the main menu.
- Click on Settings
- Wait a few seconds for the Settings page to load.
- BirdNET-Pi needs to know the latitude and longitude location up to 4 decimal places.
- Use Google Maps to pinpoint the location of the BirdNET-Pi sensor.
- Enter the GPS coordinates to 4 decimal places.
- BirdNET-Pi will keep your data localised unless you upload data to BirdWeather.com.
- If you would like to share data contact Tim (tim@birdweather.com) to request a BirdWeather ID.
- Sending data to BirdWeather.com will is important for this project because we can then access the data using an API (more on this later).
- Update the Time and Date settings.
- To commit (save) all setting click on the Update Settings button.
- To check that the Settings have been updated successfully wait for the Settings page to reload.
- Then scroll down to the bottom of the page.
- A Success message should be displayed.
Advanced Settings
- At the bottom of the Basic Settings page there is a tab to access Advanced Settings
- Click on the Advanced Settings tab
- Scroll down to the BirdNET-Pi section.
- Enter a secure password.
- This password will protect the Tools page.
- The username for the Tools page is birdnet.
- At the bottom of the Advanced Setting page don't forget to Save Changes.
- If changes are saved successfully the Save button will change to Success
Swap file expansion
- Swap files have not been implemented on the SMC Raspberry Pi 4
Expanding the swap file on the Raspberry Pi
- On a Raspberry Pi (or any other computer system), a swap file is a type of virtual memory file that acts as an extension to the system's physical RAM (Random Access Memory). When the available physical RAM is insufficient to handle the running processes and their associated data, the swap file allows the system to offload some of the least-used data from RAM onto the disk.
- Here's how it works:
- Memory Management: The operating system manages the allocation of memory for running processes. It divides the memory into pages or blocks that can be loaded into RAM when needed.
- Insufficient RAM: When the available RAM becomes limited due to the number of running processes or the memory requirements of specific applications, the system may start moving less frequently accessed or idle pages from RAM to the swap file on the disk.
- Virtual Memory: The swap file essentially provides a portion of the disk space that is used as virtual memory. It allows the system to temporarily store data that is not actively being used, freeing up physical RAM for other tasks.
- Page Swapping: As the system needs to free up RAM, it selects the least-used pages from RAM and swaps them out to the swap file. This process is called "page swapping" or "paging."
- Retrieval from Swap: If a process requires data that was swapped out to the swap file, the operating system will swap it back into RAM, making space for other pages to be swapped out if necessary.
- It's important to note that while swap files provide additional memory space, they are slower to access compared to physical RAM. Disk read and write operations are slower than accessing data directly from RAM, leading to potential performance degradation. Therefore, having excessive swapping, also known as "thrashing," can significantly impact system responsiveness.
- On a Raspberry Pi or any other system, enabling a swap file can be useful in situations where there is limited physical RAM and the system is experiencing memory pressure. However, it's generally recommended to have enough physical RAM to accommodate the expected workload, as excessive swapping can lead to a noticeable decrease in performance.
Tools - System Info
- To access System Info click on Tools then System Info
System Info - Mounted File Systems
- BirdNET-Pi 1
- BirdNET-Pi 2 (SMC)
System Info - Memory Usage
- BirdNET-Pi 1
- BirdNET-Pi 2 (SMC)
System Vitals and Hardware Information
- BirdNET-Pi 1
- BirdNET-Pi 2 (SMC)
Network Usage and Temperature
- BirdNET-Pi 1 - Friday 9 June 2023
- BirdNET-Pi 1 - Sunday 12 June 2023
- BirdNET-Pi 1 - Thursday 3 August 2023
- BirdNET-Pi 2 (SMC) - Friday 9 June 2023
- BirdNET-Pi 2 (SMC) - Monday 12 June 2023
Viewer
- BirdNET-Pi 1
- BirdNET-Pi 2 (SMC)
Viewer - Cron
- BirdNET-Pi 1
- BirdNET-Pi 2 (SMC)
WiFi Antenna Extenders
- BirdNet-Pi (Raspberry Pi) may end up in a location that has poor WiFi receiption.
- External WiFi antennas can be plugged into the USB port on the Raspberry Pi that will boost reception.
USB WiFi Module
- A small Mini USB WiFi Module - RTL8188eu - 802.11b/g/n can be added to the Raspberry Pi (Core Electronics).
- This is suitable for indoor spaces where Wifi reception is poor.
- Often, a metal case enclosing the Raspberry Pi will reduce WiFi signal strength. Metal cases are ofter used to enclose a Raspberry Pi to help remove excess heat.
Wifi High Gain Antenna
- A larger High Gain external antenna may be required especially if the Raspberry Pi is located outside a house or in a garden shed.
- This antenna will significantly extend the useful range of the existing WiFi network.
- Panda Wireless® PAU0D AC1200 Wireless AC USB Adapter with High Gain Antennas - Windows XP/Vista/7/8/10/11, Mint, Ubuntu, openSUSE, Fedora, Centos, Kali Linux and Raspbian (Amazon)
BirdNET-Pi Maintenance
- Restart BirdNET-Pi Raspberry Pi once per week.
- Monitor temperature of Raspberry Pi to check to see if it has been throttled due to exceeding 60degC.
- Check to see if the Micro SSD card is full.
- Complete any updates.
- Accessing BirdNET-Pi computer using Secure Shell (ssh)
- Accessing BirdNET-Pi specific hostname http://birdnetpi1.local/
BirdNET-Pi Monitoring
- It may be necessary to establish a regular monitoring program for the Raspberry Pi to monitor disk usage.
- In the example below, the Raspberry Pi running BirdNET-Pi was found to be unresponsive.
- Even after rebooting the system could not be fully recovered (disconnecting power and powering up again).
- The system could be accessed using secure shell using the command ssh pi@192.168.1.133
- Using the command df -h the hard drive was found to be full.
CPU temperature monitoring using vcgencmd
- To monitor the Raspberry Pi system two python libraries were installed.
- vcgencmd is a command line utility that can get various pieces of information from the VideoCore GPU on the Raspberry Pi such as temperature and determine if the processor has been throttled due to over temperature.
- It is installed with the command sudo pip3 install vcgencmd [4]
- Test to see if vcgencmd is installed with the command vcgencmd measure_temp
- Dweet can be used to send data from the Raspberry Pi every 15 minutes.
- To use Dweet the requests python library needs to be installed.
- Enter the command python3 -m pip install requests
- The bird_monitor.py python file was created in the /home/pi directory.
- The python code is reproduced below.
- A typical output is presented below.
- If the python script /home/pi/bird_monitor.py is now working, then rebooting the BirdNET-Pi computer may fix the error.
import os
import requests
from datetime import datetime
from vcgencmd import Vcgencmd
now = datetime.now()
date_stamp = now.strftime("%Y-%m-%d %H:%M:%S")
#print(now)
df_output_lines = [s.split() for s in os.popen("df -Ph").read().splitlines()]
disk_space = str(df_output_lines[1][4])
disk_space = disk_space[:-1] # remove % symbol at end of String
#print(disk_space)
vcgm = Vcgencmd()
temp = vcgm.measure_temp()
throttled = vcgm.get_throttled()
#print(throttled)
dweet_dict = {}
dweet_dict.update({"latest_update": date_stamp})
dweet_dict.update({"disk_space": disk_space})
dweet_dict.update({"temp": temp})
dweet_dict.update({"throttled": throttled})
url = "https://dweet.io/dweet/for/birdNET-pi-monitor-add-your-dweet-name?"
x = requests.post(url, json=dweet_dict)
print(f"x data {x.text}")
# check dweet with - https://dweet.io/get/latest/dweet/for/birdNET-pi-monitor-add-your-dweet-name
Scheduling tasks using crontab
- Cron can be used to schedule monitoring tasks.
- BirdNET-Pi requires disk space and processor temperature to be monitored using the bird_monitor.py file.
- Cron can be edited in the Terminal with the command crontab -e
- Cron tasks can be viewed with the command ctrontab -l
- In the listing below the bird_monitor.py file is run at 5, 20, 35 and 50 minute intervals every hour.
Managing Disk Space on the BirdNET-Pi computer
- To keep disk space free on the BirdNET-Pi computer the clear_all_data.sh script is scheduled to run every day at 4:05am
- The update_birdnet.sh is also scheduled daily at 4:10am.
- BirdNET-Pi parameters are monitored every 15 minutes and stored in a text file on a separate computer.
- The screen shot of the text file shows the following data:
- disk utilisation (%)
- CPU temperature (degC)
- CPU throttled (0x0 - indicates no throttling)
BirdNET-Pi in-built Scripts
- BirdNET-Pi has a several in-built scripts that are used to help administer BirdNET-Pi.
- clear_all_data.sh clears all bird recordings from the computer and frees up disk space.
- disk_usage.sh provides a human readable output for free disk space on the computer.
- update_birdnet.sh updates the BirdNET-Pi software.
- These same scripts are executed from the BirdNET-Pi web portal dashboard.
- Open a Browser on a computer on the local network.
- Enter the following address in the search bar http://birdnetpi1.local
- This will bring up the BirdNET-Pi web portal.
- Click on the Tools tab.
- Then click on the System Controls tab.
- Clicking on Update executes the update_birdnet.sh script.
- Clicking on Clear ALL Data executes the clear_all_data.sh script.
- The output of the command update_birdnet.sh can be seen here.
Running Shell Script cleanup.sh Daily
- If there are issues processing data during the day this can lead to a reduction in storage space on the BirdNET-Pi.
- Running the shell scrip cleanup.sh daily can help.
- If the in-built cleanup.sh script doesn't work then the Pi can also be rebooted daily or when memory is low.
- These scripts can be run using cron
- However after running this script for several days the solid state MicroSD card started to fill up to 90%.
Daily reboot
- In this trial the BirdNET-Pi was rebooted each day at 5 minutes after midnight to see if this could reduce space on the micro SD card.
- However even reboot did not seem to clear the microSD memory card.
- The best approach seems to be to clear_all_data.sh once per week using cron. An example will be provided.
Running clear_all_data.sh script once per week (Sunday)
- Using crontab the amount of memory usage on the Raspberry Pi microSD card can be managed.
- The following scripts are run.
- cleanup.sh - daily at 5 minutes past midnight
- update_birdnet.sh - daily at 15 minutes past midnight
- clear_all_data.sh - weekly on Sundays at 10 minutes past midnight. This will clear all data and free up space on the microSD card. This script is the most effective. Running weekly is sufficient to prevent the microSD card from becoming too full.
Node-RED code to analyse bird_monitor.py data via Dweet
- Node-RED running on a separate Raspberry Pi computer was used to monitor the disk space and the CPU temperature on the BirdNET-Pi computer.
- The data was made available to another computer using Dweet.
Installing Node-RED
- Node-RED may need to be re-installed or upgraded the first time you use it.
- If the current version of Node.js is version 14 or lower you should upgrade to Node.js version 16.
- Follow the instructions in the link below.
- http://www.waterwaysinwhittlesea.org/index.php/Smart_Cities_-_Node-RED_re-installation_or_version_upgrade
- The Node-RED code is reproduced below.
Node-RED code has been collapsed by default. Click here to expand:
[
{
"id": "99bda892.ebb298",
"type": "tab",
"label": "BirdWeather",
"disabled": false,
"info": ""
},
{
"id": "ec6d79ff.a14128",
"type": "debug",
"z": "99bda892.ebb298",
"name": "",
"active": true,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "payload",
"targetType": "msg",
"statusVal": "",
"statusType": "auto",
"x": 650,
"y": 120,
"wires": []
},
{
"id": "c534dd12.19d5f",
"type": "exec",
"z": "99bda892.ebb298",
"command": "python3 /home/pi/BirdWeather/birdWeather1.py",
"addpay": "",
"append": "",
"useSpawn": "false",
"timer": "",
"winHide": false,
"oldrc": false,
"name": "",
"x": 390,
"y": 200,
"wires": [
[
"ec6d79ff.a14128"
],
[],
[]
]
},
{
"id": "63e49c383d07eb36",
"type": "cronplus",
"z": "99bda892.ebb298",
"name": "23:45 every day",
"outputField": "payload",
"timeZone": "",
"persistDynamic": false,
"commandResponseMsgOutput": "output1",
"outputs": 1,
"options": [
{
"name": "schedule1",
"topic": "topic1",
"payloadType": "default",
"payload": "",
"expressionType": "cron",
"expression": "0 45 23 * * *",
"location": "",
"offset": "0",
"solarType": "all",
"solarEvents": "sunrise,sunset"
}
],
"x": 120,
"y": 60,
"wires": [
[
"c534dd12.19d5f"
]
]
},
{
"id": "85745ab65dbe7307",
"type": "function",
"z": "99bda892.ebb298",
"name": "Save data function",
"func": "var latest_update; \nvar disk_space;\nvar temp;\nvar throttled;\n\n\nlatest_update = msg.payload.with[0].content.latest_update;\ndisk_space = msg.payload.with[0].content.disk_space;\ntemp = msg.payload.with[0].content.temp;\nthrottled = msg.payload.with[0].content.throttled.raw_data;\n\ndatetime = msg.payload.with[0].content.latest_update;\n\nmsg.payload = String(msg.myepoch) + \",\" + String(msg.mydate) + \",\"\n + String(msg.mytimes) + \",\" \n + String(latest_update) + \",\" + String(disk_space) + \",\" \n + String(temp) + \",\" \n + String(throttled) + \",\"\n + String(datetime);\nreturn [msg];",
"outputs": 1,
"noerr": 0,
"initialize": "",
"finalize": "",
"libs": [],
"x": 130,
"y": 460,
"wires": [
[
"ebb8a5963b97cfd6"
]
]
},
{
"id": "9d59914e52e7c4a1",
"type": "http request",
"z": "99bda892.ebb298",
"name": "",
"method": "GET",
"ret": "obj",
"paytoqs": "ignore",
"url": "https://dweet.io/get/latest/dweet/for/birdNET-pi-dweet-project-name",
"tls": "",
"persist": false,
"proxy": "",
"authType": "",
"senderr": false,
"x": 570,
"y": 340,
"wires": [
[
"0eb9fa1b01114ab1",
"85745ab65dbe7307"
]
]
},
{
"id": "922d3407be181bd0",
"type": "debug",
"z": "99bda892.ebb298",
"name": "",
"active": true,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "payload",
"targetType": "msg",
"statusVal": "",
"statusType": "auto",
"x": 590,
"y": 460,
"wires": []
},
{
"id": "900ef8f1cd64965c",
"type": "cronplus",
"z": "99bda892.ebb298",
"name": "10,25,40,55 hourly",
"outputField": "payload",
"timeZone": "",
"persistDynamic": false,
"commandResponseMsgOutput": "output1",
"outputs": 1,
"options": [
{
"name": "schedule1",
"topic": "topic1",
"payloadType": "default",
"payload": "",
"expressionType": "cron",
"expression": "10,25,40,55 * * * * ",
"location": "",
"offset": "0",
"solarType": "all",
"solarEvents": "sunrise,sunset"
}
],
"x": 130,
"y": 340,
"wires": [
[
"195359b5ede1fae4"
]
]
},
{
"id": "195359b5ede1fae4",
"type": "simpletime",
"z": "99bda892.ebb298",
"name": "",
"mydate": true,
"myymd": true,
"myyear": true,
"mymonth": true,
"mymonthn": true,
"mydom": true,
"mydoy": true,
"myday": true,
"myhourpm": true,
"myhour": true,
"mytime": true,
"mytimes": true,
"myminute": true,
"myminutes": true,
"mysecond": true,
"mymillis": true,
"myepoch": true,
"myrawdate": true,
"mypm": true,
"x": 350,
"y": 340,
"wires": [
[
"9d59914e52e7c4a1"
]
]
},
{
"id": "ebb8a5963b97cfd6",
"type": "file",
"z": "99bda892.ebb298",
"name": "bird_monitor-data",
"filename": "/home/pi/BirdWeather/bird_monitor_data.txt",
"appendNewline": true,
"createDir": false,
"overwriteFile": "false",
"encoding": "none",
"x": 370,
"y": 460,
"wires": [
[
"922d3407be181bd0"
]
]
},
{
"id": "0eb9fa1b01114ab1",
"type": "debug",
"z": "99bda892.ebb298",
"name": "",
"active": true,
"tosidebar": true,
"console": false,
"tostatus": false,
"complete": "false",
"statusVal": "",
"statusType": "auto",
"x": 610,
"y": 260,
"wires": []
}
]
- Cronplus node in Node-RED
- The cron job will run every 10, 25, 40 and 55 minutes, every hour, every day as given by the term 10,25,40,55 * * * *
- HTTP Request Node
- Note that the actual Dweet address has been modified.
- Add your own Dweet project name.
- Function node in Node-RED
- Write file node in Node-RED
BirdNET-Pi API
- BirdWeather has an API in place and the BirdWeather (web) App makes use of it in producing the results seen via the web page portal.
- Documentation is here - [5] https://app.birdweather.com/api/index.html
- There is also an online query tool for testing queries here - https://app.birdweather.com/graphiql
Example Python code
- This is a simple python program that will return some BirdWeather results from station 273 (my station in Bundoora, Victoria Australia)
- Here is the Python code that can be copied.
import requests
import json
query = """query{
station(id: "273") {
id
location
name
topSpecies {
averageProbability
species {
id
commonName
scientificName
}
}
}
}"""
url = 'https://app.birdweather.com/graphql'
r = requests.post(url, json={'query': query})
#print(r.status_code)
#print(r.json())
data_json = json.dumps(r.json())
print(data_json)
- The output is shown in the Python Shell.
- The data is in JSON format
- Copy this output and paste it in JSONviewer to make the data human readable https://jsonviewer.com
- The data should be similar to the screen shot below.
BirdWeather online GraphiQL tool
- An online query tool for testing queries at this URL - https://app.birdweather.com/graphiql
- An example output is presented for station 273.
- Use the online documentation to modify your query https://app.birdweather.com/api/index.html.
- This will help you understand how the API work.
- In this example below the wikipediaSummary has been added to the query.
{
station(id: 2093) {
id
name
type
source
location
coords{lat lon}
latestDetectionAt
topSpecies(limit:10){
species {
commonName
}
}
}
}
API to monitor Bird species and Count
- API query to extract data from BirdWeather.com
- Python code breakdown:
- Analyse data from top 30 bird species. Data saved in text file named dailyBirdCount'txt
- Dweeting data to make available to others
- Saving BirdWeather data as a JSON file (JSON data)
- Output of Python file birdWeather1.py
Bird Weather Hourly Data Collection - birdWeatherHourly.py
- Will collect data from midnight to the current hour each day.
- Two different data plots are created from the python file named birdWeatherHourly.py.
- This first plot shows how the total tally of bird recordings increases every hour.
- The Total bird count is reset to zero at midnight each day.
- This plot shows the number of unique (different) bird species recorded each day.
- The data is plotted each hour.
- The graph shows that in a typical day between 15 and 20 different species of bird are detected.
# file name - birdWeatherHourly.py
import requests
import json
import datetime
import plotly
import plotly.graph_objects as go
import pandas
now = datetime.datetime.now()
date_stamp = now.strftime("%Y-%m-%d %H:%M:%S")
hour_str = now.strftime("%H")
hour_int = int(hour_str)
print(f'The currently hour is {hour_int}')
# Will collect data from midnight to the current hour each day
query = """query{
station(id: "273") {
id
location
name
topSpecies (limit: 60, period: {count: %s, unit: "hour"}) {
count
species {
id
commonName
scientificName
ebirdUrl
imageUrl
wikipediaSummary
}
}
}
}""" % (hour_int)
print(f'QUERY = {query}')
total_count = 0
different_bird_species = 0
# Requesting data from BirdWeather.com using API
url = 'https://app.birdweather.com/graphql'
r = requests.post(url, json={'query': query})
try:
data_dict = r.json()["data"]["station"]
#print(data_dict["id"])
topSpecies = data_dict["topSpecies"]
bird_dict = {"last_updated": date_stamp} # dictionary
# Analysing data for top 60 species and saving in file named dailyBirdCount.txt
for species in topSpecies:
count = species["count"]
total_count += count
species = species["species"]
print(f"species = {species['commonName']}, count = {count}")
data = str(different_bird_species) + "," + date_stamp + "," + str(species['commonName']) + "," + str(count) + "\n"
#print(f"data = {data}")
different_bird_species += 1
bird_dict.update({str(species['commonName']): count})
ebird_url = str(species['ebirdUrl'])
image_url = str(species['imageUrl'])
summary = str(species['wikipediaSummary'])
print()
print(f"dictionary = {bird_dict}\n")
print(f"ebird_url = {ebird_url}")
print(f"Summary = {summary}")
except:
# no detections from BirdWeather API
print("No data")
ebird_url = "https://ebird.org/species/railor5"
image_url = "https://birdweather.s3.amazonaws.com/species/301/RainbowLorikeet-standard-239eeed3a59c681ef12fa644ebc8e2d6.jpg"
summary = ""
# Total count of all birds detected per day
data_count = date_stamp + "," + hour_str + "," + str(total_count) + "," + str(different_bird_species) + "\n"
f = open('/home/pi/BirdWeather/dailyBirdTotalCount.txt','a')
f.write(data_count)
f.close()
# Bird URL to display to viewers - most recent new bird species from past hour
f = open('/home/pi/BirdWeather/mostRecentUniqueBirdUrl.txt','w')
f.write(str(ebird_url))
f.close()
# Save most recent bird image from internet. wb = write binary
url = image_url
myfile = requests.get(url)
open('/home/pi/BirdWeather/BirdImage.png', 'wb').write(myfile.content)
# import and plot data text file
# date_stamp,total_count,different_bird_species
df = pandas.read_csv('/home/pi/BirdWeather/dailyBirdTotalCount.txt')
fig = go.Figure(data = go.Scatter(mode='markers',
x=df.date_stamp,
y=df.total_count,
marker=dict(
color="LightSkyBlue",
size=20)
)
)
fig.update_layout(title="Total Count of Birds in the Day",
xaxis_title = 'Time',
yaxis_title = 'Bird Count',
font=dict(
size=20,
color="RebeccaPurple"
)
)
plotly.offline.plot(fig,
filename="/home/pi/BirdWeather/daily_bird_count.html",
auto_open=False)
print("Daily Bird Count data plotted")
# Different Bird Species Plot
fig = go.Figure(data = go.Scatter(mode='markers',
x=df.date_stamp,
y=df.different_bird_species,
marker=dict(
color="LightSkyBlue",
size=20)
)
)
fig.update_layout(title="Total Count of Birds in the Day",
xaxis_title = 'Time',
yaxis_title = 'Different Bird Species',
font=dict(
size=20,
color="RebeccaPurple"
)
)
plotly.offline.plot(fig,
filename="/home/pi/BirdWeather/different_bird_species.html",
auto_open=False)
print("Different Bird Species Count data plotted")
Bird Weather Specific Species - birdWeatherSpecificSpecies.py
- The query only retrieves data generated in the past 1 hour (count: 1).
- This python code names birdWeatherSpecificSpecies.py keeps separate text files for individual bird species and tracks the number of recordings per hour.
- Both common (Rainbow lorikeet) and rare birds (Powerful owl) can be recorded separately.
- In the future this data can also be plotted using plotly.
# file name - birdWeatherSpecificSpecies.py
import requests
import json
import datetime
import plotly
import plotly.graph_objects as go
import pandas
now = datetime.datetime.now()
date_stamp = now.strftime("%Y-%m-%d %H:%M:%S")
# The query only retrieves data generated in the past 1 hour (count: 1)
query = """query{
station(id: "273") {
id
location
name
topSpecies (limit: 60, period: {count: 1, unit: "hour"}) {
count
species {
id
commonName
scientificName
ebirdUrl
imageUrl
wikipediaSummary
}
}
}
}"""
# Requesting data from BirdWeather.com using API
url = 'https://app.birdweather.com/graphql'
r = requests.post(url, json={'query': query})
try:
data_dict = r.json()["data"]["station"]
#print(data_dict["id"])
topSpecies = data_dict["topSpecies"]
bird_dict = {"last_updated": date_stamp} # dictionary
total_count = 0
# Analysing data for top 60 species and saving in file named dailyBirdCount.txt
for species in topSpecies:
count = species["count"]
#total_count += count
species = species["species"]
print(f"species = {species['commonName']}, count = {count}")
# Powerful Owl detection (Powerful Owl id = 4249)
if species['id'] == "4249":
bird_data = date_stamp + "," + species['id'] + "," + species['commonName'] + "," + str(count) + "\n"
f = open('/home/pi/BirdWeather/powerfulOwlDetected.txt','a')
f.write(bird_data)
f.close()
# Rainbow Lorikeet detection (Rainbow Lorikeet id = 301)
if species['id'] == "301":
bird_data = date_stamp + "," + species['id'] + "," + species['commonName'] + "," + str(count) + "\n"
f = open('/home/pi/BirdWeather/rainbowLorikeetDetected.txt','a')
f.write(bird_data)
f.close()
except:
# no data
print("no detections in the last hour")
Bird Weather Once Daily - birdWeatherOnceDaily.py
- This code will record all topSpecies recorded during the day and the number of detections of each species.
- Just before midnight the birdWeatherOnceDaily.py script is run.
- It collates the top 60 bird species recorded during the entire day.
- Total recordings for each bird species are recorded.
- This should give researcher long term data on the number and population of bird species recorded in an area.
# file name - birdWeatherOnceDaily.py
import requests
import json
import datetime
import plotly
import plotly.graph_objects as go
import pandas
now = datetime.datetime.now()
date_stamp = now.strftime("%Y-%m-%d %H:%M:%S")
query = """query{
station(id: "273") {
id
location
name
topSpecies (limit: 60) {
count
species {
id
commonName
scientificName
ebirdUrl
imageUrl
wikipediaSummary
}
}
}
}"""
# Requesting data from BirdWeather.com using API
url = 'https://app.birdweather.com/graphql'
r = requests.post(url, json={'query': query})
data_dict = r.json()["data"]["station"]
#print(data_dict["id"])
topSpecies = data_dict["topSpecies"]
bird_dict = {"last_updated": date_stamp} # dictionary
total_count = 0
different_bird_species = 0
# Analysing data for top 60 species and saving in file named dailyBirdCount.txt
for species in topSpecies:
count = species["count"]
total_count += count
species = species["species"]
print(f"species = {species['commonName']}, count = {count}")
data = str(different_bird_species) + "," + date_stamp + "," + str(species['commonName']) + "," + str(count) + "\n"
print(f"data = {data}")
f = open('/home/pi/BirdWeather/dailyBirdCount.txt','a')
f.write(data)
f.close()
different_bird_species += 1
bird_dict.update({str(species['commonName']): count})
print()
print(f"dictionary = {bird_dict}\n")
# Saving BirdWeather.com data in a JSON file
with open("Bird_data.json", "w") as file:
#Using json.dump() to write the data into a JSON file.
json.dump(topSpecies, file)
BirdWeather - Dweet Birds Last 24 hours
# file name - dweetLast24Hours.py
import requests
import json
import datetime
now = datetime.datetime.now()
date_stamp = now.strftime("%Y-%m-%d %H:%M:%S")
query = """query{
station(id: "273") {
id
location
name
topSpecies (limit: 60, period: {count: 24, unit: "hour"}) {
count
species {
id
commonName
scientificName
}
}
}
}"""
print(f'QUERY = {query}')
total_count = 0
different_bird_species = 0
data = ""
# Requesting data from BirdWeather.com using API
url = 'https://app.birdweather.com/graphql'
r = requests.post(url, json={'query': query})
try:
data_dict = r.json()["data"]["station"]
#print(data_dict["id"])
topSpecies = data_dict["topSpecies"]
bird_dict = {"last_updated": date_stamp} # dictionary
# Analysing data for top 60 species and saving in file named dailyBirdCount.txt
for species in topSpecies:
count = species["count"]
total_count += count
species = species["species"]
print(f"species = {species['commonName']}, count = {count}")
#data = str(different_bird_species) + "," + date_stamp + "," + str(species['commonName']) + "," + str(count) + "\n"
#print(f"data = {data}")
data = str(species['commonName']) + "-" + str(count) + ", "
different_bird_species += 1
bird_dict.update({str(different_bird_species):data})
print()
print(f"dictionary = {bird_dict}\n")
except:
# no detections from BirdWeather API
print("No data")
try:
print("Preparing dweet")
url = "https://dweet.io/dweet/for/birdnet-pi-your-dweet-name-here?"
x = requests.post(url, json=bird_dict)
print(x.text)
except:
print("Dweet failed")
# check dweet with - https://dweet.io/get/latest/dweet/for/birdnet-pi-your-dweet-name-here
