Tiny House temperature monitoring using Python, Dweepy, Plotly and Cron

Overview

• In this example we will retrieve a Dweet using Python so that we can save and graph Tiny House data.
• All the code will be written in Python.
• Data will be retrieved from the internet using Dweepy
• Data will be graphed using the python library Plotly
• The Python code will be scheduled to run hourly using Cron.

Installing Dweepy

• To install dweepy open the Terminal
• Enter the command pip3 install dweepy
• To verify that dweepy has installed correctly open Thonny and enter import dweepy into the Python Shell
• If there are no errors dweepy has been installed correctly.

Create Working Folder for Project

• Create a new folder named Tiny_House_12 in the home directory.
• You can save all your files and data in this directory.

Get Dweet Data

• Enter the following code in Python to retrieve data from a Dweet.
• Change the name of the Dweet to your specific Tiny House number
• In this case the name of the Dweet is SMC-Tiny-House-12-bundoora
• The Dweet replies with the following data:
  • thing - the name of the dweet
  • created - when the dweet was created
  • content - data within a package, such as
    • bat - battery voltage
    • intHumid - internal humidity
    • intTemp - internal temperature
    • extTemp - external temperature

Extracting Specific Data from Dweet

• To extract individual data sets from the dweet, follow the example below.
• The Dweet extracted from the provided url is added to a python dictionary dict = url[0]
• Individual data values can then be accessed from within the dictionary using attribute name.

Saving Data

import dweepy
#https://github.com/paddycarey/dweepy - Reference

url = dweepy.get_latest_dweet_for('SMC-Tiny-House-12-bundoora')
print(url)

dict = url[0]
voltage = dict['content'][('bat')]
print(f'The battery voltage is {voltage}')

internalTemp = dict['content'][('intTemp')]
print(f'The internal temperature is {internalTemp}')

internalHumid = dict['content'][('intHumid')]
print(f'The internal humidity is {internalHumid}')

externalTemp = dict['content'][('extTemp')]
print(f'The outside temperature is {externalTemp}')

datetime = dict['content'][('date')]

# Saving data
data = str(datetime) + "," + str(internalTemp) + "," + str(internalHumid) + "," + \
       str(externalTemp) + "," + str(voltage) + "\n"
print(f'The data is =  {data}')
         
f = open('/home/pi/Tiny_House_12/tiny_house_12_data.txt','a')
f.write(data)
f.close()
print("The data has been saved")

Prepare Saved Data for Graphing

• Open the File Manager
• The open the file named tiny_house_12_data.txt
• At the top line of the file add the following header data
• datetime,intTemp,intHumid,extTemp,battery
  • datetime - the date and time the data was created
  • intTemp - internal Temperature of the Tiny House
  • intHumid - internal humidity of the Tiny House
  • extTemp - the external (outside) temperature
  • battery - the battery voltage

Scheduling hourly data collection using Cron

• Python files can be scheduled to run every hour to process data from dweepy (dweet data).
• Open the Terminal
• Enter the command crontab -e
• The first time you use crontab you may be prompted for an editor. Choose nano

• Crontab is a text file that can edited using nano.

• Enter the following details to run the following files every hour.

What is Crontab (Cron)

Crontab is a utility in Unix-like operating systems that allows you to schedule and automate tasks at specified intervals. You can use crontab to schedule tasks to run at specific times, on specific days, or at regular intervals. Here's how it works and how to run a Python file and log errors using crontab:

• Edit Crontab File: You can edit your crontab file using the crontab -e command. This will open the crontab file for your user in a text editor.
• Cron Syntax: Each line in the crontab file represents a scheduled task and follows a specific syntax:

* * * * * command-to-be-executed
- - - - -
| | | | |
| | | | +----- Day of the week (0 - 6) (Sunday=0)
| | | +------- Month (1 - 12)
| | +--------- Day of the month (1 - 31)
| +----------- Hour (0 - 23)
+------------- Minute (0 - 59)

• Sheduling Python Scripts: To schedule a Python script to run, you would specify the full path to the Python interpreter and the path to your Python script as the command-to-be-executed. For example, the following code will run every minute.

* * * * * /usr/bin/python3 /path/to/your_script.py

• Logging Errors: To log errors from your Python script when it runs via crontab, you can redirect the standard error (stderr) output to a log file. Modify your crontab entry like this. In this example, 2>&1 redirects stderr (file descriptor 2) to stdout (file descriptor 1), and both stdout and stderr are appended to the specified log file.

* * * * * /usr/bin/python3 /path/to/your_script.py >> /path/to/error_log.log 2>&1

• Save and Exit: After editing your crontab file, save and exit the text editor. The tasks will run according to the schedule you specified.
• View Crontab: You can view your current crontab entries without editing by using the crontab -l command.

Remember that when scheduling tasks using crontab, it's important to specify the full paths to executables and files since the environment may not be the same as your interactive shell.

Additionally, you may need to make your Python script executable by running chmod +x /path/to/your_script.py to ensure that it can be executed by the cron job.

Plotting Data

• Plotly and Pandas are libraries used to process and plot two temperature variables (e.g., int_temp and ext_temp - internal and external temperatures) for a Tiny House from a text file.
• You can create two scatter traces within the go.Figure object. Here's an example of how to do it:

import pandas as pd
import plotly.express as px

# Step 1: Import the necessary libraries

# Step 2: Read the data from the text file into a Pandas DataFrame
data = pd.read_csv('/home/pi/Tiny_House_8/tiny_house_8_data.txt')

# Step 3: Create a Plotly figure with two scatter plots
fig = px.scatter(data, x='datetime', y=['intTemp', 'extTemp'],
                 labels={'datetime': 'Date and Time', 'value': 'Temperature (°C)'},
                 title='Internal vs. External Temperature')

# Step 4: Customize the plot layout
fig.update_layout(xaxis_title='Date and Time', yaxis_title='Temperature (°C)')

# Step 5: Save the plot as an HTML file
fig.write_html('/home/pi/Tiny_House_8/temperature_plot_tiny_house_8.html')

print("Temperature plot saved as temperature_plot.html file")

Pandas and Plotly

Plotly and Pandas are both Python libraries used for data manipulation and visualization:

Pandas:

• Type: Library
• Description: Pandas is an open-source data analysis and manipulation library for Python.
• It provides data structures like DataFrames and Series, which allow you to work with structured data more effectively.
• Pandas makes it easy to clean, filter, transform, and analyze data.

Plotly:

• Type: Library
• Description: Plotly is an open-source Python library for interactive data visualization.
• It provides a high-level interface for creating a wide range of interactive charts and graphs, including scatter plots, line charts, bar charts, heatmaps, and more.
• Plotly can be used in various Python environments, including Jupyter notebooks and web applications.

In the code you provided earlier, Pandas is used to read data from a CSV file into a DataFrame, and Plotly is used to create an interactive scatter plot to visualize the data. Both libraries are commonly used in data science and data visualization tasks in Python.