View source for Graphing data using Plotly

== Overview ==
* In this lesson we are going to learn how to '''graph data''' collected from our environmental sensors. 
* Data will be graphed using a python package called '''plotly'''. 
* Plotly can produce graphs in '''HTML format''' so that they can be easily displayed on a web page. 
* Graphing sensor data is very important because it shows trends and changes in data more clearly. 

== Learning Objectives ==
* Learn how to create graphs using plotly
* Learn how to import data from files using pandas
* Learn how to add more formatting options in plotly

== Creating graphs using plotly ==
* In this example we are going to install the plotly python package and then write a program to create some simple graphs.

=== Installation of the plotly python package ===
* From the Raspberry Pi top bar menu click on the Terminal icon.
* Open the Terminal and enter the command '''pip3 install plotly'''
* '''pip''' is short for Pip Installs Python. 
* pip3 installs packages for python3

[[File:Screen Shot 2021-12-30 at 6.00.15 pm.png]]

* The installation process will take approximately one minute.

[[File:Screen Shot 2021-12-30 at 6.01.09 pm.png]]

* When plotly has been installed you should see an output like the one below.
* The version of plotly that was installed is '''plotly-4.14.3'''
* Other python packages may also be installed to support plotly.
* In this instance, another package named '''retrying-1.3.3''' was also installed.

[[File:Screen Shot 2021-12-30 at 6.11.07 pm.png]]

=== First plotly program in Python ===
* Open '''Python3 (IDLE3)'''.
* From the File dropdown menu select '''New File'''.
* Enter the python code shown below.
* Save this file as '''temp_plot_gen.py''' in your project’s directory (the file name is short for temperature plot generator)

[[File:Screen Shot 2021-12-30 at 6.12.45 pm.png]]

* '''Save''' and '''Run''' the program. 
* After approximately 10-20 seconds the web browser Chromium will open and display a '''graph''' of your data.

[[File:Screen Shot 2021-12-30 at 6.14.03 pm.png]]

=== Adding title descriptions to your graph ===
* Update the python code with to add '''layout details''' in the graph.

[[File:Screen Shot 2021-12-30 at 6.15.16 pm.png]]

* The completed code is as follows.

[[File:Screen Shot 2021-12-30 at 6.16.16 pm.png]]

* Save and '''Run''' the program. A '''new tab''' in the web browser will open and display this graph.

[[File:Screen Shot 2021-12-30 at 6.17.43 pm.png]]

== Importing and plotting data from text files ==
* Plotly is good at plotting graphs, but it cannot important data from text files. 
* To prepare data for plotly we need another python package named '''pandas'''. 
* In the following lesson we will install pandas and demonstrate how to import data from a text file. 
* Before we can start using pandas we also need to '''clean''' and '''label''' the data in our text file to make sure it is ready to import.

=== Cleaning and adding labels to sensor data ===
* To import data from our data.txt file need to '''clean the data''' and give '''labels''' to the data.
* Click on the '''File Manager'''.
* Navigate to your project’s directory and open the '''data.txt''' file. 
* By default, it should open in the '''Mousepad''' text editor.

[[File:Screen Shot 2021-12-30 at 6.20.19 pm.png]]

* The first line of the data file needs to include a '''label''' for each data set
* Data labels for our file are: '''Date,Temp,Pres,Volt'''
* Make sure there are '''no spaces''' between the labels. Spelling must be exactly as shown.
* '''Delete''' or '''correct''' any incomplete data. 
* Prepare your '''data.txt''' file so it looks like the following.

[[File:Screen Shot 2021-12-30 at 6.23.50 pm.png]]

=== Installing pandas python package to read data from text files ===
* To read data from our data.txt file we need to install another python package named '''pandas'''.
* To install the pandas package open the Terminal and enter the following commands.
** '''pip3 install pandas''' – will take 1-2 minutes to install
** '''sudo apt-get update''' – update all libraries on the Raspberry Pi
** '''sudo apt-get install libatlas-base-dev''' – extra missing software package to support pandas
* To test that pandas has been installed correctly open the Python Shell and enter -  '''import pandas'''
* If the pandas python package has installed correctly you should see no errors.

[[File:Screen Shot 2021-12-30 at 6.25.36 pm.png]]

* If the pandas python package has not been installed then you will see the following message in the Python shell.

[[File:Screen Shot 2021-12-30 at 6.26.33 pm.png]]

=== Reading data from data.txt using pandas ===
* In the temp_plot_gen.py file add the following lines of code.

[[File:Screen Shot 2021-12-30 at 6.28.06 pm.png]]

* The complete code is shown below.

[[File:Screen Shot 2021-12-30 at 6.29.10 pm.png]]

* '''Save''' and '''Run''' the program.
* The following output will be output in the '''Python Shell'''.
* The output will show you the number of data entries extracted from the data.txt file.
* All this tabled data is contained within a '''python Object''' named '''df''' 
* To see the data in the Python Shell we used the statement '''print(df)'''

[[File:Screen Shot 2021-12-30 at 6.30.51 pm.png]]

=== Graphing data using pandas df Object ===
* Update your code to include Date and Temp data from the data.txt file using pandas df Object.
** '''x=df.Date'''
** '''y=df.Temp'''
* Update the layout so that the '''title = ‘Temperature data’''' and '''yaxis_title = ‘Temperature’'''

[[File:Screen Shot 2021-12-30 at 6.32.25 pm.png]]

* In the web browser you should see a plot of the temperature data with data taken from the data.txt file.

[[File:Screen Shot 2021-12-30 at 6.36.32 pm.png]]

== Adding more formatting options in plotly ==
* In this example we are going to expand on the formatting properties within plotly. 

=== Additional formatting options in plotly ===
* We can replace a line trace with '''dots'''. 
* Update the following '''go.Scatter''' statement to the following.

[[File:Screen Shot 2021-12-30 at 6.38.14 pm.png]]

* The complete code is shown below.

[[File:Screen Shot 2021-12-30 at 6.38.57 pm.png]]

* In the plot below, '''lines''' have been replaced with '''points'''.

[[File:Screen Shot 2021-12-30 at 6.40.13 pm.png]]

* We can change the '''size''' and colour of the points.
* Include a marker dictionary with the formatting parameters of '''go.Scatter'''.

[[File:Screen Shot 2021-12-30 at 6.41.15 pm.png]]

* The complete code is shown below.

[[File:Screen Shot 2021-12-30 at 6.41.59 pm.png]]

* '''Save''' and '''Run''' the program. 
* Individual plotted points will be '''larger and blue''' in colour.

[[File:Screen Shot 2021-12-30 at 6.43.20 pm.png]]

* Now we need to increase the size of the text in the labels for the '''title, x-axis and y-axis'''.
* Add a font dictionary to the '''fig.update_layout''' statement.

[[File:Screen Shot 2021-12-30 at 6.45.42 pm.png]]

* The full code is shown below.

[[File:Screen Shot 2021-12-30 at 6.46.35 pm.png]]

* Save and Run the program. 
* All labels should be larger and coloured purple.

[[File:Screen Shot 2021-12-30 at 6.48.19 pm.png]]

=== Saving the HTML graph as a file ===
* Rather than have the graph displayed immediately in a web browser, we can save the graph as a file.
* Once we have a file, we can open it later or link to it on a '''web page'''.
* Update your code by adding the following statement at the '''end of your code'''.
* Edit the code to include your '''project folder name'''.
* The '''auto_open=False''' statement will prevent the graph from automatically opening up in your Pi Browser.

[[File:Screen Shot 2021-12-30 at 6.52.49 pm.png]]

* To show that the program is complete we have added a '''print()''' statement at the end of the code.

[[File:Screen Shot 2021-12-30 at 6.53.52 pm.png]]

* Congratulations. Now you can print all your data, and its also ready to add to your web page.