Smart Cities - Programming the ESP-01 using YAML

Overview of ESP-01 Programming Steps

• Get the ESP-01 boards and the ESP-01 Programmer
• Add a Program Switch to the ESP-01 Programmer. See here
• Get the SARCNET ESP-01 WiFi Modules with a DS18B20 on a 15 cm lead as programmed below.
• Remove the plastic frame from the header pins of your ESP-01 board and solder it flush to the rear of the SARCNET ESP-01 WiFi Module board. Use a very fine tip soldering iron and flux. Do not cut the header pins off.
• Plug the DS18B20 into the SARCNET ESP-01 WiFi Module, 3-pin SIP socket, observing the same orientation of the TO-92 package outline. Note: The flat side of the sensor should face the PCB
• Plug the ESP-01 board into the programmer via the 8 header pins. Observe the orientation of the ESP-01 board. It should be mounted directly above the programmer board.
• Press the Program Button
• Plug the programmer into a USB socket on the Raspberry Pi
• Enter: cd ~/env
• Enter: source bin/activate
• Compile the esphome image and upload to the ESP-01 board for the first time using the USB adapter. e.g. The linux command is: esphome -s name iat -s zone 15 -s ip_address 192.168.2.17 run temp.yaml (This is for the Inside Air Temperature sensor only. This will have the ESP-01 board subscribe to topic p15 and publish to topic s15 - The server will send '?' to p15 and will receive temperature readings every 60 seconds by subscribing to s15) Change these topics if you want. See the example ESP-01 Programming.txt file attached.
• Select: [1] /dev/ttyUSB0 (USB2.0-Serial) for both uploading and logging
• Search the log output for where esphome finds the Dallas sensor: E.G. GPIO 1-wire bus: Pin: GPIO2 Found devices: 0x5e00000043063328 (DS18B20) Dallas Temperature Sensor: Address: 0x0200000042495a28 (DS18B20)
• Edit the temp.yaml file (using the Geany Programmer's Editor)
• Copy and paste the sensor address into the YAML file
• Save the file and reprogram the ESP-01 board with this address.
• Re-program the ESP-01 board over the air Enter the command: esphome -s name iat -s zone 15 -s ip_address 192.168.2.17 run temp.yaml
• Select: [2] Over The Air (192.168.2.17) for both uploading and logging
• Check the log to see that the ESP-01 board sends the temperature to the MQTT broker every 60 seconds
• Now remove the SARCNET ESP-01 WiFi Module from the programmer and supply it with 5V by soldering wires to the pads on the front left. It should boot up and work.

Add a Program Switch to the ESP-01 Programmer

Use following reference https://www.instructables.com/USB-to-ESP-01-Board-Adapter-Modification/ [1]

temp.yaml code

#This ESPHome temp.yaml file configures a black ESP-01 board fitted with a Dallas DS18B20 temperature sensor
#The temperature sensor is connected via the one-wire bus to GPIO02
#One sensor is for Inside Air Temperatire (IAT) and the other is for Outside Air Temperature (OAT)
#To upload: esphome run temp.yaml
#To upload set the name, zone and IP address: esphome -s name iat -s zone 15 -s ip_address 192.168.2.17 run temp.yaml
#To upload set the name, zone and IP address: esphome -s name oat -s zone 16 -s ip_address 192.168.2.18 run temp.yaml


#Set up constants
substitutions:
  name:          'iat'            #The name of this device
  zone:          '15'             #IAT topic
  ip_address:    '192.168.2.17'   #Static IP address of this device
#  name:          'oat'            #The name of this device
#  ip_address:    '192.168.2.18'   #Static IP address of this device
#  zone:          '16'             #OAT topic
  from_server:    'p${zone}'       #The topic for receiving messages from the server
  to_server:      's${zone}'       #The topic for sending messages to the server
  state_message:  '?'

#Set up core functionality
esphome:
  name: ${name}                   #The name of this device
  compile_process_limit: 2        #Reduce the number of cores to use. Necessary for compiling on the RPi 3.

#Set up processor and board
esp8266:                          #Use the ESP8266 platform
  board: esp01_1m                 #The Black ESP-01 board with 1MB of ram

#Set up WiFi
wifi:
  ssid:        A                  #This network is isolated from the Internet, but has an MQTT broker
  password:    <SECRET>
  manual_ip:
    static_ip: ${ip_address}
    gateway:   192.168.2.1
    subnet:    255.255.255.0

#Enable a fallback access point in case the WiFi connection fails
  ap:
    ssid: ${name}                 #Use the device name as the SSID

#Enable the captive portal for the WiFi access point
captive_portal:

#Enable logging
logger:
  logs:
    mqtt.component: DEBUG

#Enable programming Over The Air
ota:
  - platform: esphome

#Enable Message Queuing Telemetry Transport
mqtt:
  broker: 192.168.2.2             #The IP Address of the MQTT broker
  port: 1883                      #The port number of the MQTT broker
  on_message:                     #When a message is received from the MQTT broker
    - topic: ${from_server}       #When a message is received from the server
      payload: ${state_message}   #When a state message is received
      then:
        mqtt.publish:             #Publish an MQTT message
          topic: ${to_server}     #Publish a message to the server
          payload: !lambda |-
            return to_string(id(temp).state);

one_wire:
  - platform: gpio
    pin: GPIO02

sensor:                   
  - platform: dallas_temp
    id: temp
    address: 0x0200000042495a28
    update_interval: 60s
    on_value:
      then:
        mqtt.publish:             #Publish an MQTT message
          topic: ${to_server}     #Publish a message to the server
          payload: !lambda |-
            return to_string(id(temp).state);

Explanation of temp.yaml file

This temp.yaml file configures a black ESP-01 board fitted with two Dallas DS18B20 temperature sensors (one for Inside Air Temperature (IAT) and one for Outside Air Temperature (OAT)) and integrates it with MQTT to publish and receive data. Here's an interpretation of its key sections:

1. Substitutions:

The substitutions block sets up constants used throughout the file. It allows flexibility when deploying this configuration for multiple sensors.

substitutions:
  name:          'iat'            # The name of the device, 'iat' for Inside Air Temperature.
  zone:          '15'             # MQTT topic ID for this sensor (IAT).
  ip_address:    '192.168.2.17'   # Static IP address for the device.
  from_server:   'p${zone}'       # Topic to receive messages from the server, dynamically set as 'p15'.
  to_server:     's${zone}'       # Topic to send messages to the server, dynamically set as 's15'.
  state_message: '?'

• Flexibility: You can modify this to deploy different sensors by adjusting the name, zone, and ip_address values, e.g., for oat (Outside Air Temperature).
• from_server and to_server: These are dynamic MQTT topics to handle inbound (from_server) and outbound (to_server) messages for this specific device.

2. ESPHome Core Configuration:

esphome:
  name: ${name}                   # The device name, substituted from the 'substitutions' block.
  compile_process_limit: 2        # Limits the number of cores used during compilation (for performance reasons on RPi).

• Device Name: Set using the substitutions variable name, so it's easy to deploy with different names for different sensors.

3. ESP8266 Platform and Board Configuration:

esp8266:
  board: esp01_1m                 # Specifies that the ESP-01 board with 1MB flash is being used.

• Board Type: Specifically for the black ESP-01 board with 1MB of flash memory.

4. WiFi Setup:

wifi:
  ssid:        A                  # WiFi network SSID.
  password:    <SECRET>            # WiFi password.
  manual_ip:
    static_ip: ${ip_address}       # Assigns the static IP address from the 'substitutions' block.
    gateway:   192.168.2.1         # The gateway IP address.
    subnet:    255.255.255.0       # Subnet mask.

• Static IP: The static IP address is assigned using the substitutions field, ensuring the device always uses a fixed IP address (192.168.2.17 in this case).
• Fallback AP: If the device fails to connect to the specified WiFi network, it will fall back to creating its own AP.

5. MQTT Configuration:

mqtt:
  broker: 192.168.2.2             # The IP address of the MQTT broker on your isolated network.
  port: 1883                      # Default MQTT port (1883).
  on_message:
    - topic: ${from_server}       # Listens for messages from the server on the 'from_server' topic.
      payload: ${state_message}   # If a message matches the state_message ('?'), it triggers the next action.
      then:
        mqtt.publish:             # Publishes a response back to the server.
          topic: ${to_server}     # Publishes the current temperature to the 'to_server' topic.
          payload: !lambda |-
            return to_string(id(temp).state);  # Sends the current temperature value.

• Broker and Port: The device connects to the MQTT broker at 192.168.2.2 (on the experimental network) on port 1883.

on_message: When a message is received on the from_server topic (e.g., 'p15'), it checks if the payload matches state_message (which is set to '?'). If so, it publishes the current temperature to the to_server topic (e.g., 's15').

• Publishing Temperature: Uses a lambda function to publish the current temperature read from the sensor.

6. Sensor Configuration:

one_wire:
  - platform: gpio
    pin: GPIO02                   # The one-wire bus is connected to GPIO02.

sensor:                   
  - platform: dallas_temp
    id: temp
    address: 0x0200000042495a28    # The unique address of the Dallas sensor (for IAT).
    update_interval: 60s           # The sensor updates every 60 seconds.
    on_value:
      then:
        mqtt.publish:             # When the temperature is updated, publish the new value via MQTT.
          topic: ${to_server}     # Publishes to the 'to_server' topic.
          payload: !lambda |-
            return to_string(id(temp).state);

• Dallas DS18B20: Configures the temperature sensor on the one-wire bus connected to GPIO02.
• Sensor Address: The unique address of the DS18B20 sensor is specified to identify the exact sensor.
• Temperature Updates: The sensor updates every 60 seconds. When it updates, the temperature value is automatically published via MQTT to the to_server topic (e.g., 's15').

Key Features:

• Static IP Setup: The device uses a static IP address (192.168.2.17) to ensure a stable connection to the experimental router.
• MQTT Communication: The ESP-01 device communicates with an MQTT broker (192.168.2.2). It subscribes to messages from the server and publishes temperature data back to the server.
• Flexible Substitutions: The use of substitutions makes it easy to switch between sensors (IAT and OAT) by simply changing the name, zone, and ip_address.

How It Works:

• Startup: The ESP-01 connects to your WiFi network using the static IP address defined.
• MQTT Communication: It connects to the MQTT broker and listens for messages on the topic (e.g., 'p15' for IAT).
• Sensor Updates: Every 60 seconds, the DS18B20 sensor reads the temperature, and this data is published to the MQTT broker on the corresponding topic (e.g., 's15').

Uploading temp.yaml file to ESP-01

• Plug the ESP-01 board into the programmer via the 8 header pins. Observe the orientation of the ESP-01 board. It should be mounted directly above the programmer board.
• Press the Program Button
• Plug the programmer into a USB socket on the Raspberry Pi. On the Raspberry Pi 3 B+ all USB sockets are USB 2.0.
• Ensure the Raspberry Pi is connected to a router with Internet access.
• Enter: cd ~/env
• Enter: source bin/activate to activate the Virtual Environment
• Compile the esphome image and upload to the ESP-01 board for the first time using the USB adapter.
• The linux command is: esphome -s name iat -s zone 15 -s ip_address 192.168.2.17 run temp.yaml (This is for the Inside Air Temperature sensor only. This will have the ESP-01 board subscribe to topic p15 and publish to topic s15 - The server will send '?' to p15 and will receive temperature readings every 60 seconds by subscribing to s15) Change these topics if you want. See the example ESP-01 Programming.txt file attached.
• Compilation and uploading of the temp.yaml file will take 10 minutes.

esphome -s name iat -s zone 15 -s ip_address 192.168.2.17 run temp.yaml

When programming the EPS-01 why enter The linux command esphome -s name iat -s zone 15 -s ip_address 192.168.2.17 run temp.yaml when these details are already in the .yaml file being compiled?

esphome -s name iat -s zone 15 -s ip_address 192.168.2.17 run temp.yaml

• Allows you to override the values specified in the temp.yaml file using substitutions at runtime.
• This gives you flexibility in case you want to use different values temporarily without editing the .yaml file each time.

Why Use This Command?

• Flexibility Across Devices: If you're using the same .yaml file for multiple ESP-01 devices (e.g., one for inside air temperature, another for outside air temperature), you can use this command to set device-specific names, zones, or IP addresses without modifying the file itself. This is especially useful when managing many devices with slight configuration differences.
• Avoid Manual Changes in the YAML File: Instead of editing the .yaml file every time you want to change the name, zone, or IP address for different devices, you can simply pass these values via the command line. This is efficient if you frequently switch between devices or configurations.
• Testing Different Configurations: You may want to test different configurations on the same device without editing the .yaml file. By passing different arguments in the command, you can test multiple setups without permanently changing the file.
• Automation and Scripts: If you're automating the flashing process or have deployment scripts, it's easier to pass variables through the command line rather than editing each configuration file manually.

How It Works:

The -s flag is used to specify a substitution value at runtime. For example:

• -s name iat: Sets the substitution variable name to iat (used for naming the device).
• -s zone 15: Sets the zone to 15.
• -s ip_address 192.168.2.17: Sets the ip_address for this specific device.

The substitutions section of your .yaml file acts as a template, and you can dynamically fill in those values by passing them via the command line.

Example YAML:

substitutions:
  name: 'default_name'
  zone: 'default_zone'
  ip_address: 'default_ip'

esphome:
  name: ${name}
  ...
wifi:
  manual_ip:
    static_ip: ${ip_address}
    ...
mqtt:
  topic: ${zone}
  ...

By running the command with -s, you override the defaults in the .yaml file for that session, allowing for more flexibility.

How to output Log from ESPHome

To find the log output for ESPHome on a Raspberry Pi, follow these steps:

Use ESPHome's Command-Line Interface (CLI):

• To find the log output for ESPHome on a Raspberry Pi, use ESPHome's Command-Line Interface (CLI):
• Run the following command to see the logs in real-time from your ESPHome device:

esphome logs temp.yaml

Replace temp.yaml with the name of your configuration file. ESPHome will attempt to connect to the device (over WiFi) and show the logs.

Monitor Logs via Web Dashboard:

• If you’re running ESPHome's dashboard on your Raspberry Pi, you can check logs from the web interface:
• Open the ESPHome web dashboard (usually at http://<raspberry-pi-ip>:6052/).
• Locate your device in the list and click on the "Logs" button next to it.
• This will show you live logs from your ESP device.

Check ESPHome Service Logs:

If you're running ESPHome as a systemd service on your Raspberry Pi, you can also check its service logs using the following command:

sudo journalctl -u esphome -f

This will display the logs related to the ESPHome service in real-time.

Enable Logging in YAML File:

Make sure logging is enabled in your ESPHome configuration file (temp.yaml) so that logs can be captured:

logger:
  level: DEBUG  # You can set this to INFO, WARNING, or ERROR depending on what you want to see.

Logs will be available after you update and upload the configuration to the device.

Existing ESP Logger

To exit the ESPHome logger while viewing logs from the command line, simply use the following key combination:

Ctrl + C

This will stop the log streaming and bring you back to the terminal prompt.

How to safely unplug the ESP-01 programmer from the Raspberry Pi's serial port

• Stop any active communication: If you’re currently using the ESPHome logger or any other serial communication tool (e.g., minicom, screen, or a Python script), make sure to stop or exit the program first. This prevents any data transmission errors when disconnecting.
• If you’re viewing logs in ESPHome, press Ctrl + C to stop the logger.
• If using any other tool like screen, Ctrl + A followed by \ will exit the session.
• Power Off ESP-01 (Optional): Some people prefer to power off the ESP-01 by either disconnecting the Raspberry Pi or disabling the power to the device. If your ESP-01 is not running a critical operation, you can skip this step.
• Unplug the ESP-01 Programmer: Once you've stopped any active serial communication, you can simply unplug the ESP-01 programmer from the Raspberry Pi's
• Reinsert When Needed: If you need to plug the ESP-01 programmer back in, you can do so anytime, and your Raspberry Pi will detect it again.

Soldering ESP-01 to SARCNET ESP-01 WiFi Module

• The SARCNET ESP-01 WiFi Module from the programmer and supply it with 5V by soldering wires to the pads on the front left. It should boot up and work. * Here are the pinouts from top to bottom:

• • Input: +/-20V. On = > 2.5V (Requires Q1)
  • Ground
  • Output: RDS On = 2.5 Ohms, ID max = 115mA, VDSSmax = 60V (Requires Q2-4, C3, Q4 R5-8)
  • Ground
  • Power: 5-15 VDC