ESP32 OTA updates using http/https

Enfield Cat's Blog: Arduino and other projects.

Project Location...

GitHub Reference: https://github.com/enfieldcat/ota_updater

Why use Over The Air (OTA) Updates?

Having a board tethered to your workstation is generally fine while you are developing and debugging your code. But what if after having installed a number of units in inaccessible places you find some bug that causes your IOT gadget to stop working every new moon? You could visit each unit with your laptop and update it. ...OR... you could use an OTA update to flash it remotely.

The standard Arduino OTA examples imply that you would need to have your Arduino IDE running in order to perform the update. That sounds OK if you only have a few units to update and they are on the same TCP/IP subnet as your IDE. So lets investigate an option where you just download your compiled code to a web server (say in the cloud), and your IOT gadgets can update themselves even if they are 100 miles away.

How OTA works for ESP32

The ESP32 typically will have 4 megabytes of EEPROM, and this is partitioned into segments. The fist 64k is used to hold boot information, partition table and nvram settings. Executable code should always be placed to start on a 64k boundary. So the remaining memory is typically then split into 2 OTA partitions and a SPIFFS filesystem. When you flash your ESP32 from the IDE the code is written into the first OTA partition. This leaves the second partition open to receive a code update while the running code continues to operate out of the first. The update process in brief description works as follows:

Check which partition we are running in, and prepare the other one as the OTA destination
Download the code into the OTA destination partition
Verify the downloaded code matches its advertised size, sha256 check sum etc.
If the checks pass, then change the boot partion to the OTA partition and reboot the unit.
The code should run the updated code, and the original partition becomes the target for the next OTA update.
If you need to rollback the update, then changing the boot partition back to the first partition is possible, Then to rollback reboot again.

Prepare web server

On the web server we will create a directory under the webserver root directory to hold data for our project. eg:
```
mkdir /var/www/html/projects/ota_updater
```
The compiled bin file from our project will be copied"sketch" --> "export compiled binary" to this directory. Either use the Arduino CLI, ESP32 IDE, or from the Arduino IDE:

We need a metadata file called "metadata.php" to hold the file name, size and checksum of the project. This may be created manually, or generated on the fly by PHP code if the web server has PHP enabled. Here is an example of such PHP code:

<?php
$filename="esp32.img";
$statvar=stat($filename);
$sha256var=hash_file("sha256", $filename);
echo "name: $filename\n";
echo "size: $statvar[size]\n";
echo "sequence: $statvar[mtime]\n";
echo "sha256: $sha256var\n"; ?>

The result this generates is a simple 4 line yaml file as follows:

name: esp32.img
size: 711264
sequence: 1586381426
sha256: 24977a3fc2537fc289d32c717db8dad4f43856fefcb27faf0de9034e730c2eba

The name is used to identify the binary to be used for the OTA. The size and checksum are used to verify completeness and correctness of the download. The sequence number is used to determine if the image on board needs an update. No update is done if the current image matches or exceeds this sequence number. Either generate sequence numbers, or treat the file age in seconds since 1-Jan-1970 as the sequence number. The default sequence number of the original installation will be zero. The sequence number of the latest successful OTA is stored in NVRAM of the board.

Include ota_updater code in your Arduino sketch

Take a look at the ota_updater project and the code you need to include in your project is under the "ota_control" tab. This creates an ota_control class. A quirk of the Arduino IDE is you'll need to append any logic to use this class at the bottom of this file or in ant tab to the right of it.
The class has a number of functions it makes public:
- ota_control() This is the class constructor.
- bool update(char *baseurl, const char *cert, char *metadata) The metadata file and binary are both expected to be found relative the baseurl. The metadata file is as described previously, and could be either php, or hardcoded yaml. Returns true on successful update or false on failure. When using http, the cert field can be set to NULL, otherwise this is a pointer to the CA root certificate string.
- bool revert() Revert to previous image if available. True on success or false on failure.
- const char* get_boot_partition_label() returns the name of the boot partition. Typically for Arduino IDE this will be either "app0" or "app1".
- uint32_t get_boot_partition_size() returns the size of the boot (running) partition in bytes.
- const char* get_next_partition_label() Returns the partition name of the next partition to be used for OTA update.
- uint32_t get_next_partition_size() Returns the size of the next partition to be used for OTA. The update call uses this to determine if there is sufficient space available to deploy the OTA image.
- char* get_status_message() Returns human readable message indicating status of update or revert calls.

Further considerations

The ota_control class will check the prefix of the url to determine if http or https is being used. If https is being used then you need to ensure:
- Time is is correct on the ESP32, using NTP is recommended. You do not have to set the local timezone unless you want to. Note certificates will have a not before and after time to indicate their period of validity.
- The code has not been tested using a self-signed certificate. If you want to use a self signed certificate, then it is up to you resolve any issues caused by having a self signed certificate.
- Set your CA certificate before calling the update. In testing my CA's root certificate did not work, and I needed to use my intermediate certificate. To get this I used openssl to get my certificate chain, and then inspect each certificate before copying the CA cert to my code:
```
openssl s_client -showcerts -verify 5 -connect camelthorn.cloud:443 < /dev/null | awk '/BEGIN/,/END/{ if(/BEGIN/){a++}; out="cert"a".pem"; print > out}'
openssl x509 -noout -in cert1.pem -text
openssl x509 -noout -in cert2.pem -text
```
While http can be used instead of https, note this does not give you any assurance you are downloading from your expected OTA hosts.
In the event you update your image using OTA and are now using an unusable/unbootable image in either OTA partition, then updating the image from the Arduino IDE will reset the OTA partition to the first partition and over-write the image in the first partition with your new one.
If you support multiple images on your web server, make sure you update the correct one or your deployed unis may boot and do something unexpected. Reasons for multiple images may be one base directory holding your development image and another the live image, or supporting multiple projects.

Thank you for visiting camelthorn.cloud

Home