ESP32: firmware development
New to development for the ESP32 platform firmware? This section will help you get up-and-running with firmware development.
Did you receive an ESP32 based badge at an event for which support is not yet available? Did you build your own ESP32 based device or badge? This section helps get you started with adding support for your badge.
Interested in releasing a badge using our firmware? This section explains the factory testing and provisioning features, as well as how OTA updates and other release and support releated parts of our project work.
1 - Getting started
Welcome developer! This section describes how to get your development environment set-up so that you can build the Badge.team ESP32 platform firmware for any of the supported targets.
Introduction
Our firmware has been set-up as an ESP-IDF project. The ESP-IDF is the development framework or SDK released by Espressif. Espressif is improving and updating the ESP-IDF constantly. Unfortunately these updates often introduce breaking changes. Because of this we have included the exact version of the ESP-IDF that we use as a submodule in our firmware GIT repository.
Downloading the project
You can clone the project by running git clone https://github.com/badgeteam/ESP32-platform-firmware.git --recursive. Git will then create a folder called “ESP32-platform-firmware” containing the project files.
Installing required packages (Linux only)
Debian / Ubuntu: sudo apt-get install make unzip git libncurses5-dev flex bison gperf python-serial libffi-dev libsdl2-dev libmbedtls-dev perl
Currently we’re using ESP-IDF version 3.2 to build the firmware. After cloning our repository you will find the exact version of the ESP-IDF used in the ESP32-platform-firmware/esp-idf folder. You don’t need to download or install a version of the ESP-IDF yourself.
The toolchain is another story: the newest ESP-IDF version (v4.x and newer) uses a different toolchain than the older (v3.3 / stable) version of the IDF. Because of this you can’t simply download the “latest greatest” ESP32 toolchain, instead you need to use a specific version.
You can download the correct version of the toolchain directly from Espressif using the following URLs:
You can find the official toolchain installation instructions here:
The very, very short version of these instructions for Linux is as follows:
- Extract the archive
- Add the path to the
bin folder in the archive, containing the toolchain executables, to your path export PATH="$PATH:/path/to/my/toolchain/xtensa-esp32-elf/bin"
Configuring the firmware
The firmware can be built for many different targets. Because of this the firmware has to be configured before it can be built. By default a “generic” configuration is used. Building the firmware using the generic configuration will get you to a Python shell on any ESP32 based device. However: almost all hardware support will be missing.
To apply the configuration for a specific badge navigate to the firmware/configs folder and overwrite (/create) the file firmware/sdkconfig with the configuration relevant to your badge.
(Note that running make clean to remove the build output is a bit broken / insufficient at the moment. Please remove the firmware/build folder manually after switching configurations to make sure the firmware is built correctly.)
Manually changing the configuration of the firmware is explained in the menuconfig section.
Building the firmware
After you’ve downloaded the firmware, applied the correct configuration and installed the correct toolchain you have to build the Micropython cross compiler. This extra compiler converts Python scripts into a Micropython specific binary format so that these Python scripts can be integrated inside the firmware image.
Building the Micropython cross compiler can be done by running bash mpy_cross.sh from inside the firmware folder.
After you’ve built the Micropython cross compiler you can build the firmware by navigating to the firmware folder and running make. On multi-core systems compilation speeds can be improved by adding the number of threads to be used: make -j 4.
1.1 - Menuconfig
You can start menuconfig either by running ./config.sh in the root of the repository or by executing make menuconfig in the firmware folder.
You will then be greeted by the main menu.
The menu contains the following submenus:
- SDK tool configuration
- Bootloader config
- Security features
- Serial flasher config
- Firmware & device configuration
- Partition table
- Compiler options
- Component config
In the SDK tool configuration menu you can configure the compiler toolchain path and prefix and the Python 2 interpretter to use. The default settings configured here use the toolchain found in your $PATH and the python2 executable found in your path as the Python interpretter. You should not have to change these settings.
Bootloader config
The bootloader config menu allows configuration of the bootloader. Changing these settings requires advanced knowledge of the ESP32 platform. The default values configured here should work.
Security features
The security features menu allows for configuring secure boot by encrypting the flash and signing the firmware. Use of these features on a badge would defeat the purpose of a hackable device and is thus not supported. Do not attempt to enable any of the options in this menu: you will brick your device!
Serial flasher config
This is the first interesting item in the list. In the serial flasher config menu you can configure the serial port to use when executing make flash, as well as the baudrate. This menu also allows you to tell the bootloader about the flash chip mode, speed an size. Most of the Badge.team badges have a 16MB flash chip, the CampZone2019 has a 8MB chip and most chinese boards have 4MB.
Firmware & device configuration
This menu allows you to configure the identity of your device.
| Item | Description |
|---|
| Code-name of the firmware | Name of your device, lowercase and without spaces or special characters |
| Build number of the firmware | Version information in the format “YYMMDDNN”: year, month, day and build |
| Name of the device | Human readable name of the device |
| MicroPython modules directory | subdirectory of /firmware/python_modules to use for the built-in MicroPython modules |
| Name of the badge on the app hatchery | Name of your device or a compatible device supported by our Hatchery, lowercase and without spaces or special characters |
| Hostname of server for OTA updates | Domain name of the server used for OTA updating (Example: “Badge.team”) |
| Use HTTPS for OTA updates | If enabled HTTPS can be used with a Letsencrypt SSL certificate. Other certificate authorities are not supported. |
| Port of server for OTA updates | Port to use for OTA updates. Usually 443 for HTTPS or 80 for HTTP |
| Path on the server for OTA updates | Path of the OTA firmware binary on the server, starting with a / |
| Path on the server for OTA update version | Path to the JSON file with version information (used by the update check Python app shipped with some badges) |
| Hostname of server for app hatchery that contains user apps | Domain name at which the Hatchery is hosted (used by the installer app shipped with some badges) |
| Default WiFi ssid | The default WiFi SSID to use when the user has not yet configured WiFi |
| Default WiFi password | The default WiFi password to use when the user has not yet configured WiFi (leave empty for ipen/insecure network) |
| Default display orientation | For badges which use the same display as another type of badge but in a different orientation (explained below) |
The HackerHotel 2019 badge is a direct derrivative of the SHA2017 badge, but with the display mounted in portrait mode instead of landscape. To allow for backwards compatibility with SHA2017 apps the real orientation has been set to landscape, while HackerHotel 2019 apps can call import orientation; orientation.default() to switch to the real orientation of the badge they are running on. The default orientation is configured here.
Partition table
In this menu a partition table can be selected. A set of partition tables has been provided in the /firmware/partitions folder. The partitions.ods spreadsheet can help you when designing a custom partition table.
The partition table offset and the MD5 checksum options should be left at their default settings.
Compiler options
Advanced options for compiler optimization level and assertions. We suggest leaving the assertions enabled.
Component config
The component config submenu allows for configuring various components fo the firmware such as the MicroPython interpretter and the device drivers.
MicroPython configuration
To-do
Device driver configuration
The Badge.team firmware contains drivers for multiple devices such as displays and sensors. These drivers are written in C and part of the firmware itself, but they can be accessed from withing MicroPython using the bindings provided.
Is a menu empty? Does a feature not work?
- To be able to use I2C based devices you have to enable the I2C bus driver first.
- To be able to use SPI based devices you have to enable the VSPI driver first.
- To be able to access displays using the “display” API from within MicroPython you have to enable the framebuffer driver.
- The “double buffered” mode of the framebuffer driver is only relevant for devices which do not have their own buffer such as the Flipdot and HUB75 drivers. In all other cases it’s a waste of memory! Only enable this to use the
display.flush() command with displays that stream directly from the framebuffer.
2.1 - Adding drivers
If you need low-level support for hardware that isn’t available yet, you can write your own drivers, and can expose them to the Python app layer.
- Create the folder
/firmware/components/driver_<category>_<name>. - In this folder, create files
component.mk, Kconfig, and driver_<name>.c. Kconfig allows you to add configurable switches and variables from the ./config.sh step. The driver source file exposes an initialisation function that will be called upon boot. Have a look at e.g. /firmware/components/driver_bus_i2c to see how to populate these files. - In
/main/platform.c:platform_init(), add INIT_DRIVER(<name>) to have your driver actually initialise during boot. - Add your driver’s header directory to
firmware/micropython/component.mk, e.g. MP_EXTRA_INC += -I$(PROJECT_PATH)/components/driver_<category>_<name>/include. - Add python bindings to your driver by creating
components/micropython/esp32/mod<name>.c (see e.g. modi2c.c). - Tell micropython about your bindings by adding the following to
firmware/micropython/component.mk:
ifdef CONFIG_DRIVER_<NAME>_ENABLE
SRC_C += esp32/mod<name>.c
endif
- Add the following to
components/micropython/esp32/mpconfigport.h to add the module symbols to the python environment (replace i2c with your name):
#ifdef CONFIG_DRIVER_I2C_ENABLE
extern const struct _mp_obj_module_t i2c_module;
#endif
#ifdef CONFIG_DRIVER_I2C_ENABLE
#define BUILTIN_MODULE_I2C { MP_OBJ_NEW_QSTR(MP_QSTR_i2c), (mp_obj_t)&i2c_module },
#else
#define BUILTIN_MODULE_I2C
#endif
(to the define called MICROPY_PORT_BUILTIN_MODULES, add the following line after the other drivers):
BUILTIN_MODULE_I2C \