ESP32 Serial Swiss Army Knife

Universal wireless serial adaptor firmware for the ESP32-S3.

Initial hardware target

• ESP32-S3-DevKitC-1-compatible development board
• ESP32-S3-WROOM-1-N16R8 module
• 16 MB flash
• 8 MB octal PSRAM
• Adafruit MAX3243 full-pinout RS-232 breakout, product 5988

The firmware has completed Phase 0 hardware characterization, the Phase 1 serial-core foundation, and the Phase 2 transport-neutral session broker. The current phase adds the first real broker transport: native USB CDC-ACM on the ESP32-S3 USB port. The MAX3243 diagnostics and persistent serial configuration remain available. No electrical test starts automatically; UART1 starts when requested explicitly or when a host opens native USB CDC.

Hardware wiring

See wiring.md for the hardware profile, GPIO assignments, loopback diagrams, safety notes, and the recommended test sequence. The initial profile covers the ESP32-S3-DevKitC-1 N16R8 and the Adafruit MAX3243 full-pinout RS-232 breakout.

Flash partition layout

The N16R8 target has 16 MiB flash and 8 MiB octal PSRAM. PlatformIO uses the custom partitions.csv layout:

Partition	Offset	Size	Purpose
nvs	0x009000	512 KiB	Serial configuration and future Wi-Fi/provisioning data
otadata	0x089000	8 KiB	Active OTA-slot selection metadata
phy_init	0x08B000	4 KiB	Optional PHY initialization data
nvs_key	0x08C000	4 KiB	Reserved for future encrypted-NVS keys
coredump	0x08D000	128 KiB	Reserved for flash core dumps
ota_0	0x0B0000	4 MiB	Primary application/OTA slot
ota_1	0x4B0000	4 MiB	Alternate application/OTA slot
storage	0x8B0000	7488 KiB	Future LittleFS web assets, certificates, logs, and files

Application offsets are aligned to the ESP32-S3's required 64 KiB boundary. The final storage partition ends at 0x1000000, exactly the end of the 16 MiB flash chip.

The partition table reserves OTA and LittleFS space but does not by itself implement OTA downloads, rollback confirmation, core-dump handling, NVS encryption, or filesystem mounting. Those features will be enabled deliberately in later phases.

One-time migration from the default partition table

The previous 1 MiB factory application began at 0x10000, which is now inside the enlarged NVS address range. A normal upload does not erase all stale bytes there. Perform a full flash erase once when first switching to this layout:

pio run --target erase
pio run --target upload
pio device monitor -b 115200

This erases the currently saved serial configuration and all other flash contents. The firmware will boot with safe serial defaults and recreate NVS. Subsequent ordinary uploads do not require another full erase.

PlatformIO's application-size report should now use the 4 MiB ota_0 slot instead of the previous 1 MiB factory partition.

Build

pio run

Upload and monitor

Connect the board's USB-to-UART port for firmware upload and the UART0 development console, then run:

pio run --target upload
pio device monitor -b 115200

The firmware starts an interactive console on UART0 with the prompt serial-tool>. Type help to display command descriptions. This USB-to-UART device normally appears as /dev/ttyUSB*; it is separate from the native USB CDC serial transport described below.

Phase 1 serial service

The serial command manages the working configuration and UART1 service:

serial status
serial start
serial stop
serial set <baud|data-bits|parity|stop-bits|flow|dtr|rts-threshold> <value>
serial save
serial load
serial defaults
serial reset
serial counters
serial clear-counters

Safe defaults are 115200 baud, 8 data bits, no parity, one stop bit, no flow control, and inactive DTR. Supported configuration values are:

Parameter	Values
baud	110–1000000
data-bits	7, 8
parity	none, even, odd
stop-bits	1, 2
flow	none, rts-cts
dtr	inactive, active, on-connect
rts-threshold	1–127 bytes

serial set changes the working configuration and safely restarts UART1 if the service is running. It does not write flash; use serial save to commit the current configuration to NVS. serial defaults changes RAM only, while serial reset applies and persists defaults. The firmware never erases the shared NVS partition automatically when storage is incompatible or unavailable.

The service uses independent software RX and TX streams. Calls into those streams are nonblocking, and a deasserted CTS cannot block service shutdown. UART data access is intentionally reserved for the session broker; the serial command controls configuration and lifecycle only.

UART1 has exclusive ownership while the service runs. Phase 0 commands will refuse to touch the port until serial stop releases it.

Phase 2 session broker

The broker is initialized at boot and continuously drains the serial service whenever UART1 is running. It is transport-neutral: console test clients and native USB CDC use the same API that WebSocket and SSH transports will use later.

broker status
broker clients
broker counters
broker clear-counters
broker connect <name>
broker disconnect <client-id>
broker request-writer <client-id>
broker release-writer <client-id>
broker force-writer <client-id|none>
broker send-hex <client-id> <hex-bytes>
broker read <client-id> [maximum-bytes]
broker events <client-id>

Each connection receives a generation-safe numeric ID. Stale IDs from disconnected clients cannot address a newly reused slot. Up to eight clients may connect, each with a bounded 4096-byte output queue and a 16-entry event queue.

UART RX is copied to every connected client. A full observer queue drops bytes only for that observer and records the loss; it never blocks UART reception or another client. With no clients, the broker still drains UART data and records it as unobserved.

Exactly one client may hold the writer lease. Competing requests are denied and generate events. Administrative forced reassignment atomically revokes the old writer and grants the new one. Bytes already accepted before revocation remain queued for transmission; revocation prevents future admission rather than purging the UART TX stream.

Connect, disconnect, writer grant, release, revoke, and denial events have a broker-global sequence number. Event queues are intentionally bounded, so future transports should reconcile sequence gaps against broker snapshots. The first and last broker connection also drive the Phase 1 DTR=on-connect policy.

Native USB CDC-ACM transport

The ESP32-S3's native USB OTG peripheral presents one CDC-ACM serial interface through the development board's connector labelled USB. It uses GPIO19 (USB D-) and GPIO20 (USB D+) and normally appears on Linux as /dev/ttyACM*. It is not the USB-to-UART bridge used for upload and logs.

The UART0 development console provides these diagnostics and controls:

usb status
usb counters
usb clear-counters
usb request-writer
usb release-writer

Opening the CDC port with DTR asserted automatically starts UART1, connects a broker client named usb-cdc, and requests the writer lease. If another client already owns the lease, USB remains connected as a read-only observer; usb status reports its current role. Closing the port or unplugging native USB disconnects that broker client and discards transport-local pending data. The serial service itself remains running until it is stopped explicitly with serial stop.

The data path is binary-transparent. UTF-8 bytes, NUL bytes, terminal escape sequences, and color sequences are passed unchanged; interpretation remains the terminal application's responsibility. USB output is bounded and nonblocking, so a host that stops reading can lose only its own observer data rather than stall UART1 or another client.

Host line coding is accepted for baud rates 110–1000000 with 7 or 8 data bits, none/odd/even parity, and 1 or 2 stop bits. USB's 1.5 stop bits and mark/space parity are rejected. Supported settings are applied to the working UART configuration only when USB owns the writer lease and queued UART TX has drained. They are not saved to NVS automatically; use serial save deliberately if the setting should survive reboot. USB RTS is reported as host status only. It does not drive the physical RS-232 RTS line, which remains controlled by UART1's configured RTS/CTS flow control.

The development VID/PID comes from Espressif's TinyUSB defaults. The USB serial-number string is derived from the ESP32-S3 station MAC so multiple adapters can be distinguished consistently.

Linux loopback validation

Keep the USB-to-UART cable connected for logs and commands, and connect a second data-capable cable to the native USB connector. On the host, identify the new CDC device:

dmesg
ls -l /dev/ttyACM*

With power removed and no external RS-232 peer attached, connect only DE-9 pin 3 (TX) to pin 2 (RX), then power the board. Open the native port with a serial terminal such as:

picocom -b 115200 /dev/ttyACM0

Use the actual device path assigned by the host. Typed data should return through USB → broker → UART1 → MAX3243 loopback → broker → USB. On the UART0 console, verify usb status, usb counters, broker clients, and serial status. The USB client should normally be the writer and counters should increase without drops.

For a binary check, install PySerial on the host and send all byte values:

import serial

payload = bytes(range(256))
with serial.Serial("/dev/ttyACM0", 115200, timeout=2) as port:
    port.reset_input_buffer()
    port.write(payload)
    echoed = port.read(len(payload))

assert echoed == payload, (len(echoed), echoed.hex())
print("256-byte binary USB/RS-232 loopback passed")

Close the terminal and check usb status and broker clients; DTR-aware applications should cause the USB broker client to disconnect. Physically unplugging the native USB cable is the definitive detach test. To test observer mode, assign a console test client as writer before opening /dev/ttyACM0; USB should connect as an observer, receive UART output, and discard host-originated input until ownership is granted.

Power down and remove the DE-9 pin 3-to-2 jumper before connecting an external serial peer.

Phase 0 diagnostics

The retained hardware-characterization commands are:

status
transceiver <enable|disable>
drivers <tx 0|1> <dtr 0|1> <rts 0|1>
loopback-a
loopback-b
valid-test
uart-loopback <baud> [8N1|8E1|8O1|8N2|7E1|7O1] [bytes]
uart-suite
cts-flow-test
rts-flow-test

uart-loopback defaults to 8N1 and 256 bytes. Its accepted payload range is 1–512 bytes. uart-suite covers 300 through 250000 baud and all supported frame formats. cts-flow-test verifies transmit gating and exact resumption, while rts-flow-test uses UART2 as an internal traffic generator to verify automatic receive backpressure. Follow the command-specific loopback wiring in wiring.md before invoking any test.

A mutex-protected port lease prevents diagnostics, UART1 service startup, and future clients from reconfiguring the same GPIOs concurrently. If a UART driver cannot be removed during cleanup, the firmware keeps the MAX3243 shut down and marks the port faulted until reboot rather than exposing an ambiguous hardware state.

The onboard RGB LED reports the most recent test-harness state:

Color	Meaning
Blue	Idle; waiting for a command
Yellow/orange	Test running
Green	Last test passed
Red	Last test failed

This hardware profile uses the onboard RGB LED on GPIO48. Official ESP32-S3-DevKitC-1 v1.1 boards commonly use GPIO38 instead, and compatible boards or clones may vary. A different board revision requires an adjusted board pin profile before running this firmware.

License

This project is licensed under the GNU General Public License version 3 only (GPL-3.0-only). This is compatible with using the GPLv3 releases of wolfSSL and wolfSSH later. Third-party components remain subject to their respective licenses.