# ESP32 Serial Swiss Army Knife ![ESP32 Serial Swiss Army Knife logo](/images/logo.png "ESP32 Serial Swiss Army Knife") Universal wireless serial adaptor firmware for the ESP32-S3. ## Initial hardware target - ESP32-S3-DevKitC-1-compatible development board - ESP32-S3-WROOM-1-N16R8 module - 16 MB flash - 8 MB octal PSRAM - Adafruit MAX3243 full-pinout RS-232 breakout, product 5988 The firmware has completed **Phase 0 hardware characterization**, the **Phase 1 serial-core foundation**, the **Phase 2 transport-neutral session broker**, and the first real broker transport through native USB CDC-ACM. The current phase adds persistent multi-profile Wi-Fi station configuration and managed AP fallback/AP+STA operation. The MAX3243 diagnostics and recovery consoles remain available. No electrical test starts automatically; UART1 starts when requested explicitly or when a host opens native USB CDC. ## Hardware wiring See [`wiring.md`](wiring.md) for the hardware profile, GPIO assignments, loopback diagrams, safety notes, and the recommended test sequence. The initial profile covers the ESP32-S3-DevKitC-1 N16R8 and the Adafruit MAX3243 full-pinout RS-232 breakout. ## Flash partition layout The N16R8 target has 16 MiB flash and 8 MiB octal PSRAM. PlatformIO uses the custom [`partitions.csv`](partitions.csv) layout: | Partition | Offset | Size | Purpose | |---|---:|---:|---| | `nvs` | `0x009000` | 512 KiB | Serial configuration and future Wi-Fi/provisioning data | | `otadata` | `0x089000` | 8 KiB | Active OTA-slot selection metadata | | `phy_init` | `0x08B000` | 4 KiB | Optional PHY initialization data | | `nvs_key` | `0x08C000` | 4 KiB | Reserved for future encrypted-NVS keys | | `coredump` | `0x08D000` | 128 KiB | Reserved for flash core dumps | | `ota_0` | `0x0B0000` | 4 MiB | Primary application/OTA slot | | `ota_1` | `0x4B0000` | 4 MiB | Alternate application/OTA slot | | `storage` | `0x8B0000` | 7488 KiB | Future LittleFS web assets, certificates, logs, and files | Application offsets are aligned to the ESP32-S3's required 64 KiB boundary. The final storage partition ends at `0x1000000`, exactly the end of the 16 MiB flash chip. The partition table reserves OTA and LittleFS space but does not by itself implement OTA downloads, rollback confirmation, core-dump handling, NVS encryption, or filesystem mounting. Those features will be enabled deliberately in later phases. ### One-time migration from the default partition table The previous 1 MiB factory application began at `0x10000`, which is now inside the enlarged NVS address range. A normal upload does not erase all stale bytes there. Perform a full flash erase once when first switching to this layout: ```sh pio run --target erase pio run --target upload pio device monitor -b 115200 ``` This erases the currently saved serial configuration and all other flash contents. The firmware will boot with safe serial defaults and recreate NVS. Subsequent ordinary uploads do not require another full erase. PlatformIO's application-size report should now use the 4 MiB `ota_0` slot instead of the previous 1 MiB factory partition. ## Build ```sh pio run ``` ## Upload and monitor Connect the board's **USB-to-UART** port for firmware upload and the UART0 development console, then run: ```sh pio run --target upload pio device monitor -b 115200 ``` The firmware starts an interactive console on UART0 with the prompt `serial-tool>`. Type `help` to display command descriptions. This USB-to-UART device normally appears as `/dev/ttyUSB*`; it is separate from the native USB CDC serial transport described below. ### Phase 1 serial service The `serial` command manages the working configuration and UART1 service: ```text serial status serial start serial stop serial set serial save serial load serial defaults serial reset serial counters serial clear-counters ``` Safe defaults are 115200 baud, 8 data bits, no parity, one stop bit, no flow control, and inactive DTR. Supported configuration values are: | Parameter | Values | |---|---| | `baud` | 110–1000000 | | `data-bits` | `7`, `8` | | `parity` | `none`, `even`, `odd` | | `stop-bits` | `1`, `2` | | `flow` | `none`, `rts-cts` | | `dtr` | `inactive`, `active`, `on-connect` | | `rts-threshold` | 1–127 bytes | `serial set` changes the working configuration and safely restarts UART1 if the service is running. It does not write flash; use `serial save` to commit the current configuration to NVS. `serial defaults` changes RAM only, while `serial reset` applies and persists defaults. The firmware never erases the shared NVS partition automatically when storage is incompatible or unavailable. The service uses independent software RX and TX streams. Calls into those streams are nonblocking, and a deasserted CTS cannot block service shutdown. UART data access is intentionally reserved for the session broker; the `serial` command controls configuration and lifecycle only. UART1 has exclusive ownership while the service runs. Phase 0 commands will refuse to touch the port until `serial stop` releases it. ### Phase 2 session broker The broker is initialized at boot and continuously drains the serial service whenever UART1 is running. It is transport-neutral: console test clients and native USB CDC use the same API that WebSocket and SSH transports will use later. ```text broker status broker clients broker counters broker clear-counters broker connect broker disconnect broker request-writer broker release-writer broker force-writer broker send-hex broker read [maximum-bytes] broker events ``` Each connection receives a generation-safe numeric ID. Stale IDs from disconnected clients cannot address a newly reused slot. Up to eight clients may connect, each with a bounded 4096-byte output queue and a 16-entry event queue. UART RX is copied to every connected client. A full observer queue drops bytes only for that observer and records the loss; it never blocks UART reception or another client. With no clients, the broker still drains UART data and records it as unobserved. Exactly one client may hold the writer lease. Competing requests are denied and generate events. Administrative forced reassignment atomically revokes the old writer and grants the new one. Bytes already accepted before revocation remain queued for transmission; revocation prevents future admission rather than purging the UART TX stream. Connect, disconnect, writer grant, release, revoke, and denial events have a broker-global sequence number. Event queues are intentionally bounded, so future transports should reconcile sequence gaps against broker snapshots. The first and last broker connection also drive the Phase 1 `DTR=on-connect` policy. ### Native USB CDC-ACM transport The ESP32-S3's native USB OTG peripheral presents one CDC-ACM serial interface through the development board's connector labelled **USB**. It uses GPIO19 (`USB D-`) and GPIO20 (`USB D+`) and normally appears on Linux as `/dev/ttyACM*`. It is not the USB-to-UART bridge used for upload and logs. The UART0 development console provides these diagnostics and controls: ```text usb status usb counters usb clear-counters usb request-writer usb release-writer ``` Opening the CDC port with DTR asserted automatically starts UART1, connects a broker client named `usb-cdc`, and requests the writer lease. If another client already owns the lease, USB remains connected as a read-only observer; `usb status` reports its current role. Closing the port or unplugging native USB disconnects that broker client and discards transport-local pending data. The serial service itself remains running until it is stopped explicitly with `serial stop`. The data path is binary-transparent. UTF-8 bytes, NUL bytes, terminal escape sequences, and color sequences are passed unchanged; interpretation remains the terminal application's responsibility. USB output is bounded and nonblocking, so a host that stops reading can lose only its own observer data rather than stall UART1 or another client. Host line coding is accepted for baud rates 110–1000000 with 7 or 8 data bits, none/odd/even parity, and 1 or 2 stop bits. USB's 1.5 stop bits and mark/space parity are rejected. Supported settings are applied to the working UART configuration only when USB owns the writer lease and queued UART TX has drained. They are not saved to NVS automatically; use `serial save` deliberately if the setting should survive reboot. USB RTS is reported as host status only. It does not drive the physical RS-232 RTS line, which remains controlled by UART1's configured RTS/CTS flow control. The development VID/PID comes from Espressif's TinyUSB defaults. The USB serial-number string is derived from the ESP32-S3 station MAC so multiple adapters can be distinguished consistently. #### Linux loopback validation Keep the USB-to-UART cable connected for logs and commands, and connect a second data-capable cable to the native **USB** connector. On the host, identify the new CDC device: ```sh dmesg ls -l /dev/ttyACM* ``` With power removed and no external RS-232 peer attached, connect only DE-9 pin 3 (`TX`) to pin 2 (`RX`), then power the board. Open the native port with a serial terminal such as: ```sh picocom -b 115200 /dev/ttyACM0 ``` Use the actual device path assigned by the host. Typed data should return through USB → broker → UART1 → MAX3243 loopback → broker → USB. On the UART0 console, verify `usb status`, `usb counters`, `broker clients`, and `serial status`. The USB client should normally be the writer and counters should increase without drops. For a binary check, install PySerial on the host and send all byte values: ```python import serial payload = bytes(range(256)) with serial.Serial("/dev/ttyACM0", 115200, timeout=2) as port: port.reset_input_buffer() port.write(payload) echoed = port.read(len(payload)) assert echoed == payload, (len(echoed), echoed.hex()) print("256-byte binary USB/RS-232 loopback passed") ``` Close the terminal and check `usb status` and `broker clients`; DTR-aware applications should cause the USB broker client to disconnect. Physically unplugging the native USB cable is the definitive detach test. To test observer mode, assign a console test client as writer before opening `/dev/ttyACM0`; USB should connect as an observer, receive UART output, and discard host-originated input until ownership is granted. Power down and remove the DE-9 pin 3-to-2 jumper before connecting an external serial peer. ### Wi-Fi foundation Wi-Fi is managed independently of the serial-session broker in this phase. It provides network connectivity and recovery access-point policy for the later HTTPS/WebSocket transport, but it does not yet run HTTP, HTTPS, DNS interception, a captive portal, NAPT, or any TCP serial listener. Configuration uses four fixed station-profile slots. Lower numeric priority values are tried first, with slot number breaking ties. Profiles support WPA2/WPA3 mixed operation or require WPA3-SAE. ESP-IDF's station threshold can express “WPA2 or stronger” but not a strict WPA2-only maximum, so the configuration does not pretend to offer a distinct WPA2-only mode. Each profile attempt has a 12-second association/DHCP deadline. After all enabled profiles fail, the manager uses exponential retry delays from 2 to 60 seconds. AP policy is independent of the station profiles: | Policy | Behavior | |---|---| | `off` | Station only; never start the fallback AP | | `fallback` | Start the AP immediately when no profiles exist, or after one failed profile cycle; disable it after station connectivity has remained stable for 30 seconds | | `always` | Keep AP and station active concurrently | Fresh defaults enable Wi-Fi with `fallback` policy, AP channel 6, a MAC-suffixed SSID such as `ESP32-SAK-A1B2C3`, and a randomly generated 16-character password. The initial random credential is saved to NVS automatically when possible so it remains stable across reboot. Retrieve it deliberately from the physical UART0 administration console with `wifi ap show-secret`. The `wifi` command provides: ```text wifi status wifi profiles wifi start|stop|reconnect wifi profile set wifi profile secret wifi profile enable|disable|delete wifi ap policy wifi ap ssid wifi ap channel <1..11> wifi ap secret|show-secret wifi save|load|defaults|reset wifi counters|clear-counters ``` Ordinary status and profile output never displays passwords. `wifi profile secret` and `wifi ap secret` read through a dedicated no-echo UART0 prompt, keeping credentials out of the command line and its history. SSIDs containing spaces can be quoted. Profile and AP edits apply to the working RAM configuration and restart Wi-Fi asynchronously if it is running; use `wifi save` explicitly to persist them. `wifi start` and `wifi stop` also change the working `enabled-at-boot` setting, which becomes persistent only after `wifi save`. A typical station setup is: ```text wifi profile set 0 10 mixed "your SSID" wifi profile secret 0 wifi profile enable 0 wifi save wifi reconnect wifi status ``` The fallback AP uses Espressif's default `192.168.4.1/24` network for now. AP clients receive addresses through its DHCP server but are not routed to the station network. ESP32-S3 has one 2.4 GHz radio, so in AP+STA mode the AP follows the connected station's channel. Station connection attempts and scans can temporarily increase AP latency, and clients can briefly reconnect when the channel moves. Wi-Fi credentials currently reside as plaintext in the application-owned `wifi_app/config` NVS blob. Selecting `WIFI_STORAGE_RAM` prevents the ESP-IDF driver from creating a second persistent credential copy, but it does not encrypt the application's blob. The reserved `nvs_key` partition alone does not enable encryption. NVS encryption, secure boot, flash encryption, and core-dump credential exposure require a deliberate later security phase. #### Wi-Fi validation 1. Boot with no station profiles. `wifi status` should report `ap-only`, and the generated SSID should be visible from another device. 2. Use `wifi ap show-secret`, join the AP, confirm a `192.168.4.x` lease, and ping `192.168.4.1`. No web page is expected yet. 3. Configure and enable a WPA2/WPA3 station profile using the example above. `wifi status` should progress through `connecting`, `waiting-ip`, and `online` and display the acquired address, channel, RSSI, and negotiated authentication. 4. Reboot and verify profile and AP credential persistence. 5. Configure two profiles with different priorities, make the first unavailable, and verify failover to the second after its timeout. 6. Make all profiles unavailable and verify fallback AP startup plus increasing retry delays in `wifi status`/`wifi counters`. 7. Test `wifi ap policy always` while online and confirm both interfaces remain available; expect the AP channel to follow the station. 8. Test `wifi stop`, `wifi start`, and `wifi reconnect` while confirming UART0 and native USB serial operation remain unaffected. 9. If available, test a WPA3-only profile and a wrong password, then inspect the disconnect reason and counters. ### Phase 0 diagnostics The retained hardware-characterization commands are: ```text status transceiver drivers loopback-a loopback-b valid-test uart-loopback [8N1|8E1|8O1|8N2|7E1|7O1] [bytes] uart-suite cts-flow-test rts-flow-test ``` `uart-loopback` defaults to `8N1` and 256 bytes. Its accepted payload range is 1–512 bytes. `uart-suite` covers 300 through 250000 baud and all supported frame formats. `cts-flow-test` verifies transmit gating and exact resumption, while `rts-flow-test` uses UART2 as an internal traffic generator to verify automatic receive backpressure. Follow the command-specific loopback wiring in [`wiring.md`](wiring.md) before invoking any test. A mutex-protected port lease prevents diagnostics, UART1 service startup, and future clients from reconfiguring the same GPIOs concurrently. If a UART driver cannot be removed during cleanup, the firmware keeps the MAX3243 shut down and marks the port faulted until reboot rather than exposing an ambiguous hardware state. The onboard RGB LED reports the most recent test-harness state: | Color | Meaning | |---|---| | Blue | Idle; waiting for a command | | Yellow/orange | Test running | | Green | Last test passed | | Red | Last test failed | This hardware profile uses the onboard RGB LED on GPIO48. Official ESP32-S3-DevKitC-1 v1.1 boards commonly use GPIO38 instead, and compatible boards or clones may vary. A different board revision requires an adjusted board pin profile before running this firmware. ## License This project is licensed under the [GNU General Public License version 3 only](LICENSE) (`GPL-3.0-only`). This is compatible with using the GPLv3 releases of wolfSSL and wolfSSH later. Third-party components remain subject to their respective licenses.