ESP32_Serial_Swiss_Army_Knife/README.md

# ESP32 Serial Swiss Army Knife

![ESP32 Serial Swiss Army Knife logo](/images/logo.png "ESP32 Serial Swiss Army Knife")

ESP32-S3 firmware for a secure, multi-transport RS-232 adapter. It operates one MAX3243-backed UART1 serial port and safely shares it between native USB CDC-ACM, an HTTPS/WebSocket browser terminal, and SSH: one connected client can write while others observe. The firmware also provides persistent serial and Wi-Fi configuration, a UART0 recovery console, and hardware diagnostics; it is not a general-purpose router, captive portal, or unauthenticated TCP serial server.

## Initial hardware target

- ESP32-S3-DevKitC-1-compatible development board
- ESP32-S3-WROOM-1-N16R8 module
- 16 MB flash
- 8 MB octal PSRAM
- Adafruit MAX3243 full-pinout RS-232 breakout; the male connector version is preferred (see [Hardware wiring](docs/wiring.md) for supported connector variants)

## Development status

Hardware characterization, serial/USB/Wi-Fi/HTTPS/SSH and local display/control are implemented and hardware-validated. **Phase 8 role-based users and administration is complete:** 8A–8C were target-hardware validated and the user explicitly signed off tested firmware at **8D.22 (2026-09-13)**. See the [roadmap](docs/roadmap.md#phase-8--role-based-users-and-administrative-access--complete) and its [consolidated acceptance evidence](docs/roadmap.md#phase8-acceptance-evidence). Very low internal/DMA lifetime minima remain a nonblocking headroom follow-up, not an approved reserve. The reduced Phase 9 focuses on intermittent trusted-network use, standard no-dump/silent-panic defaults and a short [operational checklist](docs/security_operations.md), without dependency patches or upgrades. **The reduced Phase 9 is complete by user sign-off on 2026-09-18.** The user waived a new device check based on prior Phase 8 validation; application code is unchanged, but crash-diagnostic defaults changed. No new hardware or panic-test pass is claimed.

### Browser administration

Cookie login/logout supports both roles. Administrators also have an explicitly opened Admin shell sharing the canonical dispatcher, and typed **Serial, Accounts/password/SSH authorized keys, Network, Display, Broker, SSH, and HTTPS/Reboot** settings. Serial/Wi-Fi/client/writer quick controls reuse the same settings controllers. Navigation preserves terminal sessions and serial writer ownership; explicit transfer is confirmed and generation-safe. Browser-shell permissions remain narrower than typed Settings or UART0/admin SSH.

Network uses UTF-8 text/reversible hex SSIDs, explicit password Keep/Replace/disabled-STA Clear (never AP clear), RAM edits and explicit Save. Saved secrets are never prefilled/exported. Next profile follows enabled priority order, not the editor's selected index. Network `accepted` means owner admission, not online/DNS completion; disruption may precede acknowledgement.

HTTPS and SSH identity settings display public fingerprints and confirm both service and identity generations before rotation. HTTPS rotation commits before stop/restart and invalidates web logins; SSH stops before commit/restart and can disconnect clients even when persistence fails. A committed identity is never rolled back on restart failure. A lost response or timeout is not cancellation: inspect state rather than automatically replaying. Verify changed trust on UART0 with `web certificate info` or `ssh host-key info` before renewing browser trust/known_hosts; HTTPS restart requires fresh login. Browser identity reset/recovery/export is excluded; canonical CLI recovery remains available.

Keep UART0 ready for administrative recovery and native USB for network-independent UART1 access. USB is not an admin console, and whole-device reboot interrupts every transport. [Web administration contracts](docs/web_administration.md) cover API bounds, ownership, permissions, uncertainty and recovery. Dedicated typed network diagnostics and the unimplemented browser-session/USB control expansion were removed from scope; existing shell diagnostics and SSH settings remain.

## Documentation

- [Hardware wiring](docs/wiring.md): hardware profile, GPIO assignments, connector guidance, and safety notes.
- [Electrical tests](docs/electrical_tests.md): OLED/buttons, MAX3243, UART loopback, and session-broker verification procedures.
- [Role-based user database and UART0 administration](docs/user_administration_tests.md): user provisioning and administration, HTTPS/SSH authentication, session revocation, and reusable integrated web-administration regression procedures (not execution evidence).
- [Web administration](docs/web_administration.md): current bounded API/owner and recovery contracts; [roadmap acceptance evidence](docs/roadmap.md#phase8-acceptance-evidence) records sign-off and telemetry limits.
- [Security operations](docs/security_operations.md): trusted-network use, credentials, identity verification, shutdown and recovery; physical-extraction limits and lightweight upstream maintenance.
- [Command reference](docs/command_reference.md): UART0/admin-SSH administration, serial, broker, USB, Wi-Fi, mDNS, web, SSH, and diagnostic commands.

## Flash partition layout

The N16R8 target has 16 MiB flash and 8 MiB octal PSRAM. PlatformIO uses the custom [`partitions.csv`](partitions.csv) layout:

| Partition | Offset | Size | Purpose |
|---|---:|---:|---|
| `nvs` | `0x009000` | 512 KiB | Serial, Wi-Fi, mDNS hostname, local-display, role-based user, HTTPS identity, and SSH host-key data |
| `otadata` | `0x089000` | 8 KiB | Active OTA-slot selection metadata |
| `phy_init` | `0x08B000` | 4 KiB | Optional PHY initialization data |
| `nvs_key` | `0x08C000` | 4 KiB | Reserved for future encrypted-NVS keys |
| `coredump` | `0x08D000` | 128 KiB | Reserved for flash core dumps |
| `ota_0` | `0x0B0000` | 4 MiB | Primary application/OTA slot |
| `ota_1` | `0x4B0000` | 4 MiB | Alternate application/OTA slot |
| `storage` | `0x8B0000` | 7488 KiB | Reserved for future LittleFS web assets, logs, and files |

Application offsets are aligned to the ESP32-S3's required 64 KiB boundary. The final storage partition ends at `0x1000000`, exactly the end of the 16 MiB flash chip.

The table reserves OTA and storage space; it does not implement OTA downloads, rollback confirmation, core-dump handling, NVS encryption, or filesystem mounting.

### One-time migration from the default partition table

The previous 1 MiB factory application began at `0x10000`, which now lies inside the enlarged NVS range. A normal upload does not erase stale data in that range. When first switching to this layout, erase the flash completely:

```sh
pio run --target erase
pio run --target upload
pio device monitor -b 115200
```

This removes saved serial configuration and all other flash contents. The firmware recreates NVS with safe defaults. Subsequent ordinary uploads do not need a full erase.

## Build

```sh
pio run
```

The small Phase 9 sets `CONFIG_ESP_COREDUMP_ENABLE_TO_NONE=y` and `CONFIG_ESP_SYSTEM_PANIC_SILENT_REBOOT=y` in `sdkconfig.defaults`. An existing generated `sdkconfig.esp32-s3-devkitc-1-n16r8` can retain earlier selections: check its resolved values after configuration, or change them using `pio run --target menuconfig`. Defaults alone do not enforce existing configurations. Silent panic reboot removes panic register/backtrace output; ordinary logs remain. No existing flash contents are erased, and stored secrets remain unencrypted.

## Upload and monitor

Connect the board's **USB-to-UART** port for firmware upload and the UART0 development console, then run:

```sh
pio run --target upload
pio device monitor -b 115200
```

The firmware provides an interactive UART0 console at `serial-tool>`. Run `help` for available commands. The USB-to-UART bridge normally appears as `/dev/ttyUSB*`; it is separate from the native USB CDC serial transport, which normally appears as `/dev/ttyACM*`.

The console supports session history, line editing, cursor movement, and hierarchical Tab completion. After an unattended boot, attach an ANSI-capable terminal and press Enter once to enable enhanced editing; this avoids blocking while no terminal is attached.

Serial, Wi-Fi, and mDNS hostname edits remain in RAM until explicitly saved with `serial save`, `wifi save`, or `mdns save`. Authenticated admin SSH sessions expose the shared operational administration registry, including interactive secrets, TLS/SSH identity management, network diagnostics, and deferred reboot/SSH lifecycle commands. Create the first administrator on UART0 with `user add <username> admin` (optionally `--generate`). Explicit recovery of an unavailable user database remains UART0-only and rebuilds it empty; it refuses a healthy database. An administrator also cannot generate a replacement password for its own account over SSH, preventing the one-time value from being lost when that mutation revokes the session. Legacy web credential commands and `user bootstrap` are removed.

## Security notes

The HTTPS interface uses a device-specific self-signed certificate and a same-origin login page with bounded server-side cookie sessions; HTTP Basic is no longer accepted. Open `/` or `/login`, sign in with a user-database password, and use **Sign out** before switching accounts. Four sessions have a one-hour absolute lifetime, including active serial connections; logout closes only that session's serial access. Login is globally limited to five credential verifications per 60 seconds, with explicit capacity/backoff errors. Direct-IP and mDNS access use separate host-only Secure/HttpOnly/SameSite=Strict cookies. Non-browser clients also require cookies, strict Origin and CSRF for mutations rather than Basic credentials. There is no plaintext HTTP or TCP serial listener. SSH accepts role-based passwords and authorized Ed25519/ECDSA P-256 public keys. User passwords are stored as salted PBKDF2-HMAC-SHA256 verifiers, but the HTTPS private key, SSH private key, and Wi-Fi credentials remain recoverable from unencrypted application-owned NVS blobs. Offline password guessing and stale append-oriented flash copies also remain possible. The reserved `nvs_key` partition does not enable encryption. This firmware is not resistant to physical flash or RAM extraction; the completed reduced Phase 9 deliberately does not add that protection.

## License

This project is licensed under the [GNU General Public License version 3 only](LICENSE) (`GPL-3.0-only`). Third-party components remain subject to their respective licenses. The integration baseline uses Espressif registry components `espressif/mdns` `1.12.0`, `wolfssl/wolfssl` `5.8.2~1`, and `wolfssl/wolfssh` `1.4.20`; review upstream security releases before production use.

### Legacy credential removal

Missing user storage is persisted as an empty database; no shared credential is imported or synchronized. Existing valid v1 user records retain their accounts, roles, IDs, verifiers and keys without a schema change. HTTPS `web_sec/material` upgrades valid 1,392-byte v1 storage to 1,340-byte TLS-only v2, retaining exact certificate/key DER, fingerprint and generation, and committing before publication. Invalid records or migration failures fail closed rather than triggering fallback replacement. `web certificate rotate --force` remains available; `web reset --force` replaces TLS identity only, not users.

**Downgrade warning:** older v1-only firmware cannot read v2 HTTPS material. Logical NVS replacement is not a secure flash wipe; historical plaintext credentials can remain in flash. This cleanup requires no factory/partition erase. See [consolidated compatibility and evidence limits](docs/roadmap.md#phase8-legacy-credential-compatibility); Phase 8 acceptance is recorded there.