Commander1024
91267b371e
Mark web administration complete, centralize current contracts and acceptance evidence, and remove superseded slice records. Update roadmap, architecture notes, and test references without changing firmware sources.
102 lines
10 KiB
Markdown
102 lines
10 KiB
Markdown
# ESP32 Serial Swiss Army Knife
|
||
|
||
|
||
|
||
ESP32-S3 firmware for a secure, multi-transport RS-232 adapter. It operates one MAX3243-backed UART1 serial port and safely shares it between native USB CDC-ACM, an HTTPS/WebSocket browser terminal, and SSH: one connected client can write while others observe. The firmware also provides persistent serial and Wi-Fi configuration, a UART0 recovery console, and hardware diagnostics; it is not a general-purpose router, captive portal, or unauthenticated TCP serial server.
|
||
|
||
## Initial hardware target
|
||
|
||
- ESP32-S3-DevKitC-1-compatible development board
|
||
- ESP32-S3-WROOM-1-N16R8 module
|
||
- 16 MB flash
|
||
- 8 MB octal PSRAM
|
||
- Adafruit MAX3243 full-pinout RS-232 breakout; the male connector version is preferred (see [Hardware wiring](docs/wiring.md) for supported connector variants)
|
||
|
||
## Development status
|
||
|
||
Hardware characterization, serial/USB/Wi-Fi/HTTPS/SSH and local display/control are implemented and hardware-validated. **Phase 8 role-based users and administration is complete:** 8A–8C were target-hardware validated and the user explicitly signed off tested firmware at **8D.22 (2026-09-13)**. See the [roadmap](docs/roadmap.md#phase-8--role-based-users-and-administrative-access--complete) and [acceptance evidence](docs/web_administration_acceptance.md). Very low internal/DMA lifetime minima remain a nonblocking headroom follow-up, not an approved reserve. Phase 9 hardening begins only on a separate request.
|
||
|
||
### Browser administration
|
||
|
||
Cookie login/logout supports both roles. Administrators also have an explicitly opened Admin shell sharing the canonical dispatcher, and typed **Serial, Accounts/password/SSH authorized keys, Network, Display, Broker, SSH, and HTTPS/Reboot** settings. Serial/Wi-Fi/client/writer quick controls reuse the same settings controllers. Navigation preserves terminal sessions and serial writer ownership; explicit transfer is confirmed and generation-safe. Browser-shell permissions remain narrower than typed Settings or UART0/admin SSH.
|
||
|
||
Network uses UTF-8 text/reversible hex SSIDs, explicit password Keep/Replace/disabled-STA Clear (never AP clear), RAM edits and explicit Save. Saved secrets are never prefilled/exported. Next profile follows enabled priority order, not the editor's selected index. Network `accepted` means owner admission, not online/DNS completion; disruption may precede acknowledgement.
|
||
|
||
HTTPS and SSH identity settings display public fingerprints and confirm both service and identity generations before rotation. HTTPS rotation commits before stop/restart and invalidates web logins; SSH stops before commit/restart and can disconnect clients even when persistence fails. A committed identity is never rolled back on restart failure. A lost response or timeout is not cancellation: inspect state rather than automatically replaying. Verify changed trust on UART0 with `web certificate info` or `ssh host-key info` before renewing browser trust/known_hosts; HTTPS restart requires fresh login. Browser identity reset/recovery/export is excluded; canonical CLI recovery remains available.
|
||
|
||
Keep UART0 ready for administrative recovery and native USB for network-independent UART1 access. USB is not an admin console, and whole-device reboot interrupts every transport. [Web administration contracts](docs/web_administration.md) cover API bounds, ownership, permissions, uncertainty and recovery. Dedicated typed network diagnostics and the unimplemented browser-session/USB control expansion were removed from scope; existing shell diagnostics and SSH settings remain.
|
||
|
||
## Documentation
|
||
|
||
- [Hardware wiring](docs/wiring.md): hardware profile, GPIO assignments, connector guidance, and safety notes.
|
||
- [Electrical tests](docs/electrical_tests.md): OLED/buttons, MAX3243, UART loopback, and session-broker verification procedures.
|
||
- [Role-based user database and UART0 administration](docs/user_administration_tests.md): user provisioning and administration, HTTPS/SSH authentication, session revocation, and reusable integrated web-administration regression procedures (not execution evidence).
|
||
- [Web administration](docs/web_administration.md): current bounded API/owner and recovery contracts; [acceptance evidence](docs/web_administration_acceptance.md) records sign-off and telemetry limits.
|
||
- [Command reference](docs/command_reference.md): UART0/admin-SSH administration, serial, broker, USB, Wi-Fi, mDNS, web, SSH, and diagnostic commands.
|
||
|
||
## Flash partition layout
|
||
|
||
The N16R8 target has 16 MiB flash and 8 MiB octal PSRAM. PlatformIO uses the custom [`partitions.csv`](partitions.csv) layout:
|
||
|
||
| Partition | Offset | Size | Purpose |
|
||
|---|---:|---:|---|
|
||
| `nvs` | `0x009000` | 512 KiB | Serial, Wi-Fi, mDNS hostname, local-display, role-based user, HTTPS identity, and SSH host-key data |
|
||
| `otadata` | `0x089000` | 8 KiB | Active OTA-slot selection metadata |
|
||
| `phy_init` | `0x08B000` | 4 KiB | Optional PHY initialization data |
|
||
| `nvs_key` | `0x08C000` | 4 KiB | Reserved for future encrypted-NVS keys |
|
||
| `coredump` | `0x08D000` | 128 KiB | Reserved for flash core dumps |
|
||
| `ota_0` | `0x0B0000` | 4 MiB | Primary application/OTA slot |
|
||
| `ota_1` | `0x4B0000` | 4 MiB | Alternate application/OTA slot |
|
||
| `storage` | `0x8B0000` | 7488 KiB | Reserved for future LittleFS web assets, logs, and files |
|
||
|
||
Application offsets are aligned to the ESP32-S3's required 64 KiB boundary. The final storage partition ends at `0x1000000`, exactly the end of the 16 MiB flash chip.
|
||
|
||
The table reserves OTA and storage space; it does not implement OTA downloads, rollback confirmation, core-dump handling, NVS encryption, or filesystem mounting.
|
||
|
||
### One-time migration from the default partition table
|
||
|
||
The previous 1 MiB factory application began at `0x10000`, which now lies inside the enlarged NVS range. A normal upload does not erase stale data in that range. When first switching to this layout, erase the flash completely:
|
||
|
||
```sh
|
||
pio run --target erase
|
||
pio run --target upload
|
||
pio device monitor -b 115200
|
||
```
|
||
|
||
This removes saved serial configuration and all other flash contents. The firmware recreates NVS with safe defaults. Subsequent ordinary uploads do not need a full erase.
|
||
|
||
## Build
|
||
|
||
```sh
|
||
pio run
|
||
```
|
||
|
||
## Upload and monitor
|
||
|
||
Connect the board's **USB-to-UART** port for firmware upload and the UART0 development console, then run:
|
||
|
||
```sh
|
||
pio run --target upload
|
||
pio device monitor -b 115200
|
||
```
|
||
|
||
The firmware provides an interactive UART0 console at `serial-tool>`. Run `help` for available commands. The USB-to-UART bridge normally appears as `/dev/ttyUSB*`; it is separate from the native USB CDC serial transport, which normally appears as `/dev/ttyACM*`.
|
||
|
||
The console supports session history, line editing, cursor movement, and hierarchical Tab completion. After an unattended boot, attach an ANSI-capable terminal and press Enter once to enable enhanced editing; this avoids blocking while no terminal is attached.
|
||
|
||
Serial, Wi-Fi, and mDNS hostname edits remain in RAM until explicitly saved with `serial save`, `wifi save`, or `mdns save`. Authenticated admin SSH sessions expose the shared operational administration registry, including interactive secrets, TLS/SSH identity management, network diagnostics, and deferred reboot/SSH lifecycle commands. Create the first administrator on UART0 with `user add <username> admin` (optionally `--generate`). Explicit recovery of an unavailable user database remains UART0-only and rebuilds it empty; it refuses a healthy database. An administrator also cannot generate a replacement password for its own account over SSH, preventing the one-time value from being lost when that mutation revokes the session. Legacy web credential commands and `user bootstrap` are removed.
|
||
|
||
## Security notes
|
||
|
||
The HTTPS interface uses a device-specific self-signed certificate and a same-origin login page with bounded server-side cookie sessions; HTTP Basic is no longer accepted. Open `/` or `/login`, sign in with a user-database password, and use **Sign out** before switching accounts. Four sessions have a one-hour absolute lifetime, including active serial connections; logout closes only that session's serial access. Login is globally limited to five credential verifications per 60 seconds, with explicit capacity/backoff errors. Direct-IP and mDNS access use separate host-only Secure/HttpOnly/SameSite=Strict cookies. Non-browser clients also require cookies, strict Origin and CSRF for mutations rather than Basic credentials. There is no plaintext HTTP or TCP serial listener. SSH accepts role-based passwords and authorized Ed25519/ECDSA P-256 public keys. User passwords are stored as salted PBKDF2-HMAC-SHA256 verifiers, but the HTTPS private key, SSH private key, and Wi-Fi credentials remain recoverable from unencrypted application-owned NVS blobs. Offline password guessing and stale append-oriented flash copies also remain possible. The reserved `nvs_key` partition does not enable encryption. Do not treat this firmware as resistant to physical flash or RAM extraction until the planned hardening work is complete.
|
||
|
||
## License
|
||
|
||
This project is licensed under the [GNU General Public License version 3 only](LICENSE) (`GPL-3.0-only`). Third-party components remain subject to their respective licenses. The integration baseline uses Espressif registry components `espressif/mdns` `1.12.0`, `wolfssl/wolfssl` `5.8.2~1`, and `wolfssl/wolfssh` `1.4.20`; review upstream security releases before production use.
|
||
|
||
### Legacy credential removal
|
||
|
||
Missing user storage is persisted as an empty database; no shared credential is imported or synchronized. Existing valid v1 user records retain their accounts, roles, IDs, verifiers and keys without a schema change. HTTPS `web_sec/material` upgrades valid 1,392-byte v1 storage to 1,340-byte TLS-only v2, retaining exact certificate/key DER, fingerprint and generation, and committing before publication. Invalid records or migration failures fail closed rather than triggering fallback replacement. `web certificate rotate --force` remains available; `web reset --force` replaces TLS identity only, not users.
|
||
|
||
**Downgrade warning:** older v1-only firmware cannot read v2 HTTPS material. Logical NVS replacement is not a secure flash wipe; historical plaintext credentials can remain in flash. This cleanup requires no factory/partition erase. See [implementation and evidence limits](docs/legacy_credential_removal.md); final integration build evidence is pending.
|