107 lines
12 KiB
Markdown
107 lines
12 KiB
Markdown
# ESP32 Serial Swiss Army Knife
|
|
|
|
|
|
|
|
ESP32-S3 firmware for a secure, multi-transport RS-232 adapter. It operates one MAX3243-backed UART1 serial port and safely shares it between native USB CDC-ACM, an HTTPS/WebSocket browser terminal, and SSH: one connected client can write while others observe. The firmware also provides persistent serial and Wi-Fi configuration, a UART0 recovery console, and hardware diagnostics; it is not a general-purpose router, captive portal, or unauthenticated TCP serial server.
|
|
|
|
## Initial hardware target
|
|
|
|
- ESP32-S3-DevKitC-1-compatible development board
|
|
- ESP32-S3-WROOM-1-N16R8 module
|
|
- 16 MB flash
|
|
- 8 MB octal PSRAM
|
|
- Adafruit MAX3243 full-pinout RS-232 breakout; the male connector version is preferred (see [Hardware wiring](docs/wiring.md) for supported connector variants)
|
|
|
|
## Development status
|
|
|
|
Hardware characterization, the serial core, USB CDC-ACM, Wi-Fi, HTTPS/WebSocket, SSH serial transport, and the local display/control interface are implemented and Phase 7 target-hardware validated. Phase 8A's bounded role-based user database and UART0 administration, Phase 8B's role-aware HTTPS/SSH authentication and revocation, and Phase 8C's shared UART0/admin-SSH command shell are target-hardware validated. Phase 8D.3 browser login/logout is implemented, host-tested and build-verified; [M1 is validated by user sign-off](docs/phase8d3_implementation.md) after both-role login, mixed-client operation and post-soak evidence. Numeric memory reserve gates remain open. Browser admin-shell mode is implemented with M2 signed off; typed Serial/account settings through 8D.10 are accepted, and 8D.11 key settings are implemented. Admin-only Network settings (8D.12/8D.13, jointly authorized) now deliver STA/AP/profile and mDNS edits, explicit persistence, password replacement/disabled-STA clear and connection controls; final parent build/tests and target acceptance are pending. Settings navigation preserves terminal sessions and serial writer ownership; actual network disruption can disconnect network clients. Further contextual administration and full M3 acceptance remain pending. Configurable STA-only mDNS naming as `sak-<suffix>.local` is implemented with independent NVS persistence; target-hardware validation is pending. See the [Roadmap](docs/roadmap.md) for phase status and validation details.
|
|
|
|
### Browser Network settings (8D.12/8D.13)
|
|
|
|
Administrators can open **Settings → Network**; normal users cannot access its APIs. Refresh reads working configuration/runtime without exporting saved passwords or their lengths. SSIDs have UTF-8 text and reversible hex-byte modes (32-byte maximum). Password **Keep** preserves the current secret; **Replace** requires explicit new input; **Clear** is allowed only for a disabled STA profile, never AP. Inputs are transient and never prefilled from storage.
|
|
|
|
Apply changes RAM; **Save** explicitly persists device working state, not unsent drafts. Wi-Fi Load uses stored configuration only; there is no browser Wi-Fi reset/default-secret generation or secret export. mDNS Set/Load/Defaults request STA reannouncement; Save persists the name. The profile selector chooses what to edit, not what to connect to: **Next profile** follows enabled profiles in canonical priority order.
|
|
|
|
Confirm disruptive actions only with a recovery route ready. `accepted` does not mean online or verified DNS, and HTTPS/SSH/both browser terminals may disconnect before acknowledgement. Never automatically replay uncertain operations: reconnect via STA/AP, use Check Result/Refresh and inspect state. UART0 remains administrative recovery and native USB remains network-independent UART1 access. Changed hostnames require client DNS/trust/login checks. Browser-shell command restrictions are unchanged. See the [full bounded API, implementation evidence and pending target checklist](docs/phase8d12_13_implementation.md); no new commands or generated assets are introduced.
|
|
|
|
### Browser HTTPS identity (8D.21 HTTPS-only slice)
|
|
|
|
Administrators can use **Settings → HTTPS / Reboot → Refresh** to inspect the stored HTTPS certificate's public SHA-256 fingerprint and identity/service generations, then explicitly confirm **Rotate HTTPS identity**. Rotation persists a new certificate and private key and restarts HTTPS, invalidating all web logins and both browser terminal routes. Save browser drafts first. User accounts, SSH identity, Wi-Fi and device configuration are unchanged; UART0 administration and native USB UART1 access remain independent.
|
|
|
|
A lost acknowledgement, expired login or failed stop/start does **not** prove that the old identity remains: the new identity may already be persisted, with an old certificate still served if shutdown failed. There is no rollback or automatic retry. Inspect the result through trusted UART0 using `web certificate info`; verify that fingerprint before renewing browser trust, then reload and sign in freshly. Accepting a certificate warning alone is not trusted verification. Use canonical `web stop` / `web start` for retained-server recovery when necessary. The existing CLI `web reset --force` retains TLS-only replacement/recovery semantics and starts a stopped service; the browser has no duplicate reset or unavailable-material recovery action. No certificate/private-key export is provided.
|
|
|
|
The [HTTPS slice is implemented and host/build verified](docs/phase8d21_implementation.md); independent parent review and target trust/persistence/full-mix checks remain pending. SSH identity work requires a separate request, so full 8D.21 remains incomplete.
|
|
|
|
## Documentation
|
|
|
|
- [Hardware wiring](docs/wiring.md): hardware profile, GPIO assignments, connector guidance, and safety notes.
|
|
- [Electrical tests](docs/electrical_tests.md): OLED/buttons, MAX3243, UART loopback, and session-broker verification procedures.
|
|
- [Role-based user database and UART0 administration](docs/user_administration_tests.md): user provisioning and administration, HTTPS/SSH authentication, session revocation, and the planned integrated web-administration acceptance matrix.
|
|
- [Command reference](docs/command_reference.md): UART0/admin-SSH administration, serial, broker, USB, Wi-Fi, mDNS, web, SSH, and diagnostic commands.
|
|
|
|
## Flash partition layout
|
|
|
|
The N16R8 target has 16 MiB flash and 8 MiB octal PSRAM. PlatformIO uses the custom [`partitions.csv`](partitions.csv) layout:
|
|
|
|
| Partition | Offset | Size | Purpose |
|
|
|---|---:|---:|---|
|
|
| `nvs` | `0x009000` | 512 KiB | Serial, Wi-Fi, mDNS hostname, local-display, role-based user, HTTPS identity, and SSH host-key data |
|
|
| `otadata` | `0x089000` | 8 KiB | Active OTA-slot selection metadata |
|
|
| `phy_init` | `0x08B000` | 4 KiB | Optional PHY initialization data |
|
|
| `nvs_key` | `0x08C000` | 4 KiB | Reserved for future encrypted-NVS keys |
|
|
| `coredump` | `0x08D000` | 128 KiB | Reserved for flash core dumps |
|
|
| `ota_0` | `0x0B0000` | 4 MiB | Primary application/OTA slot |
|
|
| `ota_1` | `0x4B0000` | 4 MiB | Alternate application/OTA slot |
|
|
| `storage` | `0x8B0000` | 7488 KiB | Reserved for future LittleFS web assets, logs, and files |
|
|
|
|
Application offsets are aligned to the ESP32-S3's required 64 KiB boundary. The final storage partition ends at `0x1000000`, exactly the end of the 16 MiB flash chip.
|
|
|
|
The table reserves OTA and storage space; it does not implement OTA downloads, rollback confirmation, core-dump handling, NVS encryption, or filesystem mounting.
|
|
|
|
### One-time migration from the default partition table
|
|
|
|
The previous 1 MiB factory application began at `0x10000`, which now lies inside the enlarged NVS range. A normal upload does not erase stale data in that range. When first switching to this layout, erase the flash completely:
|
|
|
|
```sh
|
|
pio run --target erase
|
|
pio run --target upload
|
|
pio device monitor -b 115200
|
|
```
|
|
|
|
This removes saved serial configuration and all other flash contents. The firmware recreates NVS with safe defaults. Subsequent ordinary uploads do not need a full erase.
|
|
|
|
## Build
|
|
|
|
```sh
|
|
pio run
|
|
```
|
|
|
|
## Upload and monitor
|
|
|
|
Connect the board's **USB-to-UART** port for firmware upload and the UART0 development console, then run:
|
|
|
|
```sh
|
|
pio run --target upload
|
|
pio device monitor -b 115200
|
|
```
|
|
|
|
The firmware provides an interactive UART0 console at `serial-tool>`. Run `help` for available commands. The USB-to-UART bridge normally appears as `/dev/ttyUSB*`; it is separate from the native USB CDC serial transport, which normally appears as `/dev/ttyACM*`.
|
|
|
|
The console supports session history, line editing, cursor movement, and hierarchical Tab completion. After an unattended boot, attach an ANSI-capable terminal and press Enter once to enable enhanced editing; this avoids blocking while no terminal is attached.
|
|
|
|
Serial, Wi-Fi, and mDNS hostname edits remain in RAM until explicitly saved with `serial save`, `wifi save`, or `mdns save`. Authenticated admin SSH sessions expose the shared operational administration registry, including interactive secrets, TLS/SSH identity management, network diagnostics, and deferred reboot/SSH lifecycle commands. Create the first administrator on UART0 with `user add <username> admin` (optionally `--generate`). Explicit recovery of an unavailable user database remains UART0-only and rebuilds it empty; it refuses a healthy database. An administrator also cannot generate a replacement password for its own account over SSH, preventing the one-time value from being lost when that mutation revokes the session. Legacy web credential commands and `user bootstrap` are removed.
|
|
|
|
## Security notes
|
|
|
|
The HTTPS interface uses a device-specific self-signed certificate and a same-origin login page with bounded server-side cookie sessions; HTTP Basic is no longer accepted. Open `/` or `/login`, sign in with a user-database password, and use **Sign out** before switching accounts. Four sessions have a one-hour absolute lifetime, including active serial connections; logout closes only that session's serial access. Login is globally limited to five credential verifications per 60 seconds, with explicit capacity/backoff errors. Direct-IP and mDNS access use separate host-only Secure/HttpOnly/SameSite=Strict cookies. Non-browser clients also require cookies, strict Origin and CSRF for mutations rather than Basic credentials. There is no plaintext HTTP or TCP serial listener. SSH accepts role-based passwords and authorized Ed25519/ECDSA P-256 public keys. User passwords are stored as salted PBKDF2-HMAC-SHA256 verifiers, but the HTTPS private key, SSH private key, and Wi-Fi credentials remain recoverable from unencrypted application-owned NVS blobs. Offline password guessing and stale append-oriented flash copies also remain possible. The reserved `nvs_key` partition does not enable encryption. Do not treat this firmware as resistant to physical flash or RAM extraction until the planned hardening work is complete.
|
|
|
|
## License
|
|
|
|
This project is licensed under the [GNU General Public License version 3 only](LICENSE) (`GPL-3.0-only`). Third-party components remain subject to their respective licenses. The integration baseline uses Espressif registry components `espressif/mdns` `1.12.0`, `wolfssl/wolfssl` `5.8.2~1`, and `wolfssl/wolfssh` `1.4.20`; review upstream security releases before production use.
|
|
|
|
### Legacy credential removal
|
|
|
|
Missing user storage is persisted as an empty database; no shared credential is imported or synchronized. Existing valid v1 user records retain their accounts, roles, IDs, verifiers and keys without a schema change. HTTPS `web_sec/material` upgrades valid 1,392-byte v1 storage to 1,340-byte TLS-only v2, retaining exact certificate/key DER, fingerprint and generation, and committing before publication. Invalid records or migration failures fail closed rather than triggering fallback replacement. `web certificate rotate --force` remains available; `web reset --force` replaces TLS identity only, not users.
|
|
|
|
**Downgrade warning:** older v1-only firmware cannot read v2 HTTPS material. Logical NVS replacement is not a secure flash wipe; historical plaintext credentials can remain in flash. This cleanup requires no factory/partition erase. See [implementation and evidence limits](docs/legacy_credential_removal.md); final integration build evidence is pending.
|