Commander1024/ESP32_Serial_Swiss_Army_Knife
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add SSH phase roadmap documentation

Browse Source
This commit is contained in:
Commander1024
2026-08-25 15:48:02 +02:00
parent a58eacb7d2
commit a350ba2f5e
2 changed files with 336 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -14,7 +14,7 @@ ESP32-S3 firmware for a secure, multi-transport RS-232 adapter. It operates one
## Development status
Hardware characterization, the serial core, session broker, and native USB CDC-ACM are complete. The Wi-Fi foundation is implemented and substantially validated, with several smoke tests still pending. The project is currently entering the **HTTPS/web-console phase**; later work includes interface hardware, security hardening, OTA, BLE, enterprise Wi-Fi, WireGuard evaluation, and optional storage features. See the [implementation plan](GPT-logs/Implementation%20plan.md) for validation details and roadmap.
Hardware characterization, the serial core, session broker, and native USB CDC-ACM are complete. The Wi-Fi foundation is implemented and substantially validated, with several smoke tests still pending. The project is currently in the **SSH-Phase**; later work includes interface hardware, security hardening, OTA, BLE, enterprise Wi-Fi, WireGuard evaluation, and optional storage features. See the [Roadmap](docs/roadmap.md) for validation details and roadmap.
## Documentation
docs/roadmap.md
+335
View File
@@ -0,0 +1,335 @@
# Implementation roadmap
This document tracks the implementation and hardware-validation plan for the ESP32 Serial Swiss Army Knife. It describes intended sequencing rather than a release schedule. A phase is complete only when its implementation, documentation, build validation, and relevant hardware tests have passed.
## Status legend
- **Complete** — implemented and validated on the target hardware.
- **Implemented; validation pending** — code is present and builds, but the current implementation still needs the listed hardware checks.
- **Planned** — accepted project direction, not yet implemented.
- **Under evaluation** — useful candidate whose feasibility, security, or resource cost must be measured before it becomes a commitment.
## Project-wide design constraints
These constraints apply across all phases:
- UART0 remains the trusted physical administration and recovery console.
- UART1 and the MAX3243 provide the single physical RS-232 data port.
- All serial transports use the central session broker: one writer, multiple observers.
- A slow or failed client must not block UART reception or unrelated clients.
- Serial data paths remain binary-transparent; transports must not reserve in-band control sequences.
- Network serial access must be authenticated and encrypted. No unauthenticated plaintext TCP serial service is planned.
- Secrets must not be exposed by routine status output, command completion, or logs.
- Resource use must remain bounded and observable on the ESP32-S3 N16R8 target.
- New work must preserve USB CDC and UART0 recovery when Wi-Fi or a network service fails.
- Project code remains `GPL-3.0-only`; third-party license notices and compatibility must be preserved.
## Phase overview
| Phase | Scope | Status |
|---|---|---|
| 0 | RS-232 hardware characterization and diagnostics | **Complete** |
| 1 | Persistent UART1 serial-service foundation | **Complete** |
| 2 | Transport-neutral session broker | **Complete** |
| 3 | Native USB CDC-ACM serial transport | **Complete** |
| 4 | Wi-Fi station/AP foundation and network diagnostics | **Complete** |
| 5A | Authenticated HTTPS administration foundation | **Complete** |
| 5B | Offline xterm.js WebSocket serial terminal | **Complete** |
| 6 | Authenticated SSH serial transport | **Implemented; validation pending** |
| 7 | Local display and button interface | **Planned** |
| 8 | Security and production hardening | **Planned** |
| 9 | Authenticated, rollback-capable OTA | **Planned** |
| 10 | BLE serial transport and provisioning evaluation | **Planned** |
| 11 | Advanced network integration | **Under evaluation** |
| 12 | Optional filesystem-backed features | **Under evaluation** |
## Completed foundation
### Phase 0 — RS-232 hardware characterization
Implemented and hardware-validated:
- Central board pin profile for the ESP32-S3 N16R8 and MAX3243 breakout.
- MAX3243 enable, static driver, receiver, valid-voltage, and modem-signal checks.
- UART loopback across supported framing and baud-rate combinations.
- Hardware CTS transmit gating and RTS receive-backpressure tests.
- Exclusive RS-232 port ownership and fault-safe cleanup.
- RGB status indication for diagnostic state.
- Potentially disruptive tests isolated under the `debug` CLI submenu; top-level `status` retained.
Detailed procedures are in [Electrical tests](electrical_tests.md) and [Hardware wiring](wiring.md).
### Phase 1 — Serial-service foundation
Implemented and hardware-validated:
- Persistent, versioned UART configuration with safe defaults.
- UART1 lifecycle management and exclusive ownership.
- Nonblocking RX and TX software streams.
- Baud rate, data bits, parity, stop bits, RTS/CTS, DTR policy, and RTS threshold controls.
- Runtime counters and safe restart after working-configuration changes.
- Explicit save/load/default/reset behavior; edits are not silently persisted.
### Phase 2 — Session broker
Implemented and hardware-validated:
- Up to eight generation-safe broker clients.
- Exactly one writer lease with multiple observers.
- Bounded per-client output and event queues.
- Slow-observer isolation and drop accounting.
- Writer grant, release, denial, revocation, and forced administrative reassignment.
- DTR `on-connect` integration.
- Partial-transfer and retryable no-progress semantics for transports.
- Broker payload storage placed preferentially in PSRAM while FreeRTOS control structures remain in internal RAM.
### Phase 3 — Native USB CDC-ACM
Implemented and hardware-validated:
- Native ESP32-S3 USB CDC-ACM serial transport.
- Automatic broker connection when the host opens the port with DTR asserted.
- Opportunistic writer acquisition with observer fallback.
- Binary-transparent bidirectional serial data.
- Supported host line-coding application while USB owns the writer lease.
- Disconnect cleanup, bounded buffering, counters, and UART0 lifecycle controls.
### Phase 4 — Wi-Fi foundation
Implemented and hardware-validated:
- Four persistent station profiles with priority ordering.
- WPA2/WPA3 mixed mode and WPA3-SAE-required mode.
- Explicit no-echo secret entry through UART0.
- AP policies `off`, `fallback`, and `always`.
- Random persistent fallback-AP password and MAC-derived default SSID.
- Bounded profile attempts, DHCP deadlines, failover, and retry backoff.
- Root and `wifi` aliases for `ping`, `nslookup`, and `traceroute`.
- Wi-Fi/lwIP payload allocation configured to prefer PSRAM with capacities pinned in `sdkconfig.defaults`.
A WPA3-required profile correctly rejects an AP that does not advertise the required Protected Management Frame capability; mixed mode is the compatible choice for such an AP.
### Phase 5A — Authenticated HTTPS foundation
Implemented and hardware-validated:
- HTTPS-only service on TCP port 443 using ESP-IDF mbedTLS.
- Persistent random shared `admin` credential.
- Persistent device-specific ECDSA P-256 certificate and private key.
- Physical-console credential retrieval, rotation, certificate inspection, rotation, and recovery.
- Explicit failure behavior that preserves UART0, USB, serial, and Wi-Fi recovery paths.
- Credential rotation revocation hooks shared with WebSocket and SSH sessions.
### Phase 5B — Offline browser terminal
Implemented and hardware-validated:
- Vendored xterm.js and FitAddon assets; no Internet or CDN dependency.
- CSP-compatible page structure with scripts served as separate resources.
- Authenticated status and one-time WebSocket-ticket APIs.
- Binary WebSocket serial frames and broker writer/observer integration.
- Two bounded browser-terminal sessions.
- Writer request/release controls and role indication.
- Bounded TLS/socket behavior and slow-observer isolation.
- Viewport-constrained terminal fitting without recursive page growth.
- Validation with multiple clients, ANSI colors, advanced escape sequences, and full-screen terminal software.
## Current phase
### Phase 6 — Authenticated SSH serial transport
The initial SSH implementation is present and has passed single-service hardware tests. It provides:
- wolfSSH on TCP port 22 while HTTPS remains on mbedTLS.
- Password authentication using the shared `admin` credential.
- A separate persistent ECDSA P-256 SSH host key and OpenSSH-compatible fingerprint.
- Interactive shell/PTY channels only.
- No `exec`, subsystem, SFTP, SCP, agent forwarding, or TCP forwarding.
- Two fixed session/handshake slots.
- Three password attempts and a 15-second handshake deadline.
- No broker client before successful authentication and shell negotiation.
- Binary-transparent serial forwarding with no in-band writer-control protocol.
- Opportunistic writer acquisition and observer behavior through the common broker.
- Credential-rotation revocation and UART0 session/host-key administration.
- A single bounded wolfSSH owner task pinned to CPU 1.
- wolfSSL/wolfSSH allocations that prefer PSRAM with internal-memory fallback.
- A per-call wolfSSH receive budget to prevent pipelined input from monopolizing the owner task.
- Internal task stacks retained for safety during flash/NVS cache-disable windows.
- Heap and SSH stack telemetry through `memory` and `ssh status`.
SSH login, host-key verification, serial traffic, Web Serial coexistence, and USB CDC coexistence have been validated. Before the latest resource-placement changes, however, the following three-client arrangement could stall or fail the web service:
1. USB CDC connected as writer.
2. Web serial connected as observer.
3. SSH connected as observer.
The mitigation moves broker and Wi-Fi/lwIP payload pressure toward PSRAM and moves SSH processing away from the CPU 0 Wi-Fi/web path. Phase 6 remains open until that exact arrangement is revalidated.
#### Phase 6 completion gate
After flashing the current build:
1. At idle, record:
```text
memory
ssh status
```
2. Reproduce this order exactly:
1. Connect USB CDC and confirm it owns the writer lease.
2. Connect one WebSocket terminal and confirm observer status.
3. Connect one SSH terminal and confirm observer status.
3. While all three are connected, collect:
```text
memory
ssh status
web status
web counters
ssh counters
broker clients
```
4. Exercise sustained RS-232 output, writer release/reassignment, and input through the newly selected writer. Verify that all observers continue receiving data and that UART0 remains responsive.
5. Disconnect the transports in different orders and verify that sockets, broker clients, and writer ownership are cleaned up.
6. Capture the complete UART0 log if any failure occurs, especially task-watchdog output, backtraces, `ESP_ERR_NO_MEM`, TLS allocation failures, lwIP/pbuf failures, or HTTP server accept/session errors.
Interpret the telemetry as follows:
- Low internal or DMA-capable free space/largest block indicates more internal-memory reduction is needed.
- Healthy heap metrics combined with a watchdog or stall indicates a scheduling or blocking-path problem rather than simple exhaustion.
- A consistently large SSH stack minimum-free value permits a measured reduction from the current 20 KiB stack; it must not be reduced speculatively.
- Low or fragmented PSRAM indicates that queue, socket, or allocation capacities need review.
Phase 6 is complete when this concurrent configuration remains stable, cleanup succeeds, the relevant counters show no unexplained loss, and no watchdog or memory-allocation error occurs.
## Planned phases
The order below is the current plan. Detailed requirements should be finalized at the start of each phase, and optional features must not weaken the completed serial and recovery paths.
### Phase 7 — Local display and buttons
Add a standalone local status/control interface without making it a dependency of the serial core.
Planned work:
- Select the actual display, button arrangement, electrical interface, and remaining GPIO assignment before implementation.
- Add a board-profile abstraction where required instead of scattering new pin assumptions.
- Show concise serial, Wi-Fi, service, client-count, and writer-owner status.
- Provide safe button actions for navigation and common lifecycle or writer-control operations.
- Require deliberate confirmation for destructive or security-sensitive actions.
- Avoid displaying passwords or private-key material during ordinary operation.
- Keep UART0 fully functional as the authoritative recovery interface if the display or buttons fail.
- Bound display refresh and event processing so they cannot interfere with UART or network transport timing.
Completion requires electrical validation, operation with all current transports, and documented recovery behavior.
### Phase 8 — Security and production hardening
Harden stored credentials, boot integrity, update trust, failure diagnostics, and dependency maintenance. This phase must be staged carefully because some ESP32 eFuse operations are irreversible.
Planned work:
- Define development, test, and production security profiles.
- Enable and validate NVS encryption for Wi-Fi, HTTPS, and SSH material.
- Evaluate flash encryption and PSRAM encryption for the target hardware and threat model.
- Enable secure boot with a documented signing-key and recovery process.
- Define secret-safe core-dump handling; do not expose credentials or session material in diagnostics.
- Audit key and session-material zeroization across application, mbedTLS, wolfSSL, and wolfSSH allocations.
- Add authentication throttling, lockout/backoff, and security-event counters where appropriate.
- Review TLS/SSH algorithms, certificate policy, password policy, and browser security headers.
- Review current wolfSSL/wolfSSH and ESP-IDF security releases instead of treating integration versions as permanent pins.
- Recheck all third-party licenses and preserve source/license notices.
- Document provisioning, key rotation, factory reset, backup, recovery, and device decommissioning.
Security features must be validated first on expendable hardware before any irreversible production configuration is recommended.
### Phase 9 — Authenticated OTA and rollback
Use the existing dual 4 MiB application slots and `otadata` partition to add safe firmware updates.
Planned work:
- Accept only authenticated, integrity-checked firmware images compatible with the selected secure-boot policy.
- Support an update path that does not expose a plaintext management endpoint.
- Stream downloads with bounded RAM use and explicit progress/error reporting.
- Preserve serial, Wi-Fi, HTTPS, and SSH configuration across successful updates.
- Mark new firmware pending until a post-boot health check succeeds.
- Roll back automatically after failed boot or health confirmation.
- Define power-loss behavior for every update stage.
- Keep UART0 recovery and wired re-flashing documented and functional.
- Add version/compatibility checks and reject accidental downgrade unless an explicit policy allows it.
OTA is complete only after successful update, interrupted-update, invalid-image, rollback, and configuration-persistence tests on hardware.
### Phase 10 — BLE
Evaluate and, if resource limits permit, add BLE as another bounded transport or provisioning path.
Planned work:
- Decide whether BLE is primarily a serial transport, a provisioning interface, or both.
- Require authenticated pairing/bonding appropriate to the device threat model.
- Integrate serial access through the existing broker rather than bypassing writer ownership.
- Preserve binary transparency through explicit framing and MTU-aware partial transfers.
- Bound connections, queues, retries, and advertising behavior.
- Measure coexistence with 2.4 GHz Wi-Fi, HTTPS, WebSocket, SSH, USB CDC, and UART service.
- Store bond/provisioning data under the hardened secret-storage policy.
- Provide UART0 controls and a physical recovery path for clearing BLE state.
BLE remains subordinate to stable Wi-Fi and serial operation; it should be omitted if coexistence or memory costs cannot be bounded acceptably.
### Phase 11 — Advanced network integration
These features are candidates, not current commitments:
- mDNS advertisement for the certificate’s device-specific `.local` name and selected services.
- Enterprise Wi-Fi support, subject to credential-storage and certificate-validation design.
- IPv6 behavior and diagnostics beyond the current basic support.
- WireGuard feasibility evaluation, including RAM, CPU, licensing, key storage, routing, and recovery impact.
The device is not intended to become a general-purpose router. Captive-portal interception, unauthenticated DNS redirection, NAPT, and a plaintext serial listener remain out of scope unless the project requirements are explicitly revised.
### Phase 12 — Optional filesystem-backed features
The `storage` partition is reserved but not currently mounted. Possible uses must be justified individually:
- LittleFS-backed web assets to decouple large static files from the application image.
- Bounded diagnostic logs with explicit retention and secret-redaction rules.
- Import/export of non-secret configuration.
- Optional user files needed by later administration features.
Before enabling storage, define corruption recovery, wear limits, quotas, atomic update behavior, compatibility across OTA slots, and whether encryption is required. Serial operation and recovery must not depend on a mountable filesystem.
## Cross-phase release gates
Every phase should satisfy the following before being marked complete:
1. A clean release build for the ESP32-S3 N16R8 target.
2. No new compiler or project diagnostics attributable to the change.
3. Focused automated tests where practical and documented hardware tests where hardware behavior is involved.
4. UART0, native USB CDC, and previously completed network transports still operate.
5. Broker writer/observer semantics and binary transparency remain intact.
6. Failure, disconnect, stop/start, reboot, and stale-session cleanup paths are exercised.
7. Runtime memory, stack, socket, and queue bounds are measured under maximum supported concurrency.
8. Secrets remain absent from ordinary status output and logs.
9. [Command reference](command_reference.md), [Hardware wiring](wiring.md), electrical tests, and this roadmap are updated as applicable.
10. New dependencies receive security, maintenance, and GPL-3.0 compatibility review.
## Explicitly deferred work
The following are not implemented merely because flash partitions or library support exist:
- NVS, flash, or PSRAM encryption.
- Secure boot or production eFuse provisioning.
- OTA download, image confirmation, or rollback policy.
- Core-dump collection or secret-safe core-dump processing.
- Filesystem mounting.
- Public-key SSH user authentication.
- SFTP, SCP, SSH `exec`, forwarding, or subsystems.
- General routing, NAPT, captive-portal interception, or unauthenticated serial access.
Deferring these features is intentional: each changes the security model, recovery behavior, or bounded-resource assumptions and therefore requires its own design and validation gate.