From a350ba2f5efd424e2d9d3af88dc1451bf4715234 Mon Sep 17 00:00:00 2001 From: Commander1024 Date: Tue, 25 Aug 2026 15:48:02 +0200 Subject: [PATCH] Add SSH phase roadmap documentation --- README.md | 2 +- docs/roadmap.md | 335 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 336 insertions(+), 1 deletion(-) create mode 100644 docs/roadmap.md diff --git a/README.md b/README.md index e29116f..0a4cc9d 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ ESP32-S3 firmware for a secure, multi-transport RS-232 adapter. It operates one ## Development status -Hardware characterization, the serial core, session broker, and native USB CDC-ACM are complete. The Wi-Fi foundation is implemented and substantially validated, with several smoke tests still pending. The project is currently entering the **HTTPS/web-console phase**; later work includes interface hardware, security hardening, OTA, BLE, enterprise Wi-Fi, WireGuard evaluation, and optional storage features. See the [implementation plan](GPT-logs/Implementation%20plan.md) for validation details and roadmap. +Hardware characterization, the serial core, session broker, and native USB CDC-ACM are complete. The Wi-Fi foundation is implemented and substantially validated, with several smoke tests still pending. The project is currently in the **SSH-Phase**; later work includes interface hardware, security hardening, OTA, BLE, enterprise Wi-Fi, WireGuard evaluation, and optional storage features. See the [Roadmap](docs/roadmap.md) for validation details and roadmap. ## Documentation diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 0000000..a321e59 --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,335 @@ +# Implementation roadmap + +This document tracks the implementation and hardware-validation plan for the ESP32 Serial Swiss Army Knife. It describes intended sequencing rather than a release schedule. A phase is complete only when its implementation, documentation, build validation, and relevant hardware tests have passed. + +## Status legend + +- **Complete** — implemented and validated on the target hardware. +- **Implemented; validation pending** — code is present and builds, but the current implementation still needs the listed hardware checks. +- **Planned** — accepted project direction, not yet implemented. +- **Under evaluation** — useful candidate whose feasibility, security, or resource cost must be measured before it becomes a commitment. + +## Project-wide design constraints + +These constraints apply across all phases: + +- UART0 remains the trusted physical administration and recovery console. +- UART1 and the MAX3243 provide the single physical RS-232 data port. +- All serial transports use the central session broker: one writer, multiple observers. +- A slow or failed client must not block UART reception or unrelated clients. +- Serial data paths remain binary-transparent; transports must not reserve in-band control sequences. +- Network serial access must be authenticated and encrypted. No unauthenticated plaintext TCP serial service is planned. +- Secrets must not be exposed by routine status output, command completion, or logs. +- Resource use must remain bounded and observable on the ESP32-S3 N16R8 target. +- New work must preserve USB CDC and UART0 recovery when Wi-Fi or a network service fails. +- Project code remains `GPL-3.0-only`; third-party license notices and compatibility must be preserved. + +## Phase overview + +| Phase | Scope | Status | +|---|---|---| +| 0 | RS-232 hardware characterization and diagnostics | **Complete** | +| 1 | Persistent UART1 serial-service foundation | **Complete** | +| 2 | Transport-neutral session broker | **Complete** | +| 3 | Native USB CDC-ACM serial transport | **Complete** | +| 4 | Wi-Fi station/AP foundation and network diagnostics | **Complete** | +| 5A | Authenticated HTTPS administration foundation | **Complete** | +| 5B | Offline xterm.js WebSocket serial terminal | **Complete** | +| 6 | Authenticated SSH serial transport | **Implemented; validation pending** | +| 7 | Local display and button interface | **Planned** | +| 8 | Security and production hardening | **Planned** | +| 9 | Authenticated, rollback-capable OTA | **Planned** | +| 10 | BLE serial transport and provisioning evaluation | **Planned** | +| 11 | Advanced network integration | **Under evaluation** | +| 12 | Optional filesystem-backed features | **Under evaluation** | + +## Completed foundation + +### Phase 0 — RS-232 hardware characterization + +Implemented and hardware-validated: + +- Central board pin profile for the ESP32-S3 N16R8 and MAX3243 breakout. +- MAX3243 enable, static driver, receiver, valid-voltage, and modem-signal checks. +- UART loopback across supported framing and baud-rate combinations. +- Hardware CTS transmit gating and RTS receive-backpressure tests. +- Exclusive RS-232 port ownership and fault-safe cleanup. +- RGB status indication for diagnostic state. +- Potentially disruptive tests isolated under the `debug` CLI submenu; top-level `status` retained. + +Detailed procedures are in [Electrical tests](electrical_tests.md) and [Hardware wiring](wiring.md). + +### Phase 1 — Serial-service foundation + +Implemented and hardware-validated: + +- Persistent, versioned UART configuration with safe defaults. +- UART1 lifecycle management and exclusive ownership. +- Nonblocking RX and TX software streams. +- Baud rate, data bits, parity, stop bits, RTS/CTS, DTR policy, and RTS threshold controls. +- Runtime counters and safe restart after working-configuration changes. +- Explicit save/load/default/reset behavior; edits are not silently persisted. + +### Phase 2 — Session broker + +Implemented and hardware-validated: + +- Up to eight generation-safe broker clients. +- Exactly one writer lease with multiple observers. +- Bounded per-client output and event queues. +- Slow-observer isolation and drop accounting. +- Writer grant, release, denial, revocation, and forced administrative reassignment. +- DTR `on-connect` integration. +- Partial-transfer and retryable no-progress semantics for transports. +- Broker payload storage placed preferentially in PSRAM while FreeRTOS control structures remain in internal RAM. + +### Phase 3 — Native USB CDC-ACM + +Implemented and hardware-validated: + +- Native ESP32-S3 USB CDC-ACM serial transport. +- Automatic broker connection when the host opens the port with DTR asserted. +- Opportunistic writer acquisition with observer fallback. +- Binary-transparent bidirectional serial data. +- Supported host line-coding application while USB owns the writer lease. +- Disconnect cleanup, bounded buffering, counters, and UART0 lifecycle controls. + +### Phase 4 — Wi-Fi foundation + +Implemented and hardware-validated: + +- Four persistent station profiles with priority ordering. +- WPA2/WPA3 mixed mode and WPA3-SAE-required mode. +- Explicit no-echo secret entry through UART0. +- AP policies `off`, `fallback`, and `always`. +- Random persistent fallback-AP password and MAC-derived default SSID. +- Bounded profile attempts, DHCP deadlines, failover, and retry backoff. +- Root and `wifi` aliases for `ping`, `nslookup`, and `traceroute`. +- Wi-Fi/lwIP payload allocation configured to prefer PSRAM with capacities pinned in `sdkconfig.defaults`. + +A WPA3-required profile correctly rejects an AP that does not advertise the required Protected Management Frame capability; mixed mode is the compatible choice for such an AP. + +### Phase 5A — Authenticated HTTPS foundation + +Implemented and hardware-validated: + +- HTTPS-only service on TCP port 443 using ESP-IDF mbedTLS. +- Persistent random shared `admin` credential. +- Persistent device-specific ECDSA P-256 certificate and private key. +- Physical-console credential retrieval, rotation, certificate inspection, rotation, and recovery. +- Explicit failure behavior that preserves UART0, USB, serial, and Wi-Fi recovery paths. +- Credential rotation revocation hooks shared with WebSocket and SSH sessions. + +### Phase 5B — Offline browser terminal + +Implemented and hardware-validated: + +- Vendored xterm.js and FitAddon assets; no Internet or CDN dependency. +- CSP-compatible page structure with scripts served as separate resources. +- Authenticated status and one-time WebSocket-ticket APIs. +- Binary WebSocket serial frames and broker writer/observer integration. +- Two bounded browser-terminal sessions. +- Writer request/release controls and role indication. +- Bounded TLS/socket behavior and slow-observer isolation. +- Viewport-constrained terminal fitting without recursive page growth. +- Validation with multiple clients, ANSI colors, advanced escape sequences, and full-screen terminal software. + +## Current phase + +### Phase 6 — Authenticated SSH serial transport + +The initial SSH implementation is present and has passed single-service hardware tests. It provides: + +- wolfSSH on TCP port 22 while HTTPS remains on mbedTLS. +- Password authentication using the shared `admin` credential. +- A separate persistent ECDSA P-256 SSH host key and OpenSSH-compatible fingerprint. +- Interactive shell/PTY channels only. +- No `exec`, subsystem, SFTP, SCP, agent forwarding, or TCP forwarding. +- Two fixed session/handshake slots. +- Three password attempts and a 15-second handshake deadline. +- No broker client before successful authentication and shell negotiation. +- Binary-transparent serial forwarding with no in-band writer-control protocol. +- Opportunistic writer acquisition and observer behavior through the common broker. +- Credential-rotation revocation and UART0 session/host-key administration. +- A single bounded wolfSSH owner task pinned to CPU 1. +- wolfSSL/wolfSSH allocations that prefer PSRAM with internal-memory fallback. +- A per-call wolfSSH receive budget to prevent pipelined input from monopolizing the owner task. +- Internal task stacks retained for safety during flash/NVS cache-disable windows. +- Heap and SSH stack telemetry through `memory` and `ssh status`. + +SSH login, host-key verification, serial traffic, Web Serial coexistence, and USB CDC coexistence have been validated. Before the latest resource-placement changes, however, the following three-client arrangement could stall or fail the web service: + +1. USB CDC connected as writer. +2. Web serial connected as observer. +3. SSH connected as observer. + +The mitigation moves broker and Wi-Fi/lwIP payload pressure toward PSRAM and moves SSH processing away from the CPU 0 Wi-Fi/web path. Phase 6 remains open until that exact arrangement is revalidated. + +#### Phase 6 completion gate + +After flashing the current build: + +1. At idle, record: + + ```text + memory + ssh status + ``` + +2. Reproduce this order exactly: + + 1. Connect USB CDC and confirm it owns the writer lease. + 2. Connect one WebSocket terminal and confirm observer status. + 3. Connect one SSH terminal and confirm observer status. + +3. While all three are connected, collect: + + ```text + memory + ssh status + web status + web counters + ssh counters + broker clients + ``` + +4. Exercise sustained RS-232 output, writer release/reassignment, and input through the newly selected writer. Verify that all observers continue receiving data and that UART0 remains responsive. +5. Disconnect the transports in different orders and verify that sockets, broker clients, and writer ownership are cleaned up. +6. Capture the complete UART0 log if any failure occurs, especially task-watchdog output, backtraces, `ESP_ERR_NO_MEM`, TLS allocation failures, lwIP/pbuf failures, or HTTP server accept/session errors. + +Interpret the telemetry as follows: + +- Low internal or DMA-capable free space/largest block indicates more internal-memory reduction is needed. +- Healthy heap metrics combined with a watchdog or stall indicates a scheduling or blocking-path problem rather than simple exhaustion. +- A consistently large SSH stack minimum-free value permits a measured reduction from the current 20 KiB stack; it must not be reduced speculatively. +- Low or fragmented PSRAM indicates that queue, socket, or allocation capacities need review. + +Phase 6 is complete when this concurrent configuration remains stable, cleanup succeeds, the relevant counters show no unexplained loss, and no watchdog or memory-allocation error occurs. + +## Planned phases + +The order below is the current plan. Detailed requirements should be finalized at the start of each phase, and optional features must not weaken the completed serial and recovery paths. + +### Phase 7 — Local display and buttons + +Add a standalone local status/control interface without making it a dependency of the serial core. + +Planned work: + +- Select the actual display, button arrangement, electrical interface, and remaining GPIO assignment before implementation. +- Add a board-profile abstraction where required instead of scattering new pin assumptions. +- Show concise serial, Wi-Fi, service, client-count, and writer-owner status. +- Provide safe button actions for navigation and common lifecycle or writer-control operations. +- Require deliberate confirmation for destructive or security-sensitive actions. +- Avoid displaying passwords or private-key material during ordinary operation. +- Keep UART0 fully functional as the authoritative recovery interface if the display or buttons fail. +- Bound display refresh and event processing so they cannot interfere with UART or network transport timing. + +Completion requires electrical validation, operation with all current transports, and documented recovery behavior. + +### Phase 8 — Security and production hardening + +Harden stored credentials, boot integrity, update trust, failure diagnostics, and dependency maintenance. This phase must be staged carefully because some ESP32 eFuse operations are irreversible. + +Planned work: + +- Define development, test, and production security profiles. +- Enable and validate NVS encryption for Wi-Fi, HTTPS, and SSH material. +- Evaluate flash encryption and PSRAM encryption for the target hardware and threat model. +- Enable secure boot with a documented signing-key and recovery process. +- Define secret-safe core-dump handling; do not expose credentials or session material in diagnostics. +- Audit key and session-material zeroization across application, mbedTLS, wolfSSL, and wolfSSH allocations. +- Add authentication throttling, lockout/backoff, and security-event counters where appropriate. +- Review TLS/SSH algorithms, certificate policy, password policy, and browser security headers. +- Review current wolfSSL/wolfSSH and ESP-IDF security releases instead of treating integration versions as permanent pins. +- Recheck all third-party licenses and preserve source/license notices. +- Document provisioning, key rotation, factory reset, backup, recovery, and device decommissioning. + +Security features must be validated first on expendable hardware before any irreversible production configuration is recommended. + +### Phase 9 — Authenticated OTA and rollback + +Use the existing dual 4 MiB application slots and `otadata` partition to add safe firmware updates. + +Planned work: + +- Accept only authenticated, integrity-checked firmware images compatible with the selected secure-boot policy. +- Support an update path that does not expose a plaintext management endpoint. +- Stream downloads with bounded RAM use and explicit progress/error reporting. +- Preserve serial, Wi-Fi, HTTPS, and SSH configuration across successful updates. +- Mark new firmware pending until a post-boot health check succeeds. +- Roll back automatically after failed boot or health confirmation. +- Define power-loss behavior for every update stage. +- Keep UART0 recovery and wired re-flashing documented and functional. +- Add version/compatibility checks and reject accidental downgrade unless an explicit policy allows it. + +OTA is complete only after successful update, interrupted-update, invalid-image, rollback, and configuration-persistence tests on hardware. + +### Phase 10 — BLE + +Evaluate and, if resource limits permit, add BLE as another bounded transport or provisioning path. + +Planned work: + +- Decide whether BLE is primarily a serial transport, a provisioning interface, or both. +- Require authenticated pairing/bonding appropriate to the device threat model. +- Integrate serial access through the existing broker rather than bypassing writer ownership. +- Preserve binary transparency through explicit framing and MTU-aware partial transfers. +- Bound connections, queues, retries, and advertising behavior. +- Measure coexistence with 2.4 GHz Wi-Fi, HTTPS, WebSocket, SSH, USB CDC, and UART service. +- Store bond/provisioning data under the hardened secret-storage policy. +- Provide UART0 controls and a physical recovery path for clearing BLE state. + +BLE remains subordinate to stable Wi-Fi and serial operation; it should be omitted if coexistence or memory costs cannot be bounded acceptably. + +### Phase 11 — Advanced network integration + +These features are candidates, not current commitments: + +- mDNS advertisement for the certificate’s device-specific `.local` name and selected services. +- Enterprise Wi-Fi support, subject to credential-storage and certificate-validation design. +- IPv6 behavior and diagnostics beyond the current basic support. +- WireGuard feasibility evaluation, including RAM, CPU, licensing, key storage, routing, and recovery impact. + +The device is not intended to become a general-purpose router. Captive-portal interception, unauthenticated DNS redirection, NAPT, and a plaintext serial listener remain out of scope unless the project requirements are explicitly revised. + +### Phase 12 — Optional filesystem-backed features + +The `storage` partition is reserved but not currently mounted. Possible uses must be justified individually: + +- LittleFS-backed web assets to decouple large static files from the application image. +- Bounded diagnostic logs with explicit retention and secret-redaction rules. +- Import/export of non-secret configuration. +- Optional user files needed by later administration features. + +Before enabling storage, define corruption recovery, wear limits, quotas, atomic update behavior, compatibility across OTA slots, and whether encryption is required. Serial operation and recovery must not depend on a mountable filesystem. + +## Cross-phase release gates + +Every phase should satisfy the following before being marked complete: + +1. A clean release build for the ESP32-S3 N16R8 target. +2. No new compiler or project diagnostics attributable to the change. +3. Focused automated tests where practical and documented hardware tests where hardware behavior is involved. +4. UART0, native USB CDC, and previously completed network transports still operate. +5. Broker writer/observer semantics and binary transparency remain intact. +6. Failure, disconnect, stop/start, reboot, and stale-session cleanup paths are exercised. +7. Runtime memory, stack, socket, and queue bounds are measured under maximum supported concurrency. +8. Secrets remain absent from ordinary status output and logs. +9. [Command reference](command_reference.md), [Hardware wiring](wiring.md), electrical tests, and this roadmap are updated as applicable. +10. New dependencies receive security, maintenance, and GPL-3.0 compatibility review. + +## Explicitly deferred work + +The following are not implemented merely because flash partitions or library support exist: + +- NVS, flash, or PSRAM encryption. +- Secure boot or production eFuse provisioning. +- OTA download, image confirmation, or rollback policy. +- Core-dump collection or secret-safe core-dump processing. +- Filesystem mounting. +- Public-key SSH user authentication. +- SFTP, SCP, SSH `exec`, forwarding, or subsystems. +- General routing, NAPT, captive-portal interception, or unauthenticated serial access. + +Deferring these features is intentional: each changes the security model, recovery behavior, or bounded-resource assumptions and therefore requires its own design and validation gate.