feat: add bounded admin WebSocket backend (Phase 8D.5)
- Require current admin cookie sessions, Origin checks and single-use tickets - Reuse the shared console with session-aware authorization and slot allocation - Add HTTPD-owned I/O, bounded buffering and revocation cleanup - Prevent LRU eviction of serial clients and stale admin socket closure - Reject unsupported web-shell mutations before side effects - Add host regressions, a smoke client and resource accounting Validated by user sign-off after a 15-minute full-client soak at 230400 baud, with a few broker drops under heavy output. Browser UI remains for Phase 8D.6; numeric memory reserves remain open.
This commit is contained in:
@@ -98,9 +98,9 @@ TinyUSB callbacks enqueue/copy data and state; the transport task owns broker li
|
||||
|
||||
`web_server` runs HTTPS only on port 443 using the device-specific self-signed P-256 certificate from `web_security`. Current routes provide the UI, static assets, status, ticket issuance, and serial WebSocket upgrade.
|
||||
|
||||
HTTPS login uses `user_database` and opaque server-side cookie sessions; Basic authentication and its cache are removed in 8D.3. Before administrator bootstrap, the migrated role-`user` account is synchronized from the legacy credential, so that username/password can authenticate through the database; after bootstrap, the legacy blob is independent recovery material and is no longer consulted for authentication or synchronized into role-based accounts. Both `user` and `admin` roles currently receive the same web status/terminal experience; web administration is not implemented.
|
||||
HTTPS login uses `user_database` and opaque server-side cookie sessions; Basic authentication and its cache are removed in 8D.3. Before administrator bootstrap, the migrated role-`user` account is synchronized from the legacy credential, so that username/password can authenticate through the database; after bootstrap, the legacy blob is independent recovery material and is no longer consulted for authentication or synchronized into role-based accounts. Both roles retain the same shipped web status/serial UI. 8D.5 adds an admin-only backend without a normal UI entry.
|
||||
|
||||
`web_cookie_auth` owns login/session/logout policy: four 120-second digest-only pre-login challenges, explicit same-origin bootstrap, five credential verifications per 60-second global window, and no live-record eviction. Host-only `__Host-` Secure/HttpOnly/SameSite=Strict cookies have absolute lifetimes. Login consumes a challenge, validates bounded JSON and issues a fresh session; logout invalidates only its originating session. Mutations require CSRF and strict canonical HTTPS Origin; serial upgrade requires matching cookie/Origin/ticket. Neither role has web administration yet.
|
||||
`web_cookie_auth` owns login/session/logout policy: four 120-second digest-only pre-login challenges, explicit same-origin bootstrap, five credential verifications per 60-second global window, and no live-record eviction. Host-only `__Host-` Secure/HttpOnly/SameSite=Strict cookies have absolute lifetimes. Login consumes a challenge, validates bounded JSON and issues a fresh session; logout invalidates only its originating session. Mutations require CSRF and strict canonical HTTPS Origin; serial/admin upgrades require matching cookie/Origin/ticket, with admin role additionally required by the admin endpoints.
|
||||
|
||||
`web_session_store` holds four static records with token/origin digests, copied principal, separate CSRF state, one-hour absolute expiry and non-reused 64-bit session IDs. These are live cookie sessions in 8D.3, with no sliding renewal. A portMUX protects short state copies/mutations; database/RNG/SHA calls occur outside it. Resolution rechecks ID/expiry after database validation; issuance also checks an invalidation epoch. Stop wipes records without resetting IDs/epochs. Only admitted HTTPS starts initialize the store; failed starts and accepted stops disable it before cleanup. Authentication/store-init failure now gates HTTPS startup rather than falling back to Basic. Sensitive views must be wiped by callers; snapshots contain only counts and storage sizes. Focused host checks live in `tests/web_session_store/`.
|
||||
|
||||
@@ -114,6 +114,10 @@ Web serial initialization is failure-isolated from the base HTTPS service: if th
|
||||
|
||||
`web_ui.c` contains authored index/application strings and response policy; it validates `/api/session` before connect/restore, adds explicit Sign out, and cancels stale work on 401/logout/page exit. `web_login_ui` is a standalone public page without protected-asset dependencies. Both authentication documents and app script are no-store. Its restrictive CSP contains a hard-coded hash of the inline loader, so those two must change atomically; preserve same-origin connections, no-referrer behavior, frame denial, and the existing cache policy. `web_assets_data.c` contains checked-in generated arrays for vendored compressed xterm assets and the logo. Normal builds compile these arrays directly; they do not regenerate assets.
|
||||
|
||||
### Browser admin backend
|
||||
|
||||
8D.5 additionally supplies `web_admin_transport` and `web_admin_tickets`: one optional admin socket, two 30-second digest-only tickets bound to current originating session/principal, the same two shared console slots, no serial broker client. Ticket POST requires cookie/Origin/CSRF/admin; ordinary GET upgrade requires cookie/Origin/admin/ticket and console admission before 101. Six total HTTPS sockets remain, LRU purge is disabled, and two routes bring the handler budget to 16. Optional admin registration/PSRAM allocation failures do not take down M1. A 20 ms ESP timer queues at most one HTTPD poll, with no new task; only HTTPD accesses the 1,552-byte PSRAM-only RX/TX payload or socket IO. Notifiers close the generation-qualified console and flag the socket. HTTPD shuts down the verified current fd directly and owns subsequent read cleanup, avoiding IDF's queued reusable `sock_db *` close race. Detach fences submissions; failed stop retains ownership, and queued state is retired only after successful HTTPD stop. Console dispatcher/prompt and owner input/output/idle checks enforce session and principal currentness. WEB supports deferred self-close only; parsed canonical policy denies unsupported lifecycle/network/account mutations before handler side effects. No normal UI entry, typed settings or lifecycle parity is included. See `docs/phase8d5_implementation.md` for validation limits and exact restrictions.
|
||||
|
||||
### SSH
|
||||
|
||||
`ssh_transport` uses wolfSSH on port 22 with two fixed session/handshake slots. Initialization calls `wolfSSH_Init()` in the caller before task creation; after that, one owner task pinned to core 1 exclusively owns runtime contexts/sessions and wolfSSH calls. It enforces bounded handshakes, authentication attempts, receive work, and session buffers.
|
||||
@@ -157,7 +161,7 @@ admin SSH line editor ----/ |
|
||||
|
||||
The dispatcher is the sole caller of `esp_console_run()`, serializing UART0 and all admin SSH commands. This is required because the console registry is treated as non-reentrant, but it also means a long command or interactive prompt blocks all administration entry routes.
|
||||
|
||||
Phase 8D.4 retains this module and exposes `admin_ssh_console_open_owned()`: copied transport-qualified slot/session/generation identity plus a firmware-lifetime immutable owner adapter. The existing two console slots are shared, not multiplied per frontend; active/executing slots cannot be replaced. Owners serialize per-session input, consume output and enforce transport liveness; completion scratch is claimed nonblockingly across owners. The existing control task calls drain/lifecycle adapters outside console locks. SSH implements its adapter and compatible admission entry point in `ssh_transport.c`, using generation-checked published snapshots and existing control APIs, never wolfSSH from the control task. `SELF_CLOSE` is owner-relative; legacy SSH actions remain SSH-specific and unsupported owner actions are rejected. No browser admin transport is yet implemented.
|
||||
The 8D.4/8D.5 boundary retains `admin_ssh_console_open_owned()` and adds available-slot admission for runtime SSH/browser owners: copied transport-qualified slot/session/generation identity plus a firmware-lifetime immutable owner adapter. The existing two console slots are shared, not multiplied per frontend; active/executing slots cannot be replaced. Owners serialize per-session input, consume output and enforce transport liveness; completion scratch is claimed nonblockingly across owners. The existing control task calls drain/lifecycle adapters outside console locks. SSH uses generation-checked published snapshots, principal copies and its assigned console index, never wolfSSH from the control task. `SELF_CLOSE` is owner-relative; legacy SSH actions remain SSH-specific and unsupported owner actions are rejected. Dispatcher-side owner `is_current` checks run outside console locks, with full identity recheck after validation. Commands revalidate immediately before the runner; prompts revalidate before publication and after waits (250 ms polling plus check/scheduling latency), rejecting revoked submitted input and stale wakes. SSH preserves close intent through external-close consumption. Consumed output is wiped. These checks do not cancel arbitrary executing handlers or replace owner-side input/output and lifecycle validation.
|
||||
|
||||
For SSH, standard output/error is redirected to the invoking session's bounded output ring. `console_input` routes visible or hidden prompts to UART0 or the active SSH session. `exit` and Ctrl+D on an empty admin SSH line use bounded deferred self-disconnect after their acknowledgement drains; role-`user` SSH remains a binary-transparent serial stream. Session tokens include slot and generation so late queued work cannot attach to a reused SSH slot. Only the SSH owner task moves ring output through wolfSSH.
|
||||
|
||||
|
||||
@@ -74,6 +74,13 @@ This is a semantic map, not a complete file inventory. Start here, then read the
|
||||
- 8D.3 UI: `src/web_login_ui.{c,h}` serves standalone `/login`; `web_ui.c` validates session before serial connect/restore and handles logout/401 safely. Both scripts hash-bound, auth documents/app no-store. Tests: `python3 tests/web_login_ui/run.py` and `python3 tests/web_ui_session/run.py`. Live cutover host-tested/build-verified, M1 validated by user sign-off (numeric reserves open): `docs/phase8d3_implementation.md`.
|
||||
- Asset constraint: `web_assets_data.c` is checked-in generated input to the build; do not hand-edit or regenerate casually.
|
||||
|
||||
### Browser admin backend (8D.5)
|
||||
|
||||
- Files: `src/web_admin_transport.{c,h}`, `src/web_admin_tickets.{c,h}`, protected registration/lifecycle in `web_server.c`, revocation through `web_serial_transport_revoke_*`, diagnostics in `web_console.c`.
|
||||
- Routes: CSRF-protected admin-only `POST /api/admin/ws-ticket`; ordinary `GET /ws/admin` with cookie/Origin/ticket/shared-console admission before explicit 101. No UI entry or broker client. One socket, two tickets, existing two shared console slots; six total HTTPD sockets, LRU disabled, 16 URI handlers.
|
||||
- Ownership: 20 ms ESP timer queues at most one HTTPD poll, no new task; HTTPD owns 1,552 B PSRAM-only payload and IO. Closure uses HTTPD-owned `shutdown`, not IDF's reusable-pointer queued close. Detach fences submitters; only successful HTTPD stop retires queued state before restart. Session/principal currentness and generation checks protect all sensitive boundaries.
|
||||
- Tests: `python3 tests/web_admin_transport/run.py --tickets`, `python3 tests/web_admin_transport/server_lifecycle.py`, `python3 tests/web_cookie_auth/run.py --admin`; manual smoke client/procedure in `tests/web_admin_transport/README.md` and `docs/phase8d5_implementation.md`. Final shutdown fix is host-tested and build-verified by the parent's sequential final `pio run`; target validation remains pending.
|
||||
|
||||
## SSH
|
||||
|
||||
**Responsibility:** authenticate SSH, route users to serial and administrators to the command dispatcher, and own wolfSSH lifecycle.
|
||||
@@ -110,7 +117,7 @@ This is a semantic map, not a complete file inventory. Start here, then read the
|
||||
- Ownership: dispatcher is sole `esp_console_run()` caller; the SSH owner exclusively performs post-initialization wolfSSH runtime calls.
|
||||
- Lifecycle: remote session tokens include slot generation; fixed output/history/prompt state is wiped immediately on idle close or after an executing handler returns. Admin SSH `exit` and Ctrl+D on an empty command line request bounded deferred self-disconnect after best-effort output draining.
|
||||
- Constraint: one slow command or prompt serializes all administration. Admin SSH is unavailable until command registration and UART frontend creation complete; supported deferred actions wait only for a bounded application-buffer drain heuristic.
|
||||
- 8D.4 boundary: `admin_ssh_console_open_owned()` copies transport-qualified identity and retains an immutable firmware-lifetime drain/lifecycle adapter; the existing two slots are shared. SSH compatibility admission/adapter resides in `ssh_transport.c`; no browser admin caller yet. Owners handle liveness/output; completion scratch is nonblockingly serialized. Focused host command: `python3 tests/admin_console_boundary/run.py`.
|
||||
- 8D.4/8D.5 boundary: `admin_ssh_console_open_owned()` retains explicit-index admission; runtime SSH and browser owners use `admin_ssh_console_open_available()` for the same two slots. Copied transport-qualified identity and immutable firmware-lifetime currentness/drain/lifecycle adapters; SSH publishes its allocated console index separately from its physical SSH slot. Owners handle liveness/output; dispatcher and prompt waits additionally require owner currentness (250 ms polling plus check/scheduling latency). SSH publishes locked principal copies; consumed console output is wiped. Completion scratch is nonblockingly serialized. Browser unsupported lifecycle/account mutations are rejected before execution. Focused host command: `python3 tests/admin_console_boundary/run.py`.
|
||||
|
||||
## Wi-Fi
|
||||
|
||||
|
||||
@@ -4,6 +4,12 @@ This file is working memory. Update it during active work and before handoff; do
|
||||
|
||||
## Development state
|
||||
|
||||
- **8D.5 validated by explicit user sign-off (2026-09-06):** User supplied settled cold-boot telemetry and reports successful **15-minute full-client-mix active-use soak at 230400 baud**, with a few broker drops under extremely fast/dmesg output, then explicitly closes 8D.5. Supersedes older pending/incomplete notes below. Boot internal/DMA/PSRAM free **70,876 / 63,120 / 8,246,360 B**, minima **59,560 / 51,804 / 8,242,140 B**, largest **31,744 / 31,744 / 8,126,464 B**; SSH stack minimum-free **18,472 B**. Admin initialized/attached with ESP_OK and 167/240/1,552 B static/ticket/payload accounting; no active clients at boot. Full mix/soak is user-reported, not evidenced by the idle snapshot; exact drop count/client, flashed revision and loaded/post-soak/cleanup telemetry not supplied. See `docs/phase8d5_implementation.md`. These limits do not reopen sign-off or imply zero-drop operation. Numeric reserves/runtime socket cost remain open. No new build/device/source action for sign-off. Wait for separate **8D.6** request; M2 not yet complete.
|
||||
|
||||
- **8D.5 backend implemented / host-tested / build-verified (2026-09-06), target pending:** Preserved interrupted tickets/transport/server/shared-console/SSH/policy/revocation work and inherited test tooling. One admin socket, two tickets, existing two shared console slots, 1,552 B PSRAM-only payload, 20 ms ESP timer/no new task, six HTTPS sockets/no LRU eviction, 16 URI handlers. Added real cookie/store/ticket/transport endpoint integration tests. Final IDF lifecycle review found queued `httpd_sess_trigger_close` retains a reusable socket-slot pointer; changed the admin path to HTTPD-owned `shutdown`, with HTTPD read cleanup and retry/reuse regressions. Transport 19 groups + tickets 12, server lifecycle 11, combined endpoints, both console suites, store/serial, cookie/parser (268), login UI (8), serial UI (9) all pass; diff check passes. **Parent reports sequential final `pio run` after the shutdown fix passed in 23.55 s at 95,580 B RAM / 1,637,273 B flash: +416/+9,224 versus prerequisite, +496/+10,100 versus 8D.4, +1,048/+37,300 versus 8D.0.** This supersedes the continuation's earlier pre-fix build; history remains in the implementation record. Parent's final independent security integration reviewer reported no actionable findings. Target ELF payload/slot/ticket sizes 1,552/80/96 B, ticket state+lock 240 B, transport static symbols 167 B before padding, ESP timer internal allocation 32 B before heap overhead. See `docs/phase8d5_implementation.md` for exact commands, inherited-versus-final fixes, resource accounting, restrictions and manual target/client checklist. Client `--help` only exercised in the continuation; this final evidence update changed documentation only, with no source/tests/build/device/network/asset/UI/upload/commit action. **Stop before 8D.6; target acceptance, runtime socket cost, numeric reserves and M2 remain open.** This supersedes the prerequisite-only pause below.
|
||||
|
||||
- **8D.5 prerequisite resumed / host-tested / build-verified (2026-09-06), target pending:** User requested continuation. Preserved existing uncommitted dispatcher/owner-currentness and prompt-cleanup changes in the console/SSH adapter; extended tests to production SSH snapshot/principal publication and wiping. Both console suites pass; independent production review found no actionable defects. `pio run` passes in **38.46 s**, **95,164 B RAM / 1,628,049 B flash** (**+80 / +876 B** versus recorded 8D.4); map confirms 80 B for two principal copies. No new tasks/routes/sockets/UI or backend yet. See `docs/phase8d5_implementation.md` for contract, scope split, test limits and target checklist. **8D.5/M2 incomplete.** Obtain target regression or explicit user decision before stacking live backend work. Exact next increment is still 8D.5 tickets/HTTPD-owned admin transport/protected admission/revocation/restrictions and integration checks, not 8D.6. Previous 8D.4/M1 sign-offs stand; numeric reserves remain open. This supersedes older wait-for-8D.5-request notes below.
|
||||
|
||||
- **8D.4 validated by explicit user sign-off (2026-09-06):** User confirms successful empty Enter and soak testing and explicitly closes Phase 8D.4. This supersedes older pending/in-progress notes below, including the 8D.3 empty-Enter target retest. Boot/full-client-mix evidence and sign-off are in `docs/phase8d4_implementation.md`. Exact soak duration/revision, post-soak/cleanup telemetry and other detailed checklist results were not supplied; these are evidence limitations, not blockers to the user-approved closure or claims of execution. Numeric reserves remain open. No new build/device operation for sign-off. Wait for a separate **8D.5** request; no browser admin backend or M2 completion is claimed.
|
||||
|
||||
- **8D.4 target validation in progress (2026-09-06):** User supplied clean-boot/full-client-mix UART0 evidence, recorded in `docs/phase8d4_implementation.md`. Loaded internal/DMA/PSRAM free **34,632 / 26,876 / 8,089,284 B**, minima **20,648 / 12,892 / 8,077,516 B**, largest **25,600 / 25,600 / 7,995,392 B**; SSH stack minimum-free **16,280 B** (boot **18,472 B**). Four broker clients at 115200 baud: SSH writer, USB and two web observers; user/admin SSH and both browser roles admitted successfully, no reported I/O/transport failures. One cumulative SSH broker revocation retained without diagnosis. Exact flashed revision/duration not supplied. Console-specific regressions, lifecycle/soak/cleanup and explicit phase sign-off remain pending; numeric reserves remain open. No 8D.5 request or implementation.
|
||||
@@ -60,8 +66,8 @@ Based on checked-in source plus `README.md` and `docs/roadmap.md`:
|
||||
## Clearly incomplete or transitional areas
|
||||
|
||||
- Phase 8C hardware validation passed, including route separation, shared command serialization, history/completion, prompts, output backpressure, revocation during queued work, deferred SSH lifecycle/reboot actions, and full concurrent transport operation. At 460800 baud with SSH and WebSocket clients in parallel, substantial packet drops and slow display controls were observed under load, without memory exhaustion; no baud-rate reduction is planned.
|
||||
- Current HTTPS has no web-based user administration and gives both roles the same status/terminal routes.
|
||||
- Browser authentication now uses cookie login/logout without Basic fallback. M1 target checkpoint is signed off; browser administrative routes remain future separately requested M2 work.
|
||||
- Current shipped HTTPS UI gives both roles the same status/serial terminal. The 8D.5 admin backend is implemented without a UI entry; its temporary policy denies all user mutations and unsupported self-affecting lifecycle commands.
|
||||
- Browser authentication uses cookie login/logout without Basic fallback. M1 target checkpoint is signed off; 8D.5 is implemented, host-tested and build-verified, with target gates recorded above. Browser selector/lifecycle parity remain future separately requested 8D.6/8D.7 work, not completed M2.
|
||||
- NVS encryption, secure boot/flash encryption review, production certificate/provisioning policy, and OTA are not implemented. HTTPS login has a bounded global five-verifications/60-second throttle, not comprehensive cross-transport DoS protection.
|
||||
|
||||
## Known inconsistencies
|
||||
|
||||
@@ -82,12 +82,14 @@ Phase 8D.2 adds a third identity: non-reused 64-bit originating web-session IDs
|
||||
|
||||
**Consequence for future changes:** Actions that would invalidate their own SSH transport should integrate with deferred control when acknowledgement preservation matters. Prevent new input while an action is pending, keep the wait bounded, and do not describe it as guaranteed delivery.
|
||||
|
||||
Phase 8D.4 routes drain/lifecycle operations through a firmware-lifetime immutable owner adapter on the existing control task, outside console locks. Tokens include a transport namespace; owners revalidate full identity and marshal to their transport APIs. `SELF_CLOSE` targets the invoking frontend while existing SSH action meanings remain unchanged. Unsupported actions must fail before side effects. The two console slots remain a shared bounded pool, with no hypothetical browser capacity allocated.
|
||||
Phase 8D.4 routes drain/lifecycle operations through a firmware-lifetime immutable owner adapter on the existing control task, outside console locks. Tokens include a transport namespace; owners revalidate full identity and marshal to their transport APIs. `SELF_CLOSE` targets the invoking frontend while existing SSH action meanings remain unchanged. Unsupported actions must fail before side effects. The two console slots remain a shared bounded pool, with no hypothetical browser capacity allocated. The 8D.5 prerequisite additionally requires owner currentness on the dispatcher, outside console locks, before commands and during prompts; account currentness alone cannot establish originating browser-session liveness. Recheck token identity after external validation, reject revoked submitted replies, and wipe consumed output. Polling is not a hard cancellation deadline and cannot roll back arbitrary handlers; owners retain admission/input/output/lifecycle responsibilities.
|
||||
|
||||
**Relevant files:** `src/admin_ssh_console.c`, `src/system_console.c`, `src/ssh_console.c`, `src/ssh_transport.c`
|
||||
|
||||
## Authentication uses copied principals and fail-safe currentness checks
|
||||
|
||||
**8D.5 web owner extension:** Browser and runtime SSH admission allocate from the same two console slots; a physical SSH slot is not a console index. WEB supports owner-relative self-close only and rejects unsupported network/lifecycle/account mutations at parsed command policy before execution. One web-admin socket and two tickets do not increase six-socket HTTPD capacity; disable LRU rather than evict retained serial clients. The optional owner uses one PSRAM-only payload and ESP timer scheduling, not a new task. HTTPD alone sends/shuts down its verified current fd. Do not use IDF's queued raw-`sock_db *` close from admin polling: free/reuse before that work executes could close a replacement. Detach must fence queue submissions before HTTPD stop; retire queued markers only after successful stop, retaining ownership across failures. No browser UI or generic HTTP command runner is part of this boundary.
|
||||
|
||||
**Decision:** Network sessions retain secret-free copied principals. Account mutations invalidate generations/IDs; after commit, the command layer requests best-effort targeted transport revocation, while ongoing currentness checks are authoritative.
|
||||
|
||||
**Rationale/evidence:** `user_database` issues principals without secrets; web/SSH check currentness during admission and active sessions. Mutating console paths call transport revocation hooks.
|
||||
|
||||
Reference in New Issue
Block a user