ESP32_Serial_Swiss_Army_Knife/docs/phase8d5_implementation.md

# Phase 8D.5 — Admin WebSocket backend

Status (2026-09-06): **8D.5 backend implemented / host-tested / build-verified / validated by explicit user sign-off.** The user supplied settled cold-boot telemetry and reported a successful 15-minute full-client-mix active-use soak at 230400 baud, with a few broker drops under heavy output, and explicitly closed 8D.5. Final build: **23.55 s**, **95,580 B RAM / 1,637,273 B flash**. No 8D.6 UI work or M2 acceptance. Prior 8D.4/M1 sign-offs stand; numeric resource reserves remain open.

## Target Sign-Off (2026-09-06)

User reports: "With full client mix, running and active use for 15 mins, soaked, only a few dropped broker packets at 230400 baud with extremely fast and dmesg output. Mark 8D.5 as validated."

This is explicit phase acceptance and supersedes older pending/incomplete validation statements below and in project memory. The reported broker drops are preserved, not treated as zero-drop or byte-integrity evidence; no cause, exact count or affected client was supplied. This 230400-baud workload is distinct from earlier 115200-baud samples. No baud-rate or capacity reduction is made.

Settled cold-boot UART0 measurements supplied with sign-off:

| Heap (bytes) | Free | Minimum-free | Largest block |
|---|---:|---:|---:|
| Internal 8-bit | 70,876 | 59,560 | 31,744 |
| Internal DMA | 63,120 | 51,804 | 31,744 |
| External PSRAM | 8,246,360 | 8,242,140 | 8,126,464 |

SSH owner stack: **20,480 B configured / 18,472 B minimum-free**. Internal/DMA capabilities overlap; their free bytes are not additive. Minimum-free is the firmware's conservative sum of matching heap regions' lifetime minima.

- HTTPS and SSH initialized/running with ESP_OK, each with one successful start and zero startup failures. mDNS initialized/announced with ESP_OK.
- Admin backend initialized/attached with ESP_OK, no active admin socket or tickets, and all admin counters zero. Reported transport static/ticket/PSRAM payload storage **167 / 240 / 1,552 B**, matching implementation accounting.
- No SSH, cookie or serial WebSocket sessions, challenges, tickets or broker clients. Web/SSH traffic/authentication/failure counters zero at boot; this does not describe post-soak counters.
- UART service stopped, RS-232 owner idle, configuration **230400 8N1/no flow**, RX/TX pending zero. USB initialized/attached but host closed with DTR/RTS false and no broker client. Diagnostic 9600 host line coding does not reconfigure UART1.

Exact flashed revision, settling duration, loaded/post-soak/cleanup memory and counters, individual client identities and detailed checklist results were not supplied. The full client mix and 15-minute successful soak are user-reported, not reconstructed from the idle boot sample. Missing details remain evidence limitations, not blockers to user-approved phase closure or claims of unreported test execution. Runtime per-admin-socket cost and numeric reserve approval remain open. No agent build, device operation or source change was performed to record sign-off. **Next is 8D.6 only on a separate request; M2 remains incomplete.**

## Combined backend completion

The user explicitly authorized finishing the entire interrupted 8D.5 implementation, superseding the prerequisite pause below. Extensive uncommitted source/tests were preserved: ticket store, transport, protected server routes, shared-console allocator and SSH mapping, command restrictions, revocation integration, diagnostics, transport regressions, lifecycle harness and local smoke client. This continuation reviewed them, added real authenticated endpoint integration tests and fixed the final admin socket close/reuse race. No upload, erase, commit, generated asset or normal UI change.

### Admission and protocol

- `POST /api/admin/ws-ticket` runs the existing strict cookie/Origin/CSRF mutation policy, then admin-role validation. Two digest-only, non-evicting, single-use tickets expire after 30 seconds and bind the originating session ID and full current password principal. Crypto/database checks are outside critical sections; epoch/generation checks reject stale publication, consumption and prune work.
- `GET /ws/admin?ticket=<64 hex>` is an ordinary HTTP route, not an automatic WebSocket route. Cookie, strict Origin, current admin role, exact ticket shape/consumption, one transport-slot reservation and a free shared console slot precede explicit 101. Upgrade has no CSRF header requirement: the CSRF-protected ticket plus cookie/Origin authorizes it, including browser clients that cannot add custom WebSocket headers. Rejections never execute console commands.
- Exactly one admin socket, two admin tickets and the existing two shared console slots. SSH now retains its allocated console index separately from the physical SSH slot and resolves published owner state by full identity. Busy/executing console slots cannot be replaced. Admin does not join the serial broker or obtain a writer lease.
- Final unfragmented binary frames carry console input (maximum 512 bytes); binary output chunks are at most 1024 bytes. Text, fragmented, oversized and overlapping pending input fail closed. Partially consumed input has a five-second deadline, checked before retry. Consumed input/output and retired payload/console state are wiped. Empty binary frames must not invoke IDF's zero-length header probe twice.
- Saturating lifetime admin counters and allocation sizes are available through `web status`/`web counters`, without token, CSRF, verifier or private-key disclosure. `web clear-counters` does not reset admin counters; diagnostics say so.

### Ownership and failure isolation

- One permanent 20 ms ESP timer schedules at most one HTTPD poll. It does no database, console, payload or socket work. No new application task/stack/dispatcher is created. Blocking HTTPD queue-work configuration makes the optional admin initializer fail closed.
- HTTPD exclusively owns admission, frame input, payload mutation, output sends and session-context cleanup. Revocation/control callers only flag closure and close the generation-qualified console token. Authoritative session/principal checks guard admission, dispatcher execution/prompts, input, output and idle polls. These checks cannot roll back arbitrary already-running commands.
- Detach disables admission/tickets and console access before fencing timer submissions for up to two seconds. Timeout retains the live HTTPD handle and requires a stop retry. Failed SSL stop retains admin ownership; only successful HTTPD stop permits clearing queued-work state and reattachment. Queued polls after detach do no IO; successfully stopped HTTPD cannot execute discarded work.
- **Final lifecycle fix:** installed IDF 5.5.0 `httpd_sess_trigger_close()` queues a raw reusable `sock_db *`. A poll could queue closure, then a frame error free that slot and acceptance reuse it before the queued close executes, potentially closing an unrelated serial client. Admin polling now calls `shutdown(fd, SHUT_RDWR)` directly on HTTPD after checking its session context. HTTPD's next read owns deletion; there is no late queued close pointer. Failed shutdown retries on later polls, with send-failure accounting. This deliberately does not promise a graceful WebSocket close frame or peer delivery. The existing serial transport's use of IDF queued close was not changed; the new admin path cannot introduce this eviction route.
- HTTPS retains six client sockets, now with LRU purge disabled, and grows from 14 to 16 URI handlers. Full socket capacity can delay/refuse new HTTP/TLS connections rather than evict a retained serial writer. Optional admin allocation/registration failure preserves M1 routes and serial attachment; failed optional-ticket unregister leaves an authenticated but unattached/unavailable ticket handler, not a bypass.
- Logout invalidates its cookie session before serial/admin ticket/socket cleanup; account/global revocation follows the same order through the existing `web_serial_transport_revoke_*` integration hooks. Lost notifications still fail session/principal currentness. Unrelated session notifications do not close the admin socket.

### Temporary command restrictions

The parsed canonical command policy rejects unsupported actions before `esp_console_run()`, not after a handler has mutated configuration. From web: only `web status`, `wifi status`, `mdns status`; only bare `user`, `user status`, `user list`, `user show <name>` in the user group; no `reboot`, SSH stop/disconnect/reset or SSH host-key action except `ssh host-key info`. Thus web/network identity changes, all account mutations and one-time generated credentials remain unavailable here until the later lifecycle phase. Ordinary permitted commands, empty Enter and `exit`/empty-line Ctrl+D use the existing dispatcher/editor. Only owner-relative deferred self-close is supported by WEB. UART0 bootstrap/recovery remains physical-only; SSH policy otherwise remains unchanged. See the policy suite for quoted forms.

## Final local validation

All commands below were executed in this continuation and passed. Host compiler warnings are errors; tests are deterministic dependency interleavings, not real multicore execution.

| Command | Actual result |
| --- | --- |
| `python3 tests/web_admin_transport/run.py --tickets` | 19 transport groups plus 12 ticket groups; rerun after shutdown fix |
| `python3 tests/web_admin_transport/server_lifecycle.py` | 11 groups, including all 16 required registration failure positions, two optional positions, failed unregister, failed stop/retry and six-socket/no-LRU configuration |
| `python3 tests/web_cookie_auth/run.py --admin` | Real cookie policy, parser, store, tickets, transport and private adapter linked together; endpoint rejection before 101, cross-session replay burn, admission, isolated logout, missed account revocation, expiry and restart; rerun after fix |
| `python3 tests/web_cookie_auth/run.py` | Existing cookie/HTTPD policy and embedded store regressions pass |
| `python3 tests/admin_console_boundary/run.py` | Shared two-owner allocation, production SSH publication/mapping, queued currentness, prompts, deferred actions, history/completion and wiping pass |
| `python3 tests/admin_ssh_policy/run.py` | SSH policy and browser restrictions using installed IDF parser pass |
| `python3 tests/web_session_store/run.py` | Store API/failure/race suite passes with OpenSSL SHA-256 |
| `python3 tests/web_session_store/run.py --serial` | Serial/session binding, revocation, races and non-eviction regressions pass |
| `python3 tests/web_auth_parse/run.py` | 268 cases, zero failures |
| `python3 tests/web_login_ui/run.py` | C/header/CSP checks and eight browser-behavior groups pass |
| `python3 tests/web_ui_session/run.py` | C/header/CSP checks and nine browser-behavior groups pass |
| `python3 tests/web_admin_transport/client.py --help` | Local import/CLI smoke only; no network/device operation |
| `git diff --check` | Pass |

The combined endpoint harness doubles console execution/IO and the logout revocation hook (matching reviewed production ordering); the console harness separately runs real shared-console code. Installed HTTPD getter/setter/pending-reader functions are extracted, but TLS, handshake writes, actual HTTPD event processing and FreeRTOS are doubled. The server lifecycle harness extracts production lifecycle/table code, not live HTTPD. No sanitizer run/pass is claimed in this continuation; inherited harness notes record missing ASan/UBSan libraries. Manual client offline evidence in its README is inherited, not rerun here beyond `--help`.

### Firmware and resources

**Final build, parent-reported:** the necessary sequential `pio run` after the shutdown fix **passed in 23.55 seconds**, reporting **95,580 B linked RAM / 1,637,273 B flash** on the existing PlatformIO espressif32 6.12.0 / ESP-IDF 5.5.0, N16R8 release configuration. This verifies the final source, including the shutdown fix. Parent also reports the final independent security integration review found **no actionable findings**. This documentation-only follow-up ran no build or tests.

Historical build: the continuation's one 120-second-bounded `pio run` passed in **22.67 seconds**, at **95,580 B RAM / 1,637,277 B flash**, before the shutdown fix. Both affected production-C host suites passed after the fix; the parent's subsequent final build supersedes that earlier image for final-source verification and saves 4 B flash with unchanged linked RAM.

Final-image deltas (baselines not rebuilt):

| Baseline | RAM delta | Flash delta |
| --- | ---: | ---: |
| 8D.5 prerequisite: 95,164 / 1,628,049 B | +416 B | +9,224 B |
| 8D.4: 95,084 / 1,627,173 B | +496 B | +10,100 B |
| 8D.0: 94,532 / 1,599,973 B | +1,048 B | +37,300 B |

Target ELF/DWARF/map inspection, with no device access: admin payload **1,552 B PSRAM-only** (512 RX + 1024 TX + 16 metadata), slot **80 B**, ticket **96 B** x two, ticket state **232 B** + lock **8 B** = **240 B**. Transport static symbols total **167 B** before placement padding (168 B occupied); retained timer handle is included there. IDF `struct esp_timer` is **32 B**, allocated with internal/8-bit capabilities, excluding heap metadata. Payload/timer persist across HTTPS restarts; PSRAM allocation has no internal fallback. SSH adds two console-index bytes in published state and retains the prerequisite's 80 B principal copies. No added console rings, queue capacities or task stacks. Dynamic TLS/socket/request allocations, heap fragmentation, HTTPD/dispatcher/timer stack margins and internal/DMA reserves remain unmeasured; these static figures are not per-socket runtime cost or reserve approval.

## Target Regression Procedure

No target/network exercise was performed by the agent. The user's target sign-off is recorded above; this original checklist is retained as regression coverage, not as outstanding gates to that closure. Use the already present bounded, stdlib-only `tests/web_admin_transport/client.py`; full usage/security caveats are in its README. It prompts for credentials without echo, keeps cookies only in memory, never logs tickets/CSRF/credential metadata, and attempts logout in `finally`. Prefer trusted certificate/hostname validation; `--insecure` is explicit test-only exposure to active interception, not a local-routing guarantee. Its console output is intentionally raw terminal output: use a trusted target and do not record secret-bearing command output.

```sh
python3 tests/web_admin_transport/client.py --url https://device.local --cafile device-cert.pem --smoke --max-runtime 60
```

1. Have the operator flash the final build-verified image through the usual approved procedure. Record exact revision/diff, clean-boot and 60-second settled `memory`, `web status`, `ssh status` and broker/serial counters. Build verification does not establish target acceptance.
2. Run the smoke client separately with disposable role-user and role-admin accounts. Require user ticket 403; admin cookie/ticket/101, same-ticket replay 403, `help`, empty frame, empty Enter and `exit`, then logout and session 401. Repeat five times per role. No automatic credential retry; respect the five/60-second throttle.
3. Keep two browser serial sockets, USB and role-user SSH at 115200 baud, with a known sole writer; concurrently admit one admin SSH plus web admin. Verify the same broker client IDs/writer before and after web admin open/exit/failure. Attempt a second admin socket and fill both console slots with SSH before web admission: reject, never replace. Fill remaining HTTPS sockets; no serial eviction. Capture live TLS/heap cost rather than infer it from six configured sockets.
4. With a temporary authenticated development client (not a firmware endpoint), test missing/foreign/null/duplicate Origin, missing/duplicate cookie, absent/wrong CSRF on ticket POST, expired/wrong-session/replayed tickets and direct user-role upgrade. Require rejection before any 101. Check raw responses without publishing auth headers or ticket URLs. The supplied smoke client only automates the documented subset, not this full negative matrix.
5. Exercise shared command serialization with UART0 and admin SSH, completion/history, visible/hidden/cancelled prompts, disconnect/revoke/expiry while queued or prompting, and slow input/output. Use disposable secrets and approved existing non-restricted commands; do not type secrets into a retained browser developer-console history. Unsupported web lifecycle/account mutations must report rejection before any state change. The supplied smoke client is not interactive and does not claim prompt/completion coverage.
6. Logout one session with serial+admin; only that session's sockets/tickets close. Change its disposable account from UART0/admin SSH, test deletion/recreation and let a session reach its one-hour absolute expiry. Verify unrelated sessions, queued-command rejection, no stale prompt/output after slot reuse, and no lingering reserved console slot after an executing handler returns.
7. From UART0, repeat five HTTPS stop/start cycles with active admin and pending output; inject detach/queue/SSL-stop failures where feasible, retry stop and ensure no handle reuse until successful stop. Stress simultaneous peer disconnect and new serial admission during admin closure, specifically validating the shutdown/reuse fix. USB and UART0 must remain usable if HTTPS is unavailable.
8. Run at least a 15-minute full-client-mix/slow-reader soak, collect free/minimum/largest internal/DMA/PSRAM and available stack telemetry, disconnect all optional clients, wait 60 seconds and compare cleanup figures. Numeric reserve floors and real per-admin-socket cost still need approval/evidence. Stop before 8D.6; M2 also requires separately requested 8D.6/8D.7 work.

## Historical prerequisite record

The sections below record the earlier prerequisite-only checkpoint. Their no-backend statements and request to pause were superseded by the combined backend authorization/results above; their old build measurements are retained as provenance.

## Scope and provenance

Resumed at revision `e5dce12ed43154dacd086437de0f2d156014d58c` with existing uncommitted prerequisite changes in `src/admin_ssh_console.{c,h}`, `src/ssh_transport.c`, and `tests/admin_console_boundary/`. Preserved and reviewed that work, extended the harness to exercise production SSH snapshot/principal publication and wiping, ran both console suites and the firmware build, and recorded the handoff.

The plan's work-unit review splits the full ticket store, socket owner, HTTP policy/routes and integration tests from this runtime-changing prerequisite. No browser route, ticket store, new task, UI entry, broker client, generated asset, persistence change or device operation is included. No commit or branch change was made.

## Implemented contract

- The immutable console owner adapter now requires `is_current(token, principal)`. It runs on the dispatcher outside console locks, validates full transport identity and originating-session/principal binding, and must not call socket libraries or console handlers. Admission and transport input/output liveness remain owner responsibilities; admission need not already be published to this callback.
- Core checks account and owner currentness for queued work, again immediately before the canonical command runner, and after dispatch. Identity/owner are rechecked after external calls so late validation cannot close a replacement slot. UART0 remains independent.
- Visible and hidden prompts check currentness before publishing and after each wait. Waits poll at 250 ms plus validation/scheduling latency, not a hard real-time deadline; stale semaphore wakes cannot submit a still-waiting prompt. Revoked submitted replies are not returned to handlers. Close wipes prompt input immediately, including submitted input; executing session storage remains reserved until handler cleanup.
- SSH publishes two copied principals under the same lock as its snapshots. Its dispatcher adapter validates active authenticated admin route, transport/session/generation, close intent and full principal binding, without reading owner-task slots or calling wolfSSH. Consuming an external close preserves published close intent until cleanup.
- Reading console output wipes consumed ring segments, including wraparound, while retaining unread output.

These checks do not cancel or roll back arbitrary executing handlers, nor make authorization atomic with subsequent side effects. Database currentness can wait on its mutex. A future browser owner must still enforce cookie-session expiry/logout/revocation at admission, input, output and periodic cleanup; console polling is not a replacement for transport cleanup.

## Executed validation and resources

- `python3 tests/admin_console_boundary/run.py`: PASS. Production console plus extracted production SSH token/publication/adapter code; covers stale owner with current account, unrelated-session isolation, close/reuse during validation, invalidation immediately before execution, revoked/disconnected submitted prompts, unanswered prompt expiry without notification, stale wakes, UART0 recovery, output wiping, published principal cleanup, route/auth/principal/identity rejection and external-close handoff. Existing completion/history, prompts, deferred control and backpressure regressions also pass.
- `python3 tests/admin_ssh_policy/run.py`: PASS, including empty input, ordinary commands and quoted physical-only restrictions.
- Independent static review of the production prerequisite found no actionable defects. Final `git diff --check`: PASS.
- `pio run`: PASS in **38.46 s**, PlatformIO espressif32 6.12.0 / ESP-IDF 5.5.0, N16R8 release. **95,164 B linked RAM / 1,628,049 B flash**. Versus recorded 8D.4 (95,084 / 1,627,173): **+80 / +876 B**. Versus recorded 8D.0 (94,532 / 1,599,973): **+632 / +28,076 B**. Baselines were not rebuilt.
- Link map attributes 80 B (`0x50`) to `s_console_principals`. The owner callback adds code/read-only adapter storage, no per-session payload. Two shared console slots, two 4 KiB output rings, four-entry request queue, two-entry control queue, four-line history and task stacks (dispatcher 12 KiB, UART0 6 KiB, control 4 KiB) are unchanged. No new module heap/PSRAM allocation or increased socket/TLS/HTTP handler/session/ticket capacity; actual web-admin socket/slot cost is not yet available.

Host fakes are deterministic, not concurrent: locks are counters, waits use hooks, console execution/lifecycle operations are doubled. Production publication is now exercised, but the complete SSH owner loop, real FreeRTOS scheduling, task-local stdio, socket behavior and runtime memory/stack margins are not proven. No sanitizer or target pass is claimed.

## Target checkpoint and exact next increment

Before stacking the live backend on this runtime-changing prerequisite, obtain target regression or an explicit user decision to proceed under the plan:

1. Boot and capture UART0 status/`memory` and SSH stack telemetry. Confirm admin SSH empty Enter, commands, history/completion, visible/hidden/cancelled prompts and `exit`/Ctrl+D.
2. Disconnect or revoke an admin SSH account with work queued and while a prompt waits; confirm prompt cancellation, dispatcher/UART0 recovery, no reply/output crossover after reconnect, and isolation of unrelated sessions. Only use disposable test accounts and approved mutations; do not publish secrets.
3. Run browser login/serial disconnect/reconnect, native USB UART1 and user/admin SSH smoke at the supported 115200-baud workload. Repeat five serial lifecycle cycles per role and compare settled/full-client-mix/cleanup heap and SSH stack telemetry. Check slow readers do not compromise UART0 recovery.

Next implementation remains **8D.5**, not 8D.6: bounded admin-only digest tickets bound to current cookie session/principal; coordinated admission to the existing two shared console slots; a separate bounded admin transport using HTTPD-owned socket work; full pre-101 Origin/cookie/ticket admission; session/account/global invalidation and authoritative liveness checks; fail-before-side-effect restrictions for unsupported self-affecting commands; counters and focused authenticated integration checks. Review scope and socket/allocation budgets before coding and split further if needed. No normal UI entry or generic HTTP command runner. Admission must never evict a serial client/writer. Test real two-WebSocket coexistence before claiming M2 capacity or acceptance.