Document integrated web administration scope

README.md

@@ -14,13 +14,13 @@ ESP32-S3 firmware for a secure, multi-transport RS-232 adapter. It operates one
 
 ## Development status
 
-Hardware characterization, the serial core, USB CDC-ACM, Wi-Fi, HTTPS/WebSocket, SSH serial transport, and the local display/control interface are implemented and Phase 7 target-hardware validated. Phase 8A's bounded role-based user database and UART0 administration are complete. Phase 8B's role-aware HTTPS passwords, SSH passwords/public keys, and per-account session revocation are target-hardware validated. Phase 8C adds a bounded administrator SSH shell backed by the same serialized command registry as UART0 and awaits target-hardware validation. Configurable STA-only mDNS naming as `sak-<suffix>.local` is implemented with independent NVS persistence; target-hardware validation is pending. See the [Roadmap](docs/roadmap.md) for phase status and validation details.
+Hardware characterization, the serial core, USB CDC-ACM, Wi-Fi, HTTPS/WebSocket, SSH serial transport, and the local display/control interface are implemented and Phase 7 target-hardware validated. Phase 8A's bounded role-based user database and UART0 administration, Phase 8B's role-aware HTTPS/SSH authentication and revocation, and Phase 8C's shared UART0/admin-SSH command shell are target-hardware validated. Phase 8D plans integrated browser login/logout, an admin-shell terminal mode, typed settings, and contextual quick administration while preserving any browser-held serial writer lease across terminal-mode changes. Configurable STA-only mDNS naming as `sak-<suffix>.local` is implemented with independent NVS persistence; target-hardware validation is pending. See the [Roadmap](docs/roadmap.md) for phase status and validation details.
 
 ## Documentation
 
 - [Hardware wiring](docs/wiring.md): hardware profile, GPIO assignments, connector guidance, and safety notes.
 - [Electrical tests](docs/electrical_tests.md): OLED/buttons, MAX3243, UART loopback, and session-broker verification procedures.
-- [Role-based user database and UART0 administration](docs/user_administration_tests.md): user migration and administration, HTTPS/SSH authentication, and session-revocation validation.
+- [Role-based user database and UART0 administration](docs/user_administration_tests.md): user migration and administration, HTTPS/SSH authentication, session revocation, and the planned integrated web-administration acceptance matrix.
 - [Command reference](docs/command_reference.md): UART0/admin-SSH administration, serial, broker, USB, Wi-Fi, mDNS, web, SSH, and diagnostic commands.
 
 ## Flash partition layout

docs/agent/current-state.md

@@ -9,7 +9,7 @@ Based on checked-in source plus `README.md` and `docs/roadmap.md`:
 - Hardware characterization, serial service, session broker, USB CDC, Wi-Fi, HTTPS/WebSocket, SSH serial transport, and local display/control are implemented and documented as target-hardware validated.
 - Phase 8A role-based user storage/UART0 administration and Phase 8B role-aware HTTPS/SSH authentication and targeted revocation are documented as target-hardware validated.
 - Phase 8C admin SSH is implemented in source, uses the shared `esp_console` registry, and has passed target-hardware validation.
-- Phase 8D web user administration and Phase 8E browser login/session integration are planned, not implemented.
+- Phase 8D integrated web administration is planned, not implemented. Its ordered scope combines browser login/logout sessions, a shared admin-shell terminal route, typed settings, and contextual Serial/Wi-Fi/client/writer controls; changing terminal modes must preserve the browser serial broker client and any writer lease.
 - Security/production hardening, OTA, BLE evaluation, advanced networking, and optional filesystem features remain future roadmap work.
 - Reserved OTA, coredump, NVS-key, and storage partitions do not imply those runtime features are implemented.
 
@@ -28,7 +28,7 @@ Based on checked-in source plus `README.md` and `docs/roadmap.md`:
 
 - Phase 8C hardware validation passed, including route separation, shared command serialization, history/completion, prompts, output backpressure, revocation during queued work, deferred SSH lifecycle/reboot actions, and full concurrent transport operation. At 460800 baud with SSH and WebSocket clients in parallel, substantial packet drops and slow display controls were observed under load, without memory exhaustion; no baud-rate reduction is planned.
 - Current HTTPS has no web-based user administration and gives both roles the same status/terminal routes.
-- Browser authentication still uses HTTP Basic; integrated login/logout sessions are planned.
+- Browser authentication still uses HTTP Basic; Phase 8D plans integrated login/logout sessions before exposing administrative browser routes.
 - NVS encryption, secure boot/flash encryption review, authentication rate limiting, production certificate/provisioning policy, and OTA are not implemented.
 
 ## Known inconsistencies

docs/roadmap.md

@@ -38,7 +38,7 @@ These constraints apply across all phases:
 | 5B | Offline xterm.js WebSocket serial terminal | **Complete** |
 | 6 | Authenticated SSH serial transport | **Complete** |
 | 7 | Local display and button interface | **Complete** |
-| 8 | Role-based users and administrative access | **In progress (8A–8C complete; 8D–8E planned)** |
+| 8 | Role-based users and administrative access | **In progress (8A–8C complete; integrated web administration 8D planned)** |
 | 9 | Security and production hardening | **Planned** |
 | 10 | Authenticated, rollback-capable OTA | **Planned** |
 | 11 | BLE serial transport and provisioning evaluation | **Planned** |
@@ -171,7 +171,7 @@ Implemented and target-hardware validated:
 
 ## Current and planned phases
 
-Phase 8A through 8C are complete and target-hardware validated; 8D and 8E remain planned. Later work remains planned or under evaluation. Optional features must not weaken the completed serial and recovery paths.
+Phase 8A through 8C are complete and target-hardware validated; integrated web administration in 8D remains planned. Later work remains planned or under evaluation. Optional features must not weaken the completed serial and recovery paths.
 
 ### Phase 8 — Role-based users and administrative access
 
@@ -198,20 +198,18 @@ Implementation sequence:
    - `ssh sessions` and `ssh counters` identify broker versus admin-console routes, worker command state, queued admin output, admission failures, and input backpressure. `exit` and Ctrl+D on an empty command line request bounded deferred self-disconnect after best-effort application-buffer draining. Admin sessions are checked for a current `admin` principal before command execution and during the active-session reconciliation.
    - Keep SFTP, SCP, `exec`, forwarding, subsystems, and unauthenticated shells disabled.
    - Target-hardware validation passed for route separation, history/Tab editing, interactive visible/hidden prompts, output/backpressure, generated and entered user/password/key management including the longest ECDSA P-256 import, ping event routing, deferred reboot/SSH lifecycle drain behavior, bootstrap/recovery rejection, targeted self/other-user revocation during queued work, UART0/SSH administration serialization, and concurrent USB/WebSocket/user-SSH/admin-SSH operation. Stress at 460800 baud with SSH and WebSocket clients caused substantial expected packet drops and slower display controls, but did not exhaust memory or require lowering the supported baud-rate range.
-4. **Phase 8D — Web user administration — Planned**
+4. **Phase 8D — Integrated web administration — Planned**
-   - Add an admin-only user-management interface and typed, bounded APIs for account CRUD, roles, password generation/change, SSH-key management, and revocation. Never expose a generic HTTP endpoint that executes arbitrary CLI text.
+   - Begin with integrated authentication: replace browser-facing HTTP Basic authentication with a same-origin HTTPS login page, explicit logout, and bounded opaque server-side sessions. Store only a digest of each random session token with a copied secret-free principal, expiry, CSRF state, and authentication-generation binding. Send the raw token only in a host-only `__Host-` cookie with `Secure`, `HttpOnly`, `SameSite=Strict`, `Path=/`, no `Domain`, and an explicit lifetime; never retain passwords, Basic headers, raw tokens, verifiers, or SSH-key blobs in snapshots or logs.
-   - Hide administrative navigation and controls for normal users, and enforce every authorization decision server-side so hidden UI is not treated as a security boundary.
+   - Make logout invalidate the current server-side session, expire its cookie, close that session's serial and administrative WebSockets, and redirect to login. Password/role/key mutation, deletion, recreation, and explicit revocation invalidate the affected account's web sessions and tickets without disturbing unrelated accounts. Require the CSRF token plus strict same-origin checks for every state-changing request, including logout, and rate-limit login attempts with bounded secret-free accounting.
-   - Preserve the existing normal terminal interface. Use strict origin/CSRF protections for mutations, secret-safe JSON encoding, one-time display of generated passwords, and no-store responses for administrative material.
+   - Add an admin-only **Serial terminal**/**Admin shell** selector. The administrative route uses a short-lived, single-use, admin-principal-bound ticket and a bounded WebSocket frontend for the same serialized command registry used by UART0 and admin SSH; it is not a generic HTTP command-execution endpoint. Normal users retain the existing serial interface and cannot mint, upgrade, or invoke administrative routes. Remote policy still rejects physical-only `user bootstrap` and `user recover --force`.
-5. **Phase 8E — Integrated web authentication and sessions — Planned**
+   - Switching the visible terminal between Serial and Admin changes only the displayed terminal route. It must not disconnect the browser's serial broker client, release its writer lease, or silently stop serial observation. Keep the writer/observer badge plus Request control/Release control visible in both modes so an administrator knows the retained state and cannot unintentionally lose the lease to another client. Explicit Disconnect, logout, revocation, session expiry, or an explicit release/transfer operation still performs normal broker cleanup.
-   - Replace the browser-facing HTTP Basic authentication flow with a same-origin HTTPS login page and an explicit logout action. This avoids relying on browser credential caches, private windows, or a clean browser session to switch accounts after login.
+   - Add an admin-only Settings area backed by typed, bounded subsystem APIs rather than generated CLI strings. Cover user/password/role/SSH-key management, serial configuration and persistence, Wi-Fi profiles/AP policy/secrets, service and session controls, display settings, network diagnostics, and carefully separated security/danger-zone operations. The admin shell provides full remote operational parity and the settings pages provide guided high-frequency workflows; unusual electrical/debug operations may remain shell-only.
-   - Authenticate login submissions against the common user database over TLS, then create a bounded, opaque, random server-side session. Store only a digest of the session token together with a copied secret-free principal, expiry, and authentication-generation binding; never retain passwords, Basic headers, raw session tokens, verifiers, or SSH key blobs in snapshots or logs.
+   - Add contextual admin quick settings to the existing status cards. Hover, keyboard focus, or click on **Serial** opens a popover with current framing/lifecycle state, safe common edits, apply/save semantics, and a link to full Serial settings. The **Wi-Fi** card similarly exposes connection/profile controls and a link to full Network settings without revealing saved secrets by default. Touch and keyboard users must receive the same functionality as pointer hover, with Escape/outside-click dismissal and no action triggered merely by opening a popover.
-   - Send the opaque token only in a host-only `__Host-` cookie with `Secure`, `HttpOnly`, `SameSite=Strict`, `Path=/`, no `Domain` attribute, and an explicit lifetime. Reject missing, malformed, expired, replayed, or stale sessions. Avoid local-storage tokens and URL/query-string credentials.
+   - Hover, focus, or click on **Broker clients** opens a live, secret-free client list with IDs, transport, writer/observer state, and bounded queue/drop information. Activating **Active writer** opens an admin-only transfer dialog listing current eligible clients; transfer is an explicit confirmed generation-safe administrative assignment, never a side effect of opening or hovering. Stale/disconnected targets fail visibly without changing the current lease. Normal users may retain ordinary aggregate status but receive neither client-management details nor mutation controls.
-   - Make logout invalidate the current server-side session, expire its cookie, terminate its WebSocket terminal session, and redirect to the login page. Account password/role/key mutations, deletion, recreation, and explicit revocation must invalidate that account's web sessions and tickets; unrelated accounts remain connected.
+   - Preserve strict CSP, no-referrer/frame-denial policy, no-store responses for login/session/admin material, secret-safe JSON encoding, one-time generated-password display, and bounded request/response bodies. Keep UART0 recovery, native USB UART1 access, and existing SSH behavior available if web sessions or administration cannot initialize.
-   - Require a per-session CSRF token and same-origin checks for every state-changing web request, including logout and the future Phase 8D administration APIs. Continue `Cache-Control: no-store` for login, session, and administration responses; define CSP and redirect behavior so the login page cannot become an open redirect or token-reflection surface.
+   - Add secret-free counters/snapshots for session capacity, expiry, login failure/backoff, logout, invalidation, CSRF/origin rejection, admin-console admission/backpressure, typed-API failure, and writer-transfer races. Compatibility HTTP Basic, if retained temporarily during migration, must be explicit, separately constrained, and unable to bypass logout or revocation.
-   - Preserve a bounded physical-UART0 recovery path when the database or web-session service is unavailable. Do not retain a legacy network-authentication fallback after bootstrap. Decide and document whether compatibility HTTP Basic is removed entirely or is an explicitly temporary, separately rate-limited recovery mechanism; it must not silently bypass session logout or revocation.
-   - Add secret-free web-session counters and snapshots for capacity, expiry, login failures, logout, invalidation, and WebSocket association. Rate-limit login attempts and retain only bounded failure/accounting state.
 
-Completion requires migration and power-loss recovery tests, final-admin protection, malformed credential/key rejection, password and key login for both roles, targeted session revocation, concurrent normal/admin SSH operation, hidden and server-rejected user-level web administration, explicit web login/logout and stale-session handling, and continued UART0 recovery.
+Completion requires login/logout and account switching without browser credential-cache dependence; stale-session, CSRF, origin, capacity, expiry, and revocation tests; hidden and server-rejected normal-user administration; shared admin-console serialization and backpressure; terminal switching that demonstrably preserves the browser broker client and writer lease; accessible Serial/Wi-Fi/client/writer popovers; generation-safe explicit writer transfer; typed settings and secret-handling tests; concurrent USB/WebSocket/user-SSH/admin-SSH/browser-admin operation; and continued UART0 recovery.
 
 ### Phase 9 — Security and production hardening
 

docs/user_administration_tests.md

@@ -165,3 +165,31 @@ Exercise `ssh disconnect` for another session and the current session. For the o
 While an administrative command is queued or running, use UART0 to change that admin's role/password/key or delete it. Confirm no second remote administrative command runs after the mutation, the SSH session is revoked promptly, and queued output is not delivered to a reused SSH slot. Repeat with a different account mutation and confirm the administrator remains connected.
 
 Finally, issue commands concurrently from UART0 and admin SSH, including `user list`, long `help` output, and one UART0 interactive password or key prompt while an SSH command waits. Confirm the single dispatcher serializes all `esp_console_run()` calls, UART0 retains its line editing/history/completion, prompt input is consumed only from UART0, outputs are not mixed between transports, and there is no stack overflow, corrupted argument parsing, database damage, or broker disruption.
+
+## Planned Phase 8D integrated web administration
+
+These are acceptance requirements for the planned implementation, not tests that have passed yet.
+
+### 1. Integrated login and authorization
+
+Authenticate as both roles through the same-origin login page, explicitly log out, and switch accounts without clearing browser state or relying on an HTTP Basic credential cache. Verify bounded session capacity, expiry, login throttling, malformed cookies, CSRF rejection, Origin rejection, password/role/key invalidation, deletion/recreation, and unrelated-account isolation. A normal user must be unable to discover through navigation or invoke directly any admin ticket, admin WebSocket, typed mutation API, quick setting, detailed client list, or writer-transfer operation.
+
+### 2. Serial/admin terminal switching and lease preservation
+
+As an administrator, connect the browser serial terminal, acquire the writer lease, send and observe serial data, then switch repeatedly between **Serial terminal** and **Admin shell**. The visible terminal contents and input route must change, but `broker clients`, the displayed browser client ID, and active writer ID must remain unchanged. Request control/Release control and writer/observer state must stay visible in both modes. While Admin shell is selected, have a normal user request the writer lease and confirm the retained browser lease prevents unintended takeover. Only explicit Release control, confirmed writer transfer, Disconnect, logout, revocation, expiry, or connection failure may release it.
+
+Verify the browser admin shell executes the canonical registry through the single dispatcher, preserves bounded history/completion/prompts and backpressure, and does not itself become a second broker client. Physical-only bootstrap/recovery commands remain rejected. Closing only the admin-console route must leave the browser serial client and its lease intact.
+
+### 3. Quick settings and client popovers
+
+For an administrator, open the **Serial** and **Wi-Fi** card popovers by hover, keyboard focus, and click/tap. Confirm each path exposes the same controls, opening alone performs no mutation, Escape/outside click closes cleanly, current values refresh safely, common edits have clear apply-versus-save behavior, and links reach the full Settings section. Saved Wi-Fi secrets must not appear by default or enter routine DOM text, logs, URLs, or status responses.
+
+Open **Broker clients** and confirm its secret-free list matches authoritative broker snapshots: client ID, transport/name, writer/observer role, and bounded queue/drop state. Open **Active writer**, choose another currently connected eligible client, confirm the transfer explicitly, and verify exactly one writer remains. Disconnect or recycle the target before confirmation and confirm its stale generation/ID is rejected without releasing or reassigning the current writer. Hover/focus alone must never transfer ownership.
+
+### 4. Typed settings and destructive operations
+
+Exercise user/password/role/key management, serial settings and persistence, Wi-Fi profiles/AP policy/secrets, service/session controls, display settings, and network diagnostics through typed bounded APIs. Compare resulting subsystem state with the equivalent canonical CLI behavior without routing API requests through command strings. Generated passwords appear once in no-store responses; destructive or self-terminating HTTPS/SSH/reboot/security actions require explicit confirmation and explain the expected connection loss.
+
+### 5. Concurrency and failure isolation
+
+Run USB, browser serial, browser admin shell, user SSH, admin SSH, UART0, and active UART1 traffic concurrently. Alternate explicit writer transfers while issuing administrative commands and opening/closing popovers. Verify one writer, isolated observers, bounded memory/queues, principal revocation, no mixed admin output, no hidden lease loss during terminal switching, and continued UART0/native-USB recovery if web-session or admin-console initialization fails.