Phase 8D - massive refactor and admin functions / admin shell in

webinterface. Memory and cross-origin problems.

docs/agent/architecture.md

@@ -39,7 +39,7 @@ SSH role=admin ------> shared administration dispatcher <------ UART0
 10. Start the local status/control task if button initialization succeeded.
 11. Construct ESP-IDF's UART REPL to initialize `esp_console`, but do not start the stock REPL task. Register command groups, install completion, and start the custom UART frontend that feeds the shared dispatcher.
 
-Several core initializers use `ESP_ERROR_CHECK`; optional display and network/security paths generally log failure while retaining UART0 administrative recovery and network-independent UART1 access through USB. SSH starts before command registration, so role-`user` sessions can be admitted in that interval while role-`admin` sessions are rejected until the administration frontend is ready.
+Several core initializers use `ESP_ERROR_CHECK`; optional display and network/security paths generally log failure while retaining UART0 administrative recovery and network-independent UART1 access through USB. Network services start before command registration, so role-`user` serial access can be admitted in that interval while remote admin-console admission is rejected until the administration frontend is ready.
 
 ## Serial service and physical ownership
 
@@ -96,15 +96,19 @@ TinyUSB callbacks enqueue/copy data and state; the transport task owns broker li
 
 ### HTTPS, WebSocket, and web serial
 
-`web_server` runs HTTPS only on port 443 using the device-specific self-signed P-256 certificate from `web_security`. Current routes provide the UI, static assets, status, ticket issuance, and serial WebSocket upgrade.
+`web_server` runs HTTPS only on port 443 using the device-specific self-signed P-256 certificate from `web_security`. Current routes provide same-origin login/logout, the UI and static assets, status, typed admin operations, and separate serial/admin WebSocket ticket and upgrade paths.
 
-HTTP Basic authentication uses `user_database`. Before administrator bootstrap, the migrated role-`user` account is synchronized from the legacy credential, so that username/password can authenticate through the database; after bootstrap, the legacy blob is independent recovery material and is no longer consulted for authentication or synchronized into role-based accounts. Both `user` and `admin` roles currently receive the same web status/terminal experience; web administration is not implemented.
+Browser passwords authenticate through `user_database`; the legacy web credential remains migration/recovery material and is not an active browser login after bootstrap. Successful login creates one of eight fixed eight-hour RAM sessions, with at most two retained per account. The raw 192-bit token is sent only in the host-only `__Host-sak-session` cookie (`Secure`, `HttpOnly`, `SameSite=Strict`, `Path=/`) while storage retains its SHA-256 digest, copied principal, generation-safe slot identity, and monotonic expiry. A boot-local key derives a deterministic session-bound CSRF token. Logout and every state-changing endpoint require strict `Origin == https://<Host>` and CSRF validation; a four-entry source-address limiter provides bounded, deliberately modest login throttling.
 
-The boot-local Basic-authentication cache has four RAM entries and a five-minute sliding lifetime. It stores a keyed digest of the complete `Authorization` header rather than the raw header, and every hit revalidates principal currentness. Its current lack of locking relies on the single-HTTPD-owner execution model.
+The serial WebSocket requires a one-time exact-browser-session-bound ticket with a maximum 30-second lifetime. Tickets are stored as digests, consumed before currentness validation, and never persisted. Admission and active input revalidate both the copied principal and exact browser-session reference. An admitted serial connection starts the service if necessary, creates a broker client, and opportunistically requests writer ownership. The serial transport has two fixed slots and four ticket slots. Binary frames carry serial data; small text messages request or release writer ownership. HTTPD owns socket send/close operations, while the permanent transport task mediates broker work through bounded scheduling.
 
-A WebSocket connection requires a one-time, principal-bound ticket with a maximum 30-second lifetime. Only four tickets can be outstanding; minting another evicts the live entry with the earliest expiry. Ticket issuance and upgrade also validate a supplied `Origin` against `https://<Host>`; absence of `Origin` is accepted for non-browser clients. Tickets are stored as digests, consumed before currentness validation, and are never persisted. An admitted session starts the serial service if necessary, creates a broker client, and opportunistically requests writer ownership. The web transport has two fixed session slots. Binary frames carry serial data; small text messages request or release writer ownership. HTTPD owns socket send/close operations, while the web transport task mediates broker work through bounded scheduling. The browser's combined Connect/Disconnect control closes the WebSocket and pauses automatic reconnect; after a user-paused disconnect it changes to Connect, which resumes connection attempts.
+Administrators can separately mint a one-time ticket for one fixed browser admin-console WebSocket. That transport has its own permanent task, bounded input/output storage, copied admin principal, exact browser-session reference, and a single canonical-console slot; it never starts serial service or joins the broker. Terminal-mode switching changes only DOM visibility/focus and lazily opens the admin route, leaving the serial socket and writer lease alive. The combined Connect/Disconnect serial control explicitly manages only the serial route.
 
-Web serial initialization is failure-isolated from the base HTTPS service: if the transport cannot initialize, `web_server_init()` can still succeed and serve authenticated non-WebSocket routes.
+Typed admin endpoints cover full serial framing/lifecycle/persistence operations; Wi-Fi lifecycle/profile rotation; guided user CRUD, roles, entered/generated passwords, and authorized Ed25519/P-256 key add/remove; generation-safe station-profile/AP/secret editing and exact-generation save; display-aging apply/save/load/defaults/reset; a bounded secret-free broker-client list; and atomic expected-writer-to-target transfer. All authenticate the admin role server-side; mutators revalidate the exact browser session after their bounded body is parsed and immediately before the typed side effect. The common URL-form parser decodes in its body buffer, accepts at most 512 bytes and 10 unique fields, and rejects duplicate fields. User edits compare the database generation plus stable user ID; Wi-Fi edits compare the working-config generation, and reads disclose only `secret_set` flags. On stale conflicts the browser reloads user/Wi-Fi state without replaying the request and clears entered/generated secret fields on failure or Settings close.
+
+The Admin shell remains the web route for broader service/session controls, network diagnostics, security/danger operations, and unusual hardware/debug commands; those operations do not have guided forms.
+
+Serial and admin WebSocket initialization are failure-isolated from base HTTPS. An unavailable transport leaves login and non-WebSocket routes available. A lifecycle mutex serializes start, stop, and TLS refresh; a generation records explicit desired-running intent so a post-material refresh cannot override a newer request. Certificate rotation and full material reset require that refresh after persistence. HTTPS stop first disables further transport-owned HTTPD calls and tracks calls already in progress. Failed HTTPD destruction retains the handle for a retry, while a timed-out or failed admin detach is finalized only after successful HTTPD destruction and remains pending for retry before a later start.
 
 `web_ui.c` contains authored index/application strings and response policy. Its restrictive CSP contains a hard-coded hash of the inline loader, so those two must change atomically; preserve same-origin connections, no-referrer behavior, frame denial, and the existing cache policy. `web_assets_data.c` contains checked-in generated arrays for vendored compressed xterm assets and the logo. Normal builds compile these arrays directly; they do not regenerate assets.
 
@@ -129,37 +133,41 @@ Network code holds copied, secret-free principals rather than pointers into data
 
 Revocation has two layers:
 
-1. after a database mutation commits, the command layer makes best-effort targeted WebSocket/SSH revocation calls; notification failure does not roll back the mutation;
+1. `user_admin_service` serializes typed web and console mutations with `admin_command_gate`; after a database mutation commits, it makes best-effort targeted WebSocket/SSH revocation calls, and notification failure does not roll back the mutation;
 2. transports periodically and at sensitive boundaries recheck principal currentness, providing authoritative fail-safe closure if notification fails.
 
-The final administrator cannot be deleted or demoted. UART0 is trusted for initial administrator bootstrap and explicit unavailable-database recovery. Authenticated admin SSH can run the operational registry but is denied those two recovery operations; other secret-bearing commands are remotely available unless their handlers deny them.
+The web user editor supplies an expected database generation for every mutation and the stable target user ID for existing-account operations. A mismatch rejects stale state, including delete/recreate of the same username. The final administrator remains protected, and remote self-generated password replacement is rejected so its one-time result cannot be lost during revocation.
+
+The final administrator cannot be deleted or demoted. UART0 is trusted for initial administrator bootstrap and explicit unavailable-database recovery. Authenticated admin SSH and the browser Admin shell can run the operational registry but are denied those two recovery operations; other secret-bearing commands are remotely available unless their handlers deny them.
 
 NVS is not encrypted. Password verifiers improve password storage, but Wi-Fi credentials, legacy recovery credentials, and TLS/SSH private keys remain recoverable under physical flash extraction.
 
 ## Console architecture
 
-UART0 and admin SSH share canonical command implementations:
+UART0, admin SSH, and the browser admin shell share canonical command implementations:
 
 ```text
-UART0 linenoise frontend --\
-                          > fixed request queue -> one dispatcher -> esp_console_run()
-admin SSH line editor ----/                              |
-                                                        +-> registered *_console handlers
+UART0 linenoise frontend -----\
+admin SSH line editor ---------> fixed request queue -> one dispatcher -> esp_console_run()
+browser admin line editor -----/                              |
+                                                             +-> registered *_console handlers
 ```
 
-`admin_ssh_console` creates the dispatcher before network services but marks command dispatch ready only after ESP-IDF console registration and successful UART frontend task creation. An admin SSH connection during that boot window is rejected rather than racing an incomplete registry.
+`admin_ssh_console` creates the transport-neutral dispatcher before network services but marks command dispatch ready only after ESP-IDF console registration and successful UART frontend task creation. Remote admin admission during that boot window is rejected rather than racing an incomplete registry.
 
-The dispatcher is the sole caller of `esp_console_run()`, serializing UART0 and all admin SSH commands. This is required because the console registry is treated as non-reentrant, but it also means a long command or interactive prompt blocks all administration entry routes.
+The dispatcher is the sole caller of `esp_console_run()`, serializing UART0, admin SSH, and browser-admin commands. This is required because the console registry is treated as non-reentrant, but it also means a long command or interactive prompt blocks all administration entry routes.
 
-For SSH, standard output/error is redirected to the invoking session's bounded output ring. `console_input` routes visible or hidden prompts to UART0 or the active SSH session. `exit` and Ctrl+D on an empty admin SSH line use bounded deferred self-disconnect after their acknowledgement drains; role-`user` SSH remains a binary-transparent serial stream. Session tokens include slot and generation so late queued work cannot attach to a reused SSH slot. Only the SSH owner task moves ring output through wolfSSH.
+For remote frontends, standard output/error is redirected to the invoking session's bounded output ring. `console_input` routes visible or hidden prompts to UART0 or the active remote session. SSH and browser admin use the same bounded editor, four-entry history, completion formatter, overlong-line discard state, prompt handling, and generation-safe console tokens. Exact frontend callbacks revalidate the transport/session binding immediately before dispatch. Only the SSH owner task moves SSH ring output through wolfSSH; only the web-admin task queues browser output to HTTPD.
 
-Admin SSH `exit`, remote reboot, SSH stop/disconnect, and host-key rotate/reset use deferred control. The control task waits up to ten seconds for command state plus administration and transport application buffers to clear, then adds a short delay; this is a bounded best-effort heuristic, not peer-delivery confirmation. UART0 invokes these actions synchronously. User mutations and their revocations are not part of this mechanism. UART0 linenoise and the SSH editor consume the same manually maintained completion matcher and candidate formatter, so the two administration routes cannot drift in offered or displayed ambiguous completions; the hints can still drift from command registration and are not an authorization list.
+Remote `exit`, reboot, SSH stop/session disconnect/host-key changes, and web-origin HTTPS stop/restart use deferred control. The control task waits up to ten seconds for command state plus administration and transport application buffers to clear, then adds a short delay; this is a bounded best-effort heuristic, not peer-delivery confirmation. UART0 invokes these actions synchronously. User mutations and their revocations are not part of this mechanism. UART0 linenoise and the SSH editor consume the same manually maintained completion matcher and candidate formatter, so the two administration routes cannot drift in offered or displayed ambiguous completions; the hints can still drift from command registration and are not an authorization list.
 
 ## Wi-Fi and persistence
 
 `wifi_config` owns a fixed-width versioned NVS schema with four prioritized station profiles and AP policy `off`, `fallback`, or `always`. Missing configuration generates per-device defaults including a random AP password. Invalid stored data is generally left untouched while RAM defaults are used.
 
-`wifi_manager` is a permanent task with one bounded command/event queue. ESP-IDF callbacks only copy compact events into the queue. The task owns association, DHCP deadlines, profile failover, AP policy, retries/backoff, next-profile requests, and the mDNS announcement lifecycle. `mdns_service` initializes the responder at most once after a validated STA `GOT_IP`; the managed component's own event handlers withdraw and restore the STA announcement across transient connectivity changes, while the project tracks whether announcement is currently expected. Initialization failure is latched rather than retried because partial upstream low-memory initialization is not safely recoverable; mDNS failure is nonfatal. It also reconciles against authoritative driver/netif state so dropped events do not permanently wedge policy. ESP-IDF Wi-Fi storage is RAM-only; the application blob is authoritative, and edits require explicit save. Edits to disabled station profiles are staged in RAM without restarting the radio; enabling/disabling a profile or changing enabled station/AP policy restarts it asynchronously. Start/stop—including local controls—intentionally update the RAM `enabled_at_boot` field. Working-configuration copies contain PSKs and must be securely wiped; routine status and the local UI use secret-free snapshots.
+`wifi_manager` is a permanent task with one bounded command/event queue. ESP-IDF callbacks only copy compact events into the queue. The task owns association, DHCP deadlines, profile failover, AP policy, retries/backoff, next-profile requests, and the mDNS announcement lifecycle. `mdns_service` initializes the responder at most once after a validated STA `GOT_IP`; the managed component's own event handlers withdraw and restore the STA announcement across transient connectivity changes, while the project tracks whether announcement is currently expected. Initialization failure is latched rather than retried because partial upstream low-memory initialization is not safely recoverable; mDNS failure is nonfatal. It also reconciles against authoritative driver/netif state so dropped events do not permanently wedge policy. ESP-IDF Wi-Fi storage is RAM-only; the application blob is authoritative, and edits require explicit save. Edits to disabled station profiles are staged in RAM without restarting the radio; enabling/disabling a profile or changing enabled station/AP policy restarts it asynchronously. Start/stop—including local controls—intentionally update the RAM `enabled_at_boot` field.
+
+A dedicated config-writer mutex serializes complete working-config writers. The typed browser editor copies the credential-bearing config with its exact nonzero generation, modifies and validates the copy, and commits only through compare-and-swap; Save holds the same writer serialization through NVS and persists only the expected generation. Generation mismatch or exhaustion fails closed. Credential-bearing copies are securely wiped, browser reads return only per-profile/AP `secret_set` booleans, and routine status/local UI use secret-free snapshots.
 
 Persistent namespaces/blobs include:
 
@@ -179,7 +187,7 @@ Configuration modules generally choose RAM defaults without erasing incompatible
 
 When button GPIO initialization succeeds, `local_status_ui` starts a firmware-lifetime low-priority task that polls/debounces buttons, renders copied public snapshots, implements aging/wake behavior, and invokes a constrained set of public service APIs for local controls. It collects snapshots before opening a display frame, so service/broker locks are not held across I2C. It never parses CLI output, becomes a broker client, edits credentials, or assigns a writer; emergency action can only release the expected current writer.
 
-The task can run with an absent OLED, and a fresh button press can request one bounded panel reprobe after successful I2C bus setup. Failed I2C bus creation is not recoverable through that path. The `display` configuration commands depend on the UI task. Long confirmation holds protect disruptive local actions, and stuck buttons are quarantined.
+The task can run with an absent OLED, and a fresh button press can request one bounded panel reprobe after successful I2C bus setup. Failed I2C bus creation is not recoverable through that path. The `display` configuration commands and typed `/api/admin/display` aging operations depend on the UI task; web Apply/Save/Load/Defaults/Reset reuse the local-UI validation and persistence contracts and share `admin_command_gate` with console display writers so each complete RAM/NVS operation is serialized. Long confirmation holds protect disruptive local actions, and stuck buttons are quarantined.
 
 Hardware diagnostics are synchronous console commands. RS-232 tests own the physical port exclusively and restore safe GPIO state; OLED tests reuse the display service rather than taking independent I2C ownership.
 
@@ -187,9 +195,9 @@ Hardware diagnostics are synchronous console commands. RS-232 tests own the phys
 
 - Broker, USB, web-transport, Wi-Fi, and SSH owner tasks are firmware-lifetime tasks; the local-UI task is also firmware-lifetime when button initialization allowed it to start. Stopping a service generally stops its runtime/listener, not the owner task.
 - Bounded queues, stream buffers, work bursts, and drop counters are part of slow-client and watchdog isolation.
-- Transport slot generations and account authentication generations solve different stale-reference problems; preserve both.
+- Transport slot generations, account authentication generations/stable IDs, Wi-Fi working-config generations, and HTTPS lifecycle generations solve different stale-reference problems; preserve each domain's checks.
 - Library/hardware ownership is centralized: serial task owns UART1 while running, display service owns I2C/framebuffer, the SSH owner task owns post-initialization wolfSSH runtime calls, and the console dispatcher owns `esp_console_run()`.
-- Password authentication performs PBKDF2 outside the user-database mutex and revalidates afterward. Some password mutation paths currently derive verifiers while holding the mutation lock; do not generalize the authentication locking pattern without checking the exact path.
+- Password authentication performs PBKDF2 outside the user-database mutex and revalidates afterward. Typed user mutations are serialized by `user_admin_service` plus `admin_command_gate`; typed display Apply/Save/Load/Defaults/Reset also use that gate with console display writers. Some password mutation paths derive verifiers while holding the mutation lock, so do not generalize the authentication locking pattern without checking the exact path.
 - Avoid holding service/database/broker locks across I2C, network sends, or other potentially long operations unless the existing contract explicitly requires it. Preserve the existing broker-before-serial lock order.
 - Serial RX/TX stream payloads, broker per-client payloads, the transactional user-database candidate, and selected cryptographic allocations prefer PSRAM with internal fallback. The live user database, FreeRTOS control structures, UART driver buffers, and task stacks remain internal where deterministic/cache-disable access matters.
 - The build disables wolfSSL ESP32 AES/SHA acceleration, and the HTTPS path uses software AES for PSRAM-backed records. This preserves the validated workaround for uncoordinated mbedTLS/wolfSSL hardware-crypto locks and a prior mbedTLS external-RAM DMA watchdog stall.