Replace Web Basic Auth With Cookie Sessions

Add bounded login challenges, CSRF/origin enforcement, logout, and
session-bound WebSocket admission. Isolate private HTTPD access behind a
version-guarded adapter and add focused host coverage. Also let empty
admin
SSH input reach the normal console handler.
This commit is contained in:
2026-09-05 23:55:05 +02:00
parent 4435a7fddd
commit 5a609fa40b
36 changed files with 1940 additions and 360 deletions
+8 -6
View File
@@ -98,19 +98,21 @@ TinyUSB callbacks enqueue/copy data and state; the transport task owns broker li
`web_server` runs HTTPS only on port 443 using the device-specific self-signed P-256 certificate from `web_security`. Current routes provide the UI, static assets, status, ticket issuance, and serial WebSocket upgrade.
HTTP Basic authentication uses `user_database`. Before administrator bootstrap, the migrated role-`user` account is synchronized from the legacy credential, so that username/password can authenticate through the database; after bootstrap, the legacy blob is independent recovery material and is no longer consulted for authentication or synchronized into role-based accounts. Both `user` and `admin` roles currently receive the same web status/terminal experience; web administration is not implemented.
HTTPS login uses `user_database` and opaque server-side cookie sessions; Basic authentication and its cache are removed in 8D.3. Before administrator bootstrap, the migrated role-`user` account is synchronized from the legacy credential, so that username/password can authenticate through the database; after bootstrap, the legacy blob is independent recovery material and is no longer consulted for authentication or synchronized into role-based accounts. Both `user` and `admin` roles currently receive the same web status/terminal experience; web administration is not implemented.
The boot-local Basic-authentication cache has four RAM entries and a five-minute sliding lifetime. It stores a keyed digest of the complete `Authorization` header rather than the raw header, and every hit revalidates principal currentness. Its current lack of locking relies on the single-HTTPD-owner execution model.
`web_cookie_auth` owns login/session/logout policy: four 120-second digest-only pre-login challenges, explicit same-origin bootstrap, five credential verifications per 60-second global window, and no live-record eviction. Host-only `__Host-` Secure/HttpOnly/SameSite=Strict cookies have absolute lifetimes. Login consumes a challenge, validates bounded JSON and issues a fresh session; logout invalidates only its originating session. Mutations require CSRF and strict canonical HTTPS Origin; serial upgrade requires matching cookie/Origin/ticket. Neither role has web administration yet.
Phase 8D.1 adds `web_session_store` primitives alongside Basic auth: four static records with token/origin digests, copied principal, separate CSRF state, one-hour absolute expiry and non-reused 64-bit session IDs. No HTTP handler issues cookie sessions yet. A portMUX protects short state copies/mutations; database/RNG/SHA calls occur outside it. Resolution rechecks ID/expiry after database validation; issuance also checks an invalidation epoch. Stop wipes records without resetting IDs/epochs. Only admitted HTTPS starts initialize the store; failed starts and accepted stops disable it before cleanup. Store-init failure cannot fail existing Basic HTTPS. Sensitive views must be wiped by callers; snapshots contain only counts and storage sizes. Focused host checks live in `tests/web_session_store/`.
`web_session_store` holds four static records with token/origin digests, copied principal, separate CSRF state, one-hour absolute expiry and non-reused 64-bit session IDs. These are live cookie sessions in 8D.3, with no sliding renewal. A portMUX protects short state copies/mutations; database/RNG/SHA calls occur outside it. Resolution rechecks ID/expiry after database validation; issuance also checks an invalidation epoch. Stop wipes records without resetting IDs/epochs. Only admitted HTTPS starts initialize the store; failed starts and accepted stops disable it before cleanup. Authentication/store-init failure now gates HTTPS startup rather than falling back to Basic. Sensitive views must be wiped by callers; snapshots contain only counts and storage sizes. Focused host checks live in `tests/web_session_store/`.
Phase 8D.2 binds serial tickets/slots to distinct originating web-session IDs; zero is reserved for the shipped Basic path. Trusted internal mint/upgrade callers supply the ID; bound checks also compare the session's copied principal, with no CSRF export. Mint/consume/admission/input and existing 250 ms owner checks validate session liveness/currentness. Session-specific transport revocation invalidates the store first, then clears matching tickets and flags matching reserved/active slots for existing HTTPD/broker cleanup. Account/global transport revocation now invalidates cookie records even if serial initialization failed; existing console mutation callers reach these hooks unchanged. A non-wrapping transport epoch cancels in-flight ticket publication across revocation and server detach/re-attach. Store/database checks remain authoritative if notification is missed. Bound browser paths remain dormant until 8D.3's atomic cookie/CSRF/Origin cutover; no public cookie route, capacity change or new task exists in 8D.2.
Serial tickets/slots bind to distinct originating web-session IDs; 8D.3 rejects zero instead of treating it as Basic. Trusted internal mint/upgrade callers supply the ID; bound checks also compare the session's copied principal, with no CSRF export. Mint/consume/admission/input and existing 250 ms owner checks validate session liveness/currentness. Session-specific transport revocation invalidates the store first, then clears matching tickets and flags matching reserved/active slots for existing HTTPD/broker cleanup. Account/global transport revocation now invalidates cookie records even if serial initialization failed; existing console mutation callers reach these hooks unchanged. A non-wrapping transport epoch cancels in-flight ticket publication across revocation and server detach/re-attach. Store/database checks remain authoritative if notification is missed. 8D.3 activates these checks for all browser routes, with five added authentication handlers (14 total), unchanged six HTTPS sockets and no new task.
A WebSocket connection requires a one-time, principal-bound ticket with a maximum 30-second lifetime. Only four tickets can be outstanding; minting another evicts the live entry with the earliest expiry. Ticket issuance and upgrade also validate a supplied `Origin` against `https://<Host>`; absence of `Origin` is accepted for non-browser clients. Tickets are stored as digests, consumed before currentness validation, and are never persisted. An admitted session starts the serial service if necessary, creates a broker client, and opportunistically requests writer ownership. The web transport has two fixed session slots. Binary frames carry serial data; small text messages request or release writer ownership. HTTPD owns socket send/close operations, while the web transport task mediates broker work through bounded scheduling. The browser's combined Connect/Disconnect control closes the WebSocket and pauses automatic reconnect; after a user-paused disconnect it changes to Connect, which resumes connection attempts.
A WebSocket connection requires a one-time, principal-bound ticket with a maximum 30-second lifetime. Only four tickets can be outstanding; expired/stale identities are reclaimed and live capacity is rejected with 503/Retry-After, not eviction. Ticket issuance and upgrade require `Origin` matching validated Host after host-case/default-port normalization; missing Origin fails even for non-browser clients. Tickets are stored as digests, consumed before currentness validation, and are never persisted. An admitted session starts the serial service if necessary, creates a broker client, and opportunistically requests writer ownership. The web transport has two fixed session slots. Binary frames carry serial data; small text messages request or release writer ownership. HTTPD owns socket send/close operations, while the web transport task mediates broker work through bounded scheduling. The browser's combined Connect/Disconnect control closes the WebSocket and pauses automatic reconnect; after a user-paused disconnect it changes to Connect, which resumes connection attempts.
`web_httpd_adapter` is the sole private ESP-IDF 5.5.0 boundary. Its compile-time version guard requires review on upgrades. It validates NUL-separated parsed headers because public getters return only the first field, and rejects duplicates/ambiguous framing. The serial URI is registered as ordinary HTTP GET so cookie/ticket/principal/broker admission precedes explicit 101 and frame-handler installation; automatic IDF WebSocket routing would send 101 too early. Cleanup wipes consumed scratch but preserves right-aligned unread pending data. CMake compiles HTTPD logs above ERROR out to prevent header/ticket logging. No SDK patch or component copy exists. See `docs/phase8d3_implementation.md` for source verification, tests and pending on-wire checks.
Web serial initialization is failure-isolated from the base HTTPS service: if the transport cannot initialize, `web_server_init()` can still succeed and serve authenticated non-WebSocket routes.
`web_ui.c` contains authored index/application strings and response policy. Its restrictive CSP contains a hard-coded hash of the inline loader, so those two must change atomically; preserve same-origin connections, no-referrer behavior, frame denial, and the existing cache policy. `web_assets_data.c` contains checked-in generated arrays for vendored compressed xterm assets and the logo. Normal builds compile these arrays directly; they do not regenerate assets.
`web_ui.c` contains authored index/application strings and response policy; it validates `/api/session` before connect/restore, adds explicit Sign out, and cancels stale work on 401/logout/page exit. `web_login_ui` is a standalone public page without protected-asset dependencies. Both authentication documents and app script are no-store. Its restrictive CSP contains a hard-coded hash of the inline loader, so those two must change atomically; preserve same-origin connections, no-referrer behavior, frame denial, and the existing cache policy. `web_assets_data.c` contains checked-in generated arrays for vendored compressed xterm assets and the logo. Normal builds compile these arrays directly; they do not regenerate assets.
### SSH
+6 -6
View File
@@ -61,17 +61,17 @@ This is a semantic map, not a complete file inventory. Start here, then read the
**Responsibility:** serve authenticated HTTPS UI/API, issue WebSocket tickets, and adapt browser serial sessions to broker clients.
- Files: `src/web_server.{h,c}`, `src/web_serial_transport.{h,c}`, `src/web_ui.{h,c}`, `src/web_console.{h,c}`
- Security files: `src/web_security.{h,c}`; `src/web_session_store.{h,c}` contains dormant Phase 8D.1 cookie-session primitives, not active HTTP authentication.
- Security files: `src/web_security.{h,c}`, `src/web_cookie_auth.{h,c}`, `src/web_session_store.{h,c}`, `src/web_auth_parse.{h,c}`. Private IDF boundary: `src/web_httpd_adapter.{h,c}`.
- Asset files: authored/generated boundary in `src/web_assets_data.{h,c}`, `web_assets/SOURCES.md`, `web_assets/generate_embedded_assets.py`
- Interfaces: web init/start/stop/snapshots; HTTP handlers; ticket mint/consume; attach/detach; targeted session revocation
- Called by: startup, ESP-IDF HTTPS server, user administration revocation, console/local UI
- Dependencies: user database, secure random, broker, successful Wi-Fi manager initialization at boot, mbedTLS/HTTPS server; actual network reachability is an operational prerequisite, not an initializer invariant
- Flow: `browser -> HTTPS Basic auth -> ticket -> WebSocket -> web transport -> broker`
- Flow: `browser -> HTTPS login/cookie session -> CSRF-protected ticket -> cookie/Origin/ticket admission -> WebSocket -> web transport -> broker`
- Ownership: HTTPD owns socket send/close work; transport task owns broker mediation; two fixed WebSocket slots and four outstanding tickets.
- Security constraints: Basic-auth cache hits still revalidate principal currentness; the browser's combined Connect/Disconnect control closes the WebSocket and pauses automatic reconnect until Connect is selected. Changes to the authored inline loader must update its hard-coded CSP hash in the same change.
- Session-store boundary: admitted HTTPS start initializes four static records; failed start/accepted stop disables and wipes them. 8D.2 binds tickets/slots to non-reused session IDs (zero only for Basic); transport-specific cleanup and account/global revocation invalidate store records before socket cleanup. No cookie route yet. RNG/SHA/database calls run outside short portMUX sections; ID/expiry/epoch checks reject stale work. Run `python3 tests/web_session_store/run.py` and its `--serial` integration mode.
- 8D.3 preparation: `src/web_auth_parse.{c,h}` provides inert, allocation-free origin/cookie/login-JSON parsing; no HTTP caller yet. Test with `python3 tests/web_auth_parse/run.py`. These helpers do not authenticate or replace HTTP header/method/CSRF policy.
- 8D.3 login rendering: `src/web_login_ui.{c,h}` contains a standalone no-store login document and hash-bound script; no registered route or live caller. `python3 tests/web_login_ui/run.py` checks production C rendering, CSP and Node DOM/fetch doubles. Existing `web_ui.c` and generated assets are unchanged.
- Security constraints: Basic/cache removed; four absolute one-hour cookie sessions revalidate principal currentness. Four pre-login challenges (120 s), five credential attempts/60 s globally, no live session/challenge/ticket eviction. Origin/CSRF required for mutations; Origin/cookie/ticket before upgrade. Disconnect pauses reconnect but retains login; Sign out invalidates its session. Authored loader changes must update their hard-coded CSP hashes atomically.
- Session-store boundary: admitted HTTPS start initializes records; auth-init failure gates HTTPS. Failed start/accepted stop disables and wipes state. Tickets/slots require nonzero non-reused session IDs; session/account/global revocation invalidates store records before socket cleanup. RNG/SHA/database calls run outside short portMUX sections; ID/expiry/epoch checks reject stale work. Run `python3 tests/web_session_store/run.py` and its `--serial` integration mode.
- 8D.3 HTTP policy: `web_cookie_auth` owns public login/challenge/login POST/session/logout routes and protected-route checks; `web_auth_parse` handles bounded values/JSON. `web_httpd_adapter` alone reads private IDF 5.5.0 header scratch, rejects duplicate fields, defers 101 until transport admission and wipes consumed scratch while preserving right-aligned pending bytes. No SDK patch. `src/CMakeLists.txt` supplies private includes and compiles HTTPD warning/debug logs out. Test with `python3 tests/web_cookie_auth/run.py` and `python3 tests/web_auth_parse/run.py`.
- 8D.3 UI: `src/web_login_ui.{c,h}` serves standalone `/login`; `web_ui.c` validates session before serial connect/restore and handles logout/401 safely. Both scripts hash-bound, auth documents/app no-store. Tests: `python3 tests/web_login_ui/run.py` and `python3 tests/web_ui_session/run.py`. Live cutover host-tested/build-verified, M1 target gate pending: `docs/phase8d3_implementation.md`.
- Asset constraint: `web_assets_data.c` is checked-in generated input to the build; do not hand-edit or regenerate casually.
## SSH
+22 -5
View File
@@ -4,6 +4,15 @@ This file is working memory. Update it during active work and before handoff; do
## Development state
- **8D.3 both-role target login confirmed / mixed-client evidence (2026-09-05):** User reports successful HTTPS user+admin login after Origin fix; previous login blocker is resolved. Settled internal/DMA/PSRAM free **71,204 / 63,448 / 8,247,744 B**. Mixed load free **33,868 / 26,112 / 8,089,060 B**, minima **13,756 / 6,000 / 8,072,744 B**, largest **25,600 / 25,600 / 7,995,392 B**; SSH stack minimum-free **16,288 B**. 115200 baud, four broker clients (SSH sole writer, USB + two web observers), user/admin SSH active. No reported web transport or SSH I/O errors; 6 login attempts/3 invalid credentials/1 logout, zero security rejections. Two identical loaded heap samples are not a soak/leak or reserve proof. Full details/provenance in `docs/phase8d3_implementation.md`. **M1 validation in progress, not signed off.**
- **Admin SSH empty Enter fix (2026-09-05):** User's empty line was classified as UART0-restricted because `remote_command_allowed` required argc>0. Changed only helper classification to allow empty input through normal quiet IDF handling; currentness and physical-only commands remain protected. New `python3 tests/admin_ssh_policy/run.py` passes 15 cases using production helper/installed parser. Build passes **21.04 s**, **95,508 B RAM / 1,625,725 B flash** (+20 B flash). Not uploaded or target-tested. Ask for empty Enter/normal-command smoke on next flash; no 8D.4 refactor.
- **8D.3 Origin-null fix (2026-09-05), target retest pending:** User confirmed challenge200/login403 with `Origin: null`, same-origin Fetch Metadata and pre-login cookie; post-attempt counters show 0 password attempts and 7 security rejections. Root cause is non-CORS fetch POST under no-referrer. Login fetch now uses `mode:'cors'`; app helper uses cors for POST tickets/logout, unchanged GET mode. Same-origin credentials/fixed paths/redirect denial/CSP/no-referrer and strict server Origin/CSRF remain intact. Login CSP hash updated atomically. Both UI suites and cookie-policy suite pass; build **95,508 B RAM / 1,625,705 B flash**, **14.30 s** (+16 B flash). No upload; actual Firefox Origin header/login/serial/logout retest and M1 acceptance still pending. See implementation record; supersedes the speculative diagnosis below.
- **8D.3 target login blocker (2026-09-05):** User supplied clean-boot/60-second settled telemetry, then reports both user/admin login rejected with the page's HTTP-403 message. Record in `docs/phase8d3_implementation.md`: internal free/min/largest 69,004/66,752/31,744 B; DMA 61,248/58,996/31,744 B; PSRAM 8,223,116/8,218,204/8,126,464 B; SSH stack minimum-free 18,464 B; no active sessions/broker clients, UART stopped. Boot counters precede login attempts; no post-attempt result yet. Exact flashed revision not supplied. Production login-renderer/CSP test passes. Browser warnings name other script hashes (possibly injected scripts), plus denied favicon/file URL; do not relax CSP based on these alone. Need failed endpoint/status/error code and nonsecret Origin/Sec-Fetch-Site. Investigation hypothesis: same-origin fetch mode plus no-referrer policy may serialize POST Origin as null; confirm wire headers before changing request policy. **M1 blocked, not signed off.** No corrective firmware change for this report yet.
- **8D.3 live cutover implemented / host-tested / build-verified (2026-09-05), M1 target pending:** Resumed another agent's uncommitted completed server/browser implementation; preserved it, verified installed IDF header/upgrade semantics, and fixed right-aligned pending-buffer cleanup with an actual-IDF-reader regression. Cookie login/logout is live, Basic/cache removed. Final build **95,508 B RAM / 1,625,689 B flash** (+248 / +23,764 B versus 8D.2). See `docs/phase8d3_implementation.md`. This supersedes older inert/planned statements below. **Stop for M1 target/browser sign-off before 8D.4; numeric reserve gates remain open.**
- **8D.3 inert login renderer completed (2026-09-05):** User requested continuation after the parser split. Added standalone `web_login_ui.{c,h}` plus production-renderer/Node tests; no live route or Basic-auth change. Build and focused suites pass; 8D.3/M1 remains incomplete. Remaining work is the atomic server/app cutover, then mandatory browser/target validation. See active task below.
- **8D.3 preparatory parser split (2026-09-05):** User requested continuation. Per the plan's 600800-line scope review, selected inert private request parsing before the larger atomic login/logout cutover. `src/web_auth_parse.{c,h}` and focused host tests added; no live HTTP callers or authentication changes. See active task below. 8D.3/M1 is **not complete**; Basic remains active. Prior 8D.2 user sign-off stands; numeric reserves remain open.
@@ -25,7 +34,7 @@ Based on checked-in source plus `README.md` and `docs/roadmap.md`:
- Hardware characterization, serial service, session broker, USB CDC, Wi-Fi, HTTPS/WebSocket, SSH serial transport, and local display/control are implemented and documented as target-hardware validated.
- Phase 8A role-based user storage/UART0 administration and Phase 8B role-aware HTTPS/SSH authentication and targeted revocation are documented as target-hardware validated.
- Phase 8C admin SSH is implemented in source, uses the shared `esp_console` registry, and has passed target-hardware validation.
- Phase 8D.1 is validated by user sign-off; 8D.2 serial/session binding is implemented, host-tested and build-verified with target regression pending. Browser login/logout and integrated web administration remain planned. Follow `docs/phase8d_plan.md`: one numbered chunk per request, target-validated login/logout (M1) before the browser admin shell (M2), then one typed-settings/control domain at a time (M3). Changing terminal modes must preserve the browser serial broker client and any writer lease. The roadmap retains the full end-state requirements.
- Phase 8D.08D.2 are validated by user sign-off; 8D.3 browser login/logout is implemented, host-tested and build-verified, with mandatory M1 target/browser validation pending. Browser administration remains planned. Follow `docs/phase8d_plan.md`: one numbered chunk per request, target-validated login/logout (M1) before the browser admin shell (M2), then one typed-settings/control domain at a time (M3). Changing terminal modes must preserve the browser serial broker client and any writer lease. The roadmap retains the full end-state requirements.
- Security/production hardening, OTA, BLE evaluation, advanced networking, and optional filesystem features remain future roadmap work.
- Reserved OTA, coredump, NVS-key, and storage partitions do not imply those runtime features are implemented.
@@ -44,8 +53,8 @@ Based on checked-in source plus `README.md` and `docs/roadmap.md`:
- Phase 8C hardware validation passed, including route separation, shared command serialization, history/completion, prompts, output backpressure, revocation during queued work, deferred SSH lifecycle/reboot actions, and full concurrent transport operation. At 460800 baud with SSH and WebSocket clients in parallel, substantial packet drops and slow display controls were observed under load, without memory exhaustion; no baud-rate reduction is planned.
- Current HTTPS has no web-based user administration and gives both roles the same status/terminal routes.
- Browser authentication still uses HTTP Basic; Phase 8D plans integrated login/logout sessions before exposing administrative browser routes.
- NVS encryption, secure boot/flash encryption review, authentication rate limiting, production certificate/provisioning policy, and OTA are not implemented.
- Browser authentication now uses cookie login/logout without Basic fallback. M1 target/browser validation gates any browser administrative routes.
- NVS encryption, secure boot/flash encryption review, production certificate/provisioning policy, and OTA are not implemented. HTTPS login has a bounded global five-verifications/60-second throttle, not comprehensive cross-transport DoS protection.
## Known inconsistencies
@@ -58,9 +67,17 @@ These observations should be checked when touching the relevant area; they are n
## Items to verify in future work
- Confirm task-local Newlib standard-stream behavior if ESP-IDF/Newlib configuration changes; admin SSH command output relies on dispatcher-task stream redirection.
- If HTTPD concurrency configuration changes, add locking around the boot-local Basic-authentication cache.
- Re-audit the private HTTPD adapter on SDK changes (including same-version patches): parsed-header layout, right-aligned pending data, explicit handshake/frame installation and log suppression. Host tests do not establish real socket behavior.
## Active Task - Phase 8D.3 Inert Login Rendering
## Active Task - Phase 8D.3 Live Authentication Cutover
- **Inherited implementation:** `web_cookie_auth.{c,h}`, `web_httpd_adapter.{c,h}`, parser optional-cookie validation, server/console/transport integration, browser app/session recovery, and focused suites. No Basic authorization/cache remains. Four one-hour sessions, four 120-second pre-login challenges, five password checks per 60-second window, four non-evicting 30-second tickets/two serial sockets. Fourteen URI slots, unchanged six HTTPS sockets and application task/stack/queue capacities. No generated assets or SDK files changed.
- **HTTPD decision:** Other agent chose an isolated **private-IDF adapter**, not the previously proposed SDK patch. Verified first-only header getters, append-only pointer-backed Set-Cookie (six-header successful login), auto-101-before-handler flow and private frame installation against installed 5.5.0. Serial URI uses ordinary GET until authenticated ticket/currentness/broker admission, then explicit handshake. Exact version guard requires re-audit on update; not a source-hash guarantee. HTTPD logs above ERROR compiled out to avoid secrets/ticket queries. Durable boundary recorded in architecture/design decisions and implementation record.
- **Fix in this continuation:** Pending HTTPD bytes are right-aligned. Inherited wipe preserved the wrong end, risking pipelined HTTP/early-frame corruption. Fixed consumed-prefix wipe and tested 0128 pending lengths plus partial reads using extracted installed `httpd_recv_pending`. Kept all other inherited source work intact.
- **Validation:** `python3 tests/web_cookie_auth/run.py`, parser 268-case suite, login UI eight Node groups, serial app nine Node groups, and store `--serial` integration mode pass. Cookie suite also executes store tests and extracts installed header getter/setter/pending-reader functions; handshake/network/tasks remain doubled. Final `pio run` passed **17.62 s**, **95,508 B RAM / 1,625,689 B flash**; +248/+23,764 B versus 8D.2, +976/+25,716 B versus recorded 8D.0. Auth symbols 637 B before placement padding (including 576 B challenges); Basic cache/key removal offsets much of it. No runtime reserve or stack margin inferred.
- **Handoff:** `docs/phase8d3_implementation.md` contains route policy, source verification, exact test commands/limits, accounting and M1 checklist. No upload/erase/commit/branch operation or real target/browser execution. Obtain M1 sign-off before 8D.4. Prior 8D.08D.2 sign-offs stand. Numeric heap/largest-block/owner-stack reserve floors, real cookie/CSP/bfcache, pre-101 rejection, frame/pipelined data, five-cycle lifecycle checks, isolation/revocation/expiry latency and full-client-mix soak/cleanup evidence remain pending.
## Previous Task - Phase 8D.3 Inert Login Rendering
- **Scope:** Second permitted preparatory split after scope review of remaining challenge/throttle/HTTP-route/application work. Added only `src/web_login_ui.{c,h}`, CMake registration, `tests/web_login_ui/` and documentation. Standalone renderer has no live HTTP caller or URI registration; Basic cache/auth, existing app and serial protocol, capacities, task stacks and generated assets are unchanged. No upload, erase, commit or branch change.
- **Behavior when integrated:** No fetch on page load; explicit Sign in obtains challenge with `X-Login-Bootstrap: 1`, then POSTs JSON with CSRF. Same-origin credentials/mode, no-store fetch, redirect rejection, 512-byte response bound and UTF-8 field/body limits. Generic safe-text errors, bounded Retry-After display/manual backoff, no automatic credential retry, fixed success navigation to `/`. Inputs disabled while pending; password fields/references cleared best-effort, attempt aborted on every exit, 15-second deadline, pagehide/pageshow generation guards. No localStorage/cookie access or logging; JavaScript/browser memory cannot be securely wiped.
+13 -1
View File
@@ -40,7 +40,7 @@ Only constraints supported by implementation or current project documentation be
**Consequence for future changes:** Preserve transport-slot generations and account-authentication generations as distinct concepts. Validate tokens immediately before side effects and discard late work after disconnect/reuse/revocation.
Phase 8D.2 adds a third identity: non-reused 64-bit originating web-session IDs in serial tickets/slots. Zero identifies only the existing Basic path until cutover. Session-specific cleanup must not become account-wide cleanup; account-name notification intentionally covers deletion/recreation. Invalidate cookie records before requesting transport cleanup, and retain authoritative session/principal checks when notification fails. The transport epoch cancels in-flight ticket publication without taking store and transport locks together.
Phase 8D.2 adds a third identity: non-reused 64-bit originating web-session IDs in serial tickets/slots. 8D.3 rejects zero IDs; Basic authentication/cache are removed. Session-specific cleanup must not become account-wide cleanup; account-name notification intentionally covers deletion/recreation. Invalidate cookie records before requesting transport cleanup, and retain authoritative session/principal checks when notification fails. The transport epoch cancels in-flight ticket publication without taking store and transport locks together.
**Relevant files:** `src/session_broker.{h,c}`, `src/ssh_transport.c`, `src/web_serial_transport.c`, `src/admin_ssh_console.c`, `src/user_database.{h,c}`
@@ -94,6 +94,18 @@ Phase 8D.2 adds a third identity: non-reused 64-bit originating web-session IDs
**Relevant files:** `src/user_database.{h,c}`, `src/user_console.c`, `src/web_server.c`, `src/web_serial_transport.c`, `src/ssh_transport.c`
## Browser authentication has a narrow version-pinned HTTPD boundary
**Decision:** 8D.3 uses `web_cookie_auth` plus digest-only session/challenge stores, mandatory Origin/CSRF mutations and no live session/challenge/ticket eviction. Four one-hour absolute sessions deliberately interrupt long serial connections at expiry. No Basic compatibility path remains.
**Browser Origin serialization:** Authentication POST fetches use `mode: 'cors'` while retaining fixed same-origin URLs, `credentials: 'same-origin'`, redirect rejection and CSP `connect-src 'self'`. Under `no-referrer`, non-CORS POST mode can serialize Origin as `null` (confirmed in Firefox during M1 testing). Do not fix that by accepting null server-side or weakening CSP/referrer policy; no cross-origin server permission is added.
**HTTPD boundary:** `web_httpd_adapter` alone includes private ESP-IDF 5.5.0 structures. Public request getters expose only the first field, so the adapter validates bounded parsed headers/rejects duplicates. `/ws/serial` is an ordinary GET until authenticated transport admission explicitly sends 101 and installs the frame handler; automatic HTTPD upgrades happen before URI handlers. Preserve right-aligned unread pending bytes when wiping request memory. Two Set-Cookie calls append pointer-backed fields, whose distinct buffers must survive through send. HTTPD logs above ERROR are compiled out to prevent header/ticket exposure.
**Consequence:** The version guard is not a source-hash guarantee. Re-audit layout, scratch/pending ownership, logging and handshake/frame dispatch on SDK changes; do not scatter private accesses through application code or assume host doubles prove real socket behavior. No SDK patch is currently applied. See `docs/phase8d3_implementation.md` for verification and target gates.
**Relevant files:** `src/web_cookie_auth.{c,h}`, `src/web_session_store.{c,h}`, `src/web_httpd_adapter.{c,h}`, `src/web_server.c`, `src/web_serial_transport.c`, `src/CMakeLists.txt`.
## Security material and configuration use bounded, versioned NVS records
**Decision:** Application settings, users, and identities use separate fixed/versioned NVS blobs. Serial, Wi-Fi, mDNS-hostname, and local-UI working edits are RAM-only until explicitly saved. User mutations and HTTPS/SSH identity changes commit directly as part of the operation. Invalid ordinary configuration generally selects RAM defaults without erasing storage; malformed security material fails closed and needs explicit reset.
+139
View File
@@ -0,0 +1,139 @@
# Phase 8D.3 — Live browser authentication cutover
Status (2026-09-05): **Implemented / host-tested / build-verified; both-role login and mixed-client target samples received, M1 validation in progress.** The Origin-null login blocker is resolved by user-confirmed successful login after the fix; no full M1 sign-off is implied. Resumed another agent's uncommitted implementation, verified it against installed ESP-IDF, fixed pending-buffer cleanup, and refreshed documentation. Earlier parser/login-renderer records are historical preparatory checkpoints. 8D.08D.2 user sign-offs stand; numeric reserve gates remain open. Do not start 8D.4 before M1 sign-off or an explicit user decision.
## Successful post-fix target login and mixed-client sample (2026-09-05)
The user explicitly reports successful HTTPS login as both `commander1024` (user) and `admin` after the Origin-mode correction. This supersedes the earlier login-blocker statements below. The following are user-provided sequential snapshots, not agent-executed tests or an atomic measurement. Exact flashed revision/hash and settled duration for this new sample were not supplied.
| Heap | Settled boot free / minimum / largest (B) | Mixed load free / minimum / largest (B) |
|---|---|---|
| Internal 8-bit | 71,204 / 66,752 / 31,744 | 33,868 / 13,756 / 25,600 |
| Internal DMA | 63,448 / 58,996 / 31,744 | 26,112 / 6,000 / 25,600 |
| PSRAM | 8,247,744 / 8,245,836 / 8,126,464 | 8,089,060 / 8,072,744 / 7,995,392 |
| SSH stack minimum-free | 18,464 B (20,480 B configured) | 16,288 B |
**Settled boot:** UART service stopped/owner idle at 115200 8N1/no flow; no broker clients or SSH sessions. USB initialized/attached but host closed/DTR false. HTTPS ready, sessions/challenges/tickets zero, all supplied web and SSH request/traffic/failure counters zero. mDNS initialized/announced as `sak-1024.local`, last error ESP_OK. The USB host's diagnostic 9600 coding does not configure UART1.
**Mixed-client load:** UART running at 115200 8N1/no flow. Four broker clients: web user 8 observer, user SSH 9 **sole writer**, USB 10 observer, web admin 27 observer. Public-key user/admin SSH both active (2/2); admin SSH has no broker client. Two password-authenticated web serial sessions and two cookie sessions, no outstanding challenges/tickets. USB host open/DTR/RTS asserted; diagnostic line coding 115200. RX available/TX pending zero at the serial snapshot; all four displayed broker pending/event queues zero. mDNS remains announced without errors.
- Two consecutive loaded `memory` samples are identical. This is short-term observation, **not** leak/soak/cleanup or reserve-floor validation. Lifetime minima include handshake/earlier activity; DMA overlaps internal heap. The **6,000 B DMA minimum** leaves runtime reserve analysis important even though current DMA free is 26,112 B. No stack fault/watchdog or memory exhaustion is reported.
- SSH: two successful handshakes/auth attempts, no handshake/auth/I/O/session-revocation failures. Stream RX 73 accepted, zero rejected, TX 31,941 B. Broker initial writer request was denied once; later writer snapshot shows SSH owns the lease (no inconsistency inferred from cumulative counters). Admin command-running/output-pending fields were sampled while executing status commands, not proof of a stuck dispatcher.
- Web: 97 protected requests, 96 authenticated and one auth failure; four roots, 80 status, four tickets, eight assets; zero response errors. Four tickets issued/consumed, zero rejected/expired. Four serial connects/two disconnects; two currently active. RX one accepted frame/14 B with no rejection; TX 495 binary frames/43,738 B and 15 control frames/1,255 B. Writer requests four, grants one, denials three, releases one; no revocations. No reported send/queue/protocol/service-start/broker/connection failures.
- Cookie auth: six password checks, three invalid-credential results, zero throttle/capacity/CSRF-or-Origin rejections, one logout, two active sessions. Successful both-role login is explicit user confirmation; the counters also show logout/reconnection activity but do not establish five cycles, account-isolation coverage or logout acknowledgement delivery. The three invalid-credential results and one protected-request auth failure are retained without attributing a cause.
**Reported admin-SSH empty-line issue:** pressing Enter without text prints “Command is restricted to physical UART0.” Source trace identifies `remote_command_allowed()` classifying zero parsed arguments as a policy denial. Corrected that helper to allow empty input to reach IDF's normal quiet `ESP_ERR_INVALID_ARG` handling; existing current-admin/generation checks still run, and `user bootstrap`/`user recover` remain denied. No new dispatcher, route or 8D.4 refactor. `python3 tests/admin_ssh_policy/run.py` passes 15 policy cases with the production helper and installed IDF argument parser, including quoted restricted commands. `pio run` passes in **21.04 seconds**, **95,508 B RAM / 1,625,725 B flash** (+20 B flash versus Origin fix). This SSH fix is **not yet target-tested** and was not present in the user's sample. No upload/erase/commit performed.
Remaining: explicit M1 sign-off, repeated lifecycle/expiry/revocation/isolation and raw-client security checks, timed full-load soak and settled cleanup, numeric reserves and non-SSH owner stack margins. Do not reopen prior phase sign-offs or invent missing execution evidence.
## First target sample and login blocker (historical, user-provided, 2026-09-05)
After clean boot and 60 seconds settled, the user reports:
| Heap | Free | Minimum-free | Largest block |
|---|---:|---:|---:|
| Internal 8-bit | 69,004 B | 66,752 B | 31,744 B |
| Internal DMA | 61,248 B | 58,996 B | 31,744 B |
| PSRAM | 8,223,116 B | 8,218,204 B | 8,126,464 B |
SSH has 0/2 sessions, all supplied error/traffic counters zero, configured stack 20,480 B and minimum-free 18,464 B. HTTPS is running/ready with no lifecycle/response errors, cookie sessions 0/4 and challenges 0/4; eight protected requests were unauthenticated, with zero password-verification attempts, CSRF/origin rejections, logouts, tickets or serial sockets at this sample point. mDNS announces `sak-1024.local`. UART1 is stopped/owner idle, configured 115200 8N1/no flow; no broker clients. USB is initialized/attached but host-open/DTR false, broker disconnected. Its reported 9600 host coding does not configure UART1. These snapshots precede the reported login attempts; they do not establish post-attempt counters. Exact flashed revision/hash was not supplied.
**M1 is blocked:** subsequent login attempts for a normal user and administrator both show “The sign-in challenge expired or the request was rejected. Please try again.” Browser console reports blocked inline scripts with two hashes different from the application's login script hash, denied favicon by default-src, and a denied file URL. No browser login, loaded-memory or full M1 acceptance is claimed. These errors do not establish memory exhaustion or invalid passwords.
The production-renderer suite was rerun and the exact shipped inline-script hash still matches its CSP (`x70ID2kbifGBVYfh/pePTt5v/AVHkT7JVAV0LjT1wCo=`). The displayed login message maps to HTTP 403 in the running script; the console's other hashes may be injected-script warnings, not a reason to broaden CSP. Request-stage/status and the bounded error code plus nonsecret Origin/Fetch Metadata are needed to isolate the rejection. No corrective firmware change has yet been made for this target report.
## Confirmed Origin-null diagnosis and correction (2026-09-05)
Follow-up user evidence: `/api/login-challenge` returns 200, `/api/login` returns 403 with request `Origin: null`, `Sec-Fetch-Site: same-origin`, and the pre-login cookie present. `web status` reports ready, zero sessions/challenges/tickets, **zero password-verification attempts** and **seven CSRF/origin rejections**. This confirms the rejection occurs before password authentication; it is not evidence of wrong credentials. No secret values were requested or retained.
Cause: the authored fetch requests used non-CORS `mode: 'same-origin'` under `Referrer-Policy: no-referrer`; browser Origin-header serialization for these POSTs yields `null`. Corrected login fetch options to `mode: 'cors'` and the existing app API helper to use `cors` for POST (ticket/logout), retaining same-origin mode for app GETs. Fetch CORS mode is not permission for cross-origin service access: paths remain fixed same-origin endpoints, credentials remain `same-origin`, redirects remain rejected, CSP `connect-src 'self'` remains intact, and the server's strict Origin/CSRF checks/no-CORS-response policy are unchanged. The login script hash was updated atomically to `eZO4pMDQx6SIaa5AFlMnuf0CD5JdGSWyi8lNVmCNPBQ=`; existing app loader hash is unchanged because only its external app script changed.
Validation: login renderer/CSP eight Node groups, app nine Node groups, and cookie-policy suite all pass. Node guards assert CORS mode for every mutation (including logout), fixed endpoint destinations and no manually supplied Origin. They do not synthesize real browser Origin headers; Firefox/target retest is still required. `pio run` passed in **14.30 seconds**, **95,508 B RAM / 1,625,705 B flash** (RAM unchanged, flash +16 B versus the preceding live build). No upload/erase. Retest both roles, serial Connect/Disconnect/reconnect and Sign out; expect login POST Origin `https://sak-1024.local` (or the actual direct-IP origin), not null. M1 remains blocked until confirmed on target; other CSP warnings were not loosened or assumed resolved.
## Delivered behavior
- `web_cookie_auth.{c,h}` replaces Basic authentication/cache completely. Both roles use `/login` and the same serial/status application. No admin shell/settings routes were added. Previously cached Basic headers do not authorize a request.
- Four digest-only authenticated sessions retain the existing store's copied principal, canonical-origin binding, CSRF state, non-reused ID and one-hour absolute lifetime. Traffic/polling does not renew expiry. Failure to initialize authentication prevents HTTPS start; UART0/USB/SSH implementations remain unchanged.
- Four 120-second pre-login challenges contain only token/origin digests, CSRF state and expiry. Explicit login bootstrap requires `X-Login-Bootstrap: 1`; a matching live challenge is reused without extending its lifetime or resetting its cookie. Credential submissions consume the challenge, including wrong passwords. A global fixed window permits five password verifications per 60 seconds, including successes. Further attempts return 429 with Retry-After; no HTTPD sleep or per-IP/account table.
- Session and pre-login cookies use `__Host-sak-session` / `__Host-sak-prelogin`, `Secure; HttpOnly; SameSite=Strict; Path=/`, explicit Max-Age 3600/120 and no Domain. A consumed challenge expires its cookie; successful login additionally sets a fresh session cookie. Login with a current authenticated cookie returns 409; account switching requires logout.
- Mutations require canonical same-origin HTTPS Origin and CSRF; upgrade requires Origin and matching cookie/session/ticket. Host case and optional default port 443 normalize; non-443 ports, malformed authorities and IPv6 literals are rejected. Direct-IP and mDNS names remain distinct cookie origins. Cross-site/same-site Fetch Metadata requests are rejected (same-origin/none accepted); absent Origin is permitted only on read/bootstrap requests after Host validation.
- Exactly username/password string fields, maximum 512-byte login JSON, decoded 16/64-byte limits. Unknown/duplicate fields, NUL and malformed Unicode fail. Header/body/request scratch is wiped; rejected unread bodies close instead of invoking HTTPD's automatic body drain. Login reads have a three-second application deadline plus existing socket wait bounds. API authentication responses are at most 512 bytes; safe username JSON encoding is explicit.
- Full live session/challenge/ticket tables reject with 503 and Retry-After 5; serial-ticket earliest-expiry eviction is removed. Expired/stale tickets are reclaimed without database calls under the transport lock. Existing two serial sockets, one-writer broker model and binary protocol are unchanged.
- Logout invalidates its originating session before acknowledgement and requests only its ticket/socket cleanup. Account mutation/revocation continues to invalidate all affected account sessions, not unrelated accounts. Mint/consume/admission/input and existing periodic owner checks remain authoritative if notification fails. Zero session ID no longer falls back to Basic.
- Browser validates `/api/session` before initial connect/reconnect/restore; stores CSRF only in memory; adds Sign out and visible absolute expiry. 401 cancels work/closes local serial/navigates once to `/login`; explicit Disconnect still pauses reconnect without ending login. 403 mutation failures require explicit retry; capacity/backoff and network errors are not bad credentials. Lost logout response is reconciled with session status rather than claiming success. Pending fetch/socket callbacks are generation-guarded. Both authored inline scripts have exact CSP hashes; generated assets were not regenerated.
### Route boundary
| Route | Policy |
|---|---|
| GET `/login` | Public standalone no-store login page, no protected assets |
| GET `/api/login-challenge` | Validated Host, bootstrap header, Fetch Metadata and any supplied Origin |
| POST `/api/login` | Strict Origin, pre-login cookie/CSRF, bounded JSON and throttle |
| GET `/api/session` | Current cookie session; username/role/CSRF/remaining seconds only |
| POST `/api/logout` | Current session, strict Origin/CSRF, empty body |
| GET `/` | Current session; unauthenticated navigation gets 303 `/login` |
| GET five `/assets/` routes; GET `/api/status` | Current session; unauthenticated gets 401, not login HTML |
| POST `/api/ws-ticket` | Current session, strict Origin/CSRF, empty body |
| GET `/ws/serial?ticket=...` | Cookie/Origin authorization and ticket/principal/broker admission before explicit 101 |
No CORS/preflight compatibility or Basic fallback. Query strings outside the exact serial-ticket route and wrong methods are rejected. Error routes have bounded no-store responses. The login document itself also rejects malformed/duplicate cookies; manually corrupted cookies may require clearing those site cookies, unlike ordinary expired well-formed cookies.
## Verified HTTPD boundary and maintenance risk
The delivered solution is **not the previously proposed SDK patch**. `web_httpd_adapter.{c,h}` alone includes installed HTTPD private headers. `src/CMakeLists.txt` supplies private include paths; the adapter fails compilation unless `ESP_IDF_VERSION == 5.5.0`. No installed SDK source was changed and no full component was vendored.
Verified under `/home/mscholz/.platformio/packages/framework-espidf/components/esp_http_server/`:
- `src/httpd_parse.c`, `httpd_req_get_hdr_value_len/str`: return the **first** matching header only. Parsed fields occupy NUL-separated scratch, not a raw CRLF block. Adapter walks that bounded storage and rejects **all duplicate field names**, case-insensitively, plus malformed fields, control characters, Transfer-Encoding and Expect. This is stricter than general HTTP acceptance, deliberately fail-closed. Public getters are called only after validation and with terminator capacity.
- `src/httpd_txrx.c`, `httpd_resp_set_hdr`: appends pointers, does not replace an earlier same-name field. Sending emits each entry; login retains its two cookie values until send returns. Success uses exactly **six of eight additional-header slots**. Tests extract the installed getters and append function rather than inventing their behavior.
- `src/httpd_uri.c`: routes marked `is_websocket=true` send 101 before their handler. The application's serial URI is deliberately registered as an ordinary GET. After cookie/Origin checks, transport consumes the matching ticket and completes currentness/broker admission, then adapter calls `httpd_ws_respond_server_handshake()` and installs the existing transport frame handler. Failed pre-admission never sends 101; handshake/admission failure uses existing reserved-slot/broker cleanup. Tests stub the handshake send: real on-wire integration remains a target gate.
- `src/httpd_txrx.c`, `httpd_unrecv/httpd_recv_pending`: pending bytes are **right-aligned**. The inherited adapter incorrectly wiped the unread suffix. This continuation fixes cleanup to wipe the consumed prefix while preserving unread bytes at the end, or wipe everything when closing. Regression exercises all 0128 pending lengths and partial reads through the installed reader function. This prevents corruption of pipelined requests/early serial frames; it is not a claim of real socket execution.
- HTTPD DEBUG logs include headers, and URI warnings can include ticket queries. HTTPD is compiled with `LOG_LOCAL_LEVEL=ESP_LOG_ERROR`; ERROR sites were inspected for secret-bearing content. This deliberately removes HTTPD warning/debug diagnostics regardless of runtime log-level changes. Application count-only authentication telemetry remains available via `web status`/`web counters`.
Private layout, frame dispatch and scratch ownership must be re-audited for an SDK update, including same-version local source patches (the guard checks the version, not source hashes). Do not distribute private-structure access into other application modules. Wiping reduces request lifetime, not all TLS/allocator/browser copies; do not claim resistance to RAM extraction.
## Resource accounting
Final `pio run` passed in **17.62 seconds** after the cleanup fix:
| Metric | 8D.2 / preparatory baseline | Live 8D.3 | Increment |
|---|---:|---:|---:|
| Linked static RAM | 95,260 B | 95,508 B | +248 B |
| Reported program flash | 1,601,925 B | 1,625,689 B | +23,764 B |
Cumulative versus recorded 8D.0 build (94,532 / 1,599,973 B): **+976 B RAM / +25,716 B flash**. These are linked sizes, not runtime headroom.
- Target object symbol accounting: challenges **576 B (144 × 4)**, counters 32 B, lock 8 B, epoch 8 B, window 8 B, attempts 4 B, ready 1 B: **637 B before placement padding**. Removed Basic cache/key/readiness offset most of this; final link delta includes alignment/other changes. Existing session store remains present.
- No new application task, task-stack size change, module heap allocation, queue, TLS buffer, accepted socket or lwIP descriptor limit. HTTPD URI capacity rises **9 → 14**, with five additional dynamically allocated handler records; HTTPD error handlers use its existing table. Six HTTPS clients and two web serial slots remain unchanged. LRU purge remains enabled; retained-serial admission protection is still an M2 concern.
- Auth request locals include 513 B body/response scratch, 180 B cookie header, token/CSRF/canonical buffers, copied session/principal/challenge/credentials; cookie parsing has nested 1025 B header scratch. No task-stack reserve is inferred from source locals or static link size. Existing HTTPD stack is 10,240 B; real worst-case stack/TLS/PBKDF2/fragmentation measurements remain pending.
- Existing xterm/logo data unchanged. Login page and enlarged authored app are now actually linked; their dormant-preparation flash numbers were not their live cost. Header slots remain eight; login success six, login renderer five.
## Executed validation
All ran successfully in this continuation:
```sh
python3 tests/web_cookie_auth/run.py
python3 tests/web_auth_parse/run.py
python3 tests/web_login_ui/run.py
python3 tests/web_ui_session/run.py
python3 tests/web_session_store/run.py --serial
pio run
```
- Cookie policy suite compiles production store/parser/policy/adapter with OpenSSL SHA-256 and deterministic database/HTTPD doubles. Covers fragmented reads, challenge reuse/consumption/expiry/capacity, session-specific logout, throttle, duplicate headers/cookies, methods/Origin/CSRF/Fetch Metadata, Basic denial, currentness, failures/stop race, cookie header budget and explicit upgrade state. Installed IDF getter/setter/pending-reader functions are extracted verbatim. It does **not** execute the full IDF parser, TLS, URI dispatcher, network handshake or real tasks.
- Parser suite: **268 cases**. Login renderer: production C failure/header checks and **eight Node groups**. Serial app: production C resource/header/CSP checks and **nine Node groups**. Node VM DOM/fetch doubles are not a real browser/CSP/bfcache test.
- Serial integration mode includes store public-API tests plus transport binding/isolation/races and no Basic/no live-ticket eviction. No sanitizer pass is claimed.
No upload, erase, commit, branch change or target/browser exercise was performed. The preceding agent's changes were preserved except the focused pending-buffer fix/tests; its unrecorded executions are not evidence here.
## M1 target acceptance handoff — stop before 8D.4
Use the complete [M1 contract/checklist](phase8d_baseline.md#minimal-m1-browser-contract-planned) and [user acceptance matrix](user_administration_tests.md#planned-phase-8d-integrated-web-administration). At minimum:
1. Keep UART0 attached. Record flashed revision/configuration and settled-boot `memory`, `web status`, `web counters`, `broker clients`, `ssh status`. Confirm native USB and both SSH roles survive HTTPS stop/start and authentication failures.
2. Test both roles, fresh and previously Basic-authenticated profiles, direct IP and mDNS. Wrong credentials, refresh/back, expiry/reboot, sign out/account switch and lost logout response must remain usable. Verify actual secure cookie attributes and CSP; never include raw cookies/CSRF/tickets/passwords in shared evidence.
3. Five login/serial-disconnect/reconnect/logout cycles per role; five HTTPS stop/start cycles. Check session-specific logout across two sessions of the same account, and account password/role/key changes/deletion/recreation via UART0 while unrelated sessions survive.
4. Challenge/session/ticket capacity without eviction; bounded throttle and retry. Raw-client missing/malformed/duplicate Origin/Host/Cookie/CSRF/content-type/framing tests. Verify an unauthorized or mismatched-ticket upgrade gets **no 101**; validate actual frame routing, early/pending bytes and close cleanup after admitted upgrades. These are especially important for the private adapter.
5. Fifteen-minute full-client mix at 115200 baud (USB, two web serial clients, user SSH and admin SSH), then 60-second cleanup. Record internal/DMA/PSRAM free/minimum/largest block plus SSH stack margins at boot/login/serial/load/cleanup. Check binary integrity, writer isolation, drops and watchdogs. Measure the planned ≤1-second expiry/revocation detection target under contention separately from socket-close delivery.
6. Numeric reserve floors and non-SSH owner-stack instrumentation remain pending. Obtain explicit M1 sign-off before adding the browser admin shell. Do not equate host tests/build success with target acceptance.
+2 -2
View File
@@ -1,10 +1,10 @@
# Phase 8D.0 — Baseline and M1 browser contract
Status: **8D.0 and 8D.1 validated by user sign-off on 2026-09-05.** Documentation/source audit, builds and target runtime samples are recorded. Numeric reserve floors and future incremental budgets remain open engineering gates, not blockers to these user-approved closures. The M1 browser contract below remains planned, not implemented authentication. See [execution plan](phase8d_plan.md) and [acceptance matrix](user_administration_tests.md#planned-phase-8d-integrated-web-administration).
Status: **8D.0 and 8D.1 validated by user sign-off on 2026-09-05.** Documentation/source audit, builds and target runtime samples are recorded. Numeric reserve floors and future incremental budgets remain open engineering gates, not blockers to these user-approved closures. The M1 browser contract below was established during baseline planning; the subsequent [8D.3 live cutover record](phase8d3_implementation.md) now documents implemented/host-tested/build-verified authentication, with target/browser acceptance pending. Baseline measurements and source-behavior descriptions here remain historical, not measurements of the live cutover. See [execution plan](phase8d_plan.md) and [acceptance matrix](user_administration_tests.md#planned-phase-8d-integrated-web-administration).
## Validation sign-off
The subsequent [8D.2 implementation record](phase8d2_implementation.md) contains its separate build/resource accounting and pending target checklist. The baseline and 8D.1 measurements below remain historical evidence, not 8D.2 target validation.
The subsequent [8D.2 implementation record](phase8d2_implementation.md) contains its separate build/resource accounting, target samples and user sign-off. The baseline and 8D.1 measurements below remain historical evidence, not 8D.2 target validation.
- The user explicitly marked **8D.0 validated** and identified the tested firmware with the latest checked-in project state, resolved at sign-off to Git revision **`d8999cd4a96e477fabd392ced02d810c3cd22d0f`**. This is user-confirmed source provenance, not an independently read-back device binary hash. The earlier reproducible build revision and SHA-256 table remain historical build evidence, not newly generated hashes for this revision.
- The user attributes the SSH I/O errors to testing at **out-of-spec 460400 baud**. Preserve that exact reported rate separately from the transcripts' **460800-baud UART configuration**; the differing rate may describe the test setup, and no firmware baud-support change or independently reproduced diagnosis is implied. The session-revocation counter remains recorded without a separately supplied causal explanation.
+5 -3
View File
@@ -1,6 +1,6 @@
# Phase 8D — Incremental web administration plan
Status: **8D.0 through 8D.2 validated by user sign-off on 2026-09-05. 8D.2 implemented, host-tested and build-verified. Reserve budgets remain pending. 8D.3 has inert request-parser and login-renderer preparatory splits; its live cutover and 8D.48D.22 remain planned.** See the [8D.0 baseline and M1 contract](phase8d_baseline.md); no browser authentication cutover is claimed.
Status: **8D.0 through 8D.2 validated by user sign-off on 2026-09-05. 8D.3 live login/logout cutover implemented, host-tested and build-verified; mandatory M1 target/browser validation and numeric reserve gates remain pending. 8D.48D.22 remain planned.** See the [8D.3 implementation record](phase8d3_implementation.md) and [8D.0 baseline/M1 contract](phase8d_baseline.md).
This is the execution plan for [roadmap Phase 8D](roadmap.md#phase-8--role-based-users-and-administrative-access). The roadmap retains the feature/security requirements; this document defines small work units, dependencies, and release gates. The [administration test matrix](user_administration_tests.md#planned-phase-8d-integrated-web-administration) remains the final acceptance checklist.
@@ -90,6 +90,8 @@ After each runtime-changing chunk: build; boot and read UART0 status/`memory`; l
If 8D.3 exceeds the work-unit limit, first split out inert login-page rendering or private request-parsing helpers. Do not split the live security cutover into an insecure intermediate deployment.
**Live cutover checkpoint (2026-09-05):** [Implementation, HTTPD boundary, resource accounting and M1 handoff](phase8d3_implementation.md). Cookie login/logout replaces Basic for app/status/ticket routes; explicit pre-101 admission and strict header/Origin/CSRF policy use an isolated version-checked private IDF adapter, **not an SDK patch**. Resumed another agent's implementation and fixed pending-buffer wiping to preserve right-aligned unread data. All five focused suites and `pio run` pass. Final **95,508 B RAM / 1,625,689 B flash**, +248/+23,764 B versus 8D.2. No hardware/browser execution or reserve-floor approval. **Stop for mandatory M1 acceptance before 8D.4.** The following preparatory records are historical, superseded for current implementation status.
**Preparatory split (2026-09-05):** Scope review selected private request parsing first; the complete challenge/throttle/route/browser/test change exceeds the authored-line work-unit target. Added allocation-free `src/web_auth_parse.{c,h}` with no live HTTP callers: bounded canonical same-origin comparison, unique cookie extraction and strict UTF-8 login JSON decoding. `python3 tests/web_auth_parse/run.py` passes **268 cases** against production C; both existing session-store host modes pass. Final `pio run` passes in **7.50 seconds** and reports **95,260 B RAM / 1,601,925 B flash**, unchanged from 8D.2 because helpers are not live linked paths. No route/task/socket/stack-size/asset changes. No target/browser validation, runtime reserve approval or M1 completion is implied. Continue within **8D.3**, with the full atomic cutover still pending; see `docs/agent/current-state.md` for exact integration obligations.
**Login-renderer preparatory split (2026-09-05):** Added `src/web_login_ui.{c,h}` with no live caller/route, leaving Basic and existing serial UI unchanged. Standalone 7,387-byte HTML plus terminator has no protected asset dependencies, five security headers including no-store and exact script-hash CSP. Explicit-only challenge/login flow, bounded request/response handling, safe errors/manual backoff, disabled pending inputs, best-effort password/reference cleanup and generation-safe page-exit cancellation. Review fixes abort every attempt on exit (including unread error bodies) and clear re-entered passwords. `python3 tests/web_login_ui/run.py` passes production C rendering/failure checks, exact CSP hash and eight Node VM groups; parser and both session test modes also pass. Final `pio run` passes in **8.25 seconds**, unchanged **95,260 B RAM / 1,601,925 B flash**: unused renderer costs are not live-linked/runtime costs yet. No new task/socket/route/stack-size/module heap or generated-asset change. No real-browser/HTTPD/hardware validation or M1 completion. **Next is the atomic live 8D.3 cutover using both prepared pieces**, followed by the mandatory M1 target gate; no additional login-rendering split is needed.
@@ -163,11 +165,11 @@ Update the roadmap and user/command documentation to distinguish completed featu
## Progress and next-request template
Progress: **8D.0 through 8D.2 validated by user sign-off; 8D.2 implemented / host-tested / build-verified. Reserve gates pending. 8D.3 inert parser and login-renderer preparation implemented; live cutover and 8D.48D.22 planned.** Record incremental results in `docs/agent/current-state.md`, retaining the [baseline](phase8d_baseline.md) and cumulative resource measurements as work proceeds. The baseline records user-provided evidence and sign-off; this does not imply completion of later browser-authentication acceptance checks.
Progress: **8D.0 through 8D.2 validated by user sign-off. 8D.3 live cutover implemented / host-tested / build-verified, awaiting M1 target/browser acceptance and numeric reserves. 8D.48D.22 planned.** Record incremental results in `docs/agent/current-state.md`, retaining the [baseline](phase8d_baseline.md) and cumulative resource measurements as work proceeds. The baseline records user-provided evidence and sign-off; this does not imply completion of later browser-authentication acceptance checks.
Suggested next request:
> Continue Phase 8D.3 with the atomic live login/logout cutover; request parsing and standalone login rendering are prepared and host-tested. Preserve 8D.2 sign-off and open numeric reserve gates, and stop for M1 target/browser validation before 8D.4.
> Validate the live Phase 8D.3/M1 cutover on target using its implementation record. Record browser/transport/lifecycle and memory evidence, resolve any failures, and obtain explicit M1 sign-off before beginning 8D.4. Preserve existing phase sign-offs and open numeric reserve gates.
For later chunks:
+1 -1
View File
@@ -199,7 +199,7 @@ Implementation sequence:
- Keep SFTP, SCP, `exec`, forwarding, subsystems, and unauthenticated shells disabled.
- Target-hardware validation passed for route separation, history/Tab editing, interactive visible/hidden prompts, output/backpressure, generated and entered user/password/key management including the longest ECDSA P-256 import, ping event routing, deferred reboot/SSH lifecycle drain behavior, bootstrap/recovery rejection, targeted self/other-user revocation during queued work, UART0/SSH administration serialization, and concurrent USB/WebSocket/user-SSH/admin-SSH operation. Stress at 460800 baud with SSH and WebSocket clients caused substantial expected packet drops and slower display controls, but did not exhaust memory or require lowering the supported baud-rate range.
4. **Phase 8D — Integrated web administration — Planned, staged delivery**
- **Implementation checkpoint:** 8D.0/8D.1 validated by user sign-off; [8D.2 serial/session binding](phase8d2_implementation.md) implemented, host-tested and build-verified, with target regression and numeric reserve gates pending. Basic remains the public authentication path; browser login/logout and later milestones below are not yet implemented.
- **Implementation checkpoint:** 8D.08D.2 validated by user sign-off. [8D.3 live login/logout](phase8d3_implementation.md) implemented, host-tested and build-verified with cookie sessions and no Basic fallback; mandatory M1 target/browser validation and numeric reserves remain pending. Browser admin shell/settings and later milestones below remain planned. Stop before 8D.4 until M1 sign-off.
- **Execution plan:** [Phase 8D incremental plan](phase8d_plan.md). Implement one numbered chunk per request, with a build, focused regression checks, memory accounting, and a handoff before stopping. The requirements below describe the final scope, not one implementation task.
- **Milestones:** 8D.08D.3 establish a measured baseline and reliable login/logout with the existing serial UI; 8D.48D.7 add the shared browser admin shell and verify retained serial ownership; 8D.88D.21 add typed settings and contextual controls one domain at a time; 8D.22 performs final integration acceptance. Login and runtime-memory target validation gate the first two milestones; do not defer them until the entire phase is implemented. No wholesale import of the rolled-back experimental implementation.
- Begin with integrated authentication: replace browser-facing HTTP Basic authentication with a same-origin HTTPS login page, explicit logout, and bounded opaque server-side sessions. Store only a digest of each random session token with a copied secret-free principal, expiry, CSRF state, and authentication-generation binding. Send the raw token only in a host-only `__Host-` cookie with `Secure`, `HttpOnly`, `SameSite=Strict`, `Path=/`, no `Domain`, and an explicit lifetime; never retain passwords, Basic headers, raw tokens, verifiers, or SSH-key blobs in snapshots or logs.
+1 -1
View File
@@ -168,7 +168,7 @@ Finally, issue commands concurrently from UART0 and admin SSH, including `user l
## Planned Phase 8D integrated web administration
These are acceptance requirements for the planned implementation, not tests that have passed yet. Execute them incrementally using the [Phase 8D work-unit plan](phase8d_plan.md), not only at the end of the phase.
These are acceptance requirements, not tests that have passed yet. The [8D.3 live login/logout implementation](phase8d3_implementation.md) is host-tested/build-verified and awaits M1 target/browser execution; browser administration remains planned. Execute them incrementally using the [Phase 8D work-unit plan](phase8d_plan.md), not only at the end of the phase.
Validation checkpoints: