ESP32_Serial_Swiss_Army_Knife/docs/phase8d8_implementation.md

Phase 8D.8 Implementation Record

Target sign-off (2026-09-08)

The user supplied settled boot/full-client-mix evidence and reports thorough testing of Serial parameter display/settings and user/account settings, explicitly authorizing sign-off of implemented work. 8D.8 is accepted, together with implemented 8D.9/8D.10. This supersedes target-pending/signoff statements below, not historical build evidence. Full telemetry/counters and scope limits are recorded in docs/phase8d10_implementation.md under Target acceptance. Exact flashed revision and individual checklist results were not separately supplied; do not invent them or reopen explicit acceptance solely for absent detailed records. Numeric memory reserves/stack margins and the earlier accepted admission followup remain open. No runtime change/build/device action in this documentation update; no later-phase authorization inferred.

Historical Status and Scope

2026-09-07: separately user-authorized 8D.8 is implemented / host-tested / build-verified. Target/browser validation and new phase signoff remain pending. No 8D.9 work, mutation/persistence controls, placeholder domains, all-subsystem settings snapshot, CLI-over-HTTP, generated-asset change, upload, erase or commit. Worktree was clean at task entry; no unrelated edits were reverted.

M2 remains explicitly signed off by the user ("Jupp, sign M2 off"). Accepted 8D.7 scope does not need revalidation. Deferred browser self-target/generated-password/key/legacy-credential and other owner-specific command restrictions remain unchanged; bootstrap/recovery remain permanently UART0-only. The intermittent supported two serial + one admin admission issue remains accepted nonblocking, unresolved and undiagnosed. Numeric memory reserves, low historical internal/DMA minima and stack margins remain unapproved followups. Nothing here establishes full browser command parity or fixes that admission issue.

Implemented Contract

• src/web_ui.c: Settings joins the existing admin-only Serial/Admin selector and opens the only implemented settings page, Serial, within the same document. It displays working UART1 service state, baud, data bits, parity, stop bits, flow control, DTR behavior and RTS threshold. It explicitly does not represent saved NVS state.
• Selection never creates/closes a serial or admin socket, joins/disconnects the broker, or requests/releases the writer lease. Both hidden terminals keep draining through their existing independent 5,000-line scrollbacks and 64 KiB pending-output bounds/drop notices. Only the selected terminal accepts keyboard input; neither does in Settings. Writer status and explicit writer controls remain available.
• Entry/Refresh first revalidates the existing username/role/session-stable CSRF identity, then requests GET /api/settings/serial. Settings validation must not supersede an in-flight serial-admission session check; newer admission may supersede an older Settings check, which returns to explicit retry. Identity change still requires a clean document before adoption. Absolute expiry is never extended.
• One active Settings refresh, no new periodic poll, no automatic retries, and the existing fixed-path/same-origin/no-store/redirect-denial fetch helper. Each fetch has a 15-second deadline; the two sequential session/snapshot requests can take up to approximately 30 seconds plus scheduling. Snapshot input is capped at 256 bytes, fatal UTF-8/JSON decoded, checked for exactly eight fields with bounded types/ranges/enums, and rendered only with textContent.
• Leaving Settings aborts and clears it. Logout/401/expiry/pagehide use existing two-route teardown and also clear/hide Settings. Late replies cannot repopulate it. Same-session bfcache restore revalidates before showing the empty page; Refresh is explicit and admin is not reopened automatically. Ordinary Settings failure does not close either terminal. Shared bounded error messages never display arbitrary response/error bodies; 429/503 retry hints are capped, and retries are manual.
• The external authored app script changed; the inline loader did not. Its two existing CSP hashes remain exact and unchanged, verified against the actual rendered loader. CSP was not relaxed. No vendored/generated assets were regenerated.

Backend and Ownership

serial_service_get_snapshot() copies serial_config_t and running state under one zero-wait acquisition of the existing serial state mutex. NULL is invalid; a non-NULL output is cleared on failure; uninitialized returns invalid-state and contention returns timeout. No UART/GPIO/NVS/broker IO, additional allocation or new lock. This avoids the blocking config getter on the new HTTPD route and cannot stall behind a console stop/reconfiguration. It is a point-in-time working snapshot, not a promise that later console changes cannot occur.

web_server.c authorizes with the existing real cookie/session/current-principal policy and then requires admin before any serial read. The route accepts only bodyless GET without query parameters. Existing Host/origin binding and Fetch Metadata policy apply: same-origin GET can omit Origin; supplied cross/null Origin rejects. GET is nonmutating and requires no CSRF token. Duplicate headers, malformed/oversized credentials, transfer framing, unknown query fields and bodies reject through existing bounded policy. Unread rejected bodies cause connection closure rather than unbounded draining. Auth/principal temporaries and consumed header scratch are wiped.

Success is bounded JSON (256-byte stack buffer including terminator), containing only fixed firmware enum names, boolean and integers. No secrets, configuration version/storage blobs, counters or unrelated subsystem data are serialized. Missing/expired/revoked login gives 401; normal role gives 403; invalid request gives 400; busy/uninitialized serial gives 503 with Retry-After: 1. Common error routes give fixed 404/405 for unknown routes/methods. Responses are no-store/nosniff/no-referrer. Header/send failures stop without a second response; existing aggregate request/auth/response counters are reused.

Registration is optional and independent of admin-console attachment: failure leaves base auth/serial/admin usable, with Settings returning the existing safe 404 and manual retry message. URI capacity increases from 16 to 17; sockets remain six with LRU disabled. No admission eviction or capacity change for sessions, challenges, tickets, console slots or WebSockets.

IDF Allocation Boundary

Review of installed IDF 5.5.0 httpd_uri.c:147-163 found that public URI registration publishes an allocated descriptor before duplicating its name; name-allocation failure frees it without clearing hd_calls. Ignoring that failure for the new optional route could leave a dangling table entry. web_httpd_register_optional_get() in the existing private adapter stages both ordinary heap allocations before publishing the fully initialized entry. It is restricted to serialized startup, exact-match GET, no WebSocket/subprotocol and a 127-byte URI limit; duplicate/full/invalid shape rejects without publication. HTTPD retains normal ownership and frees both allocations. No SDK patch or runtime registry framework was added.

The new route alone uses this helper. Existing public registration callers were not refactored; broader inherited SDK allocation-failure handling remains a followup, not a diagnosed cause of the accepted admission issue. The version-pinned adapter now also requires registration/table/free-ownership re-audit on SDK updates. Host tests inject failure at both actual adapter allocations and compile the installed unregister implementation to exercise successful cleanup. They do not prove multicore registry timing or real low-heap behavior.

Resources

Final release pio run, tool timeout 120 seconds, passed in 24.31 seconds, PlatformIO espressif32 6.12.0 / ESP-IDF 5.5.0 / N16R8:

Resource	Final / Delta
Linked RAM	95,580 B, 0 B vs recorded 8D.7; +1,048 B vs 8D.0
Flash	1,654,529 B, +5,952 B vs recorded 8D.7 (1,648,577 B); +54,556 B vs 8D.0 (1,599,973 B)
HTTPD handlers	17, +1; table +4 B on target
New route allocations	Target descriptor 24 B + URI/name terminator 21 B = 45 B; with table growth 49 B requested heap, excluding allocator rounding/metadata
Snapshot / response scratch	Target snapshot 36 B; response array 256 B; caller principal 40 B
Compiled local frames	Settings handler 416 B, snapshot getter 32 B, registration adapter 48 B, excluding callees
Authored HTML/app arrays	10,432 / 32,061 B including terminators in target ELF; flash-resident, not module heap
Tasks/stacks/queues	No new task, stack-size change, queue, depth, mutex or timer
Sockets/capacity	Six HTTPS sockets, two serial + one admin WebSocket; all session/ticket/shared-console capacities unchanged
Browser resources	Eight fixed value nodes, one active refresh; snapshot body <=256 B, session body <=512 B; existing terminal bounds unchanged

Sizes/entry frames above were inspected with the target GDB against the final ELF. The URI table allocation was verified in installed httpd_main.c and descriptor/free ownership in httpd_uri.c. Ordinary malloc/calloc follow existing IDF allocator policy, with no new PSRAM-only requirement or fallback policy; conservatively budget all 49 requested bytes as possible internal heap. No module persistent heap or PSRAM payload was added beyond HTTPD's route ownership. TLS/request allocations, allocator overhead, actual internal versus PSRAM placement, browser engine storage and full call-chain stack usage remain unmeasured. Static RAM and local frames are not runtime reserve evidence.

Earlier pre-review build passed in 25.14 seconds at 95,580 B RAM / 1,654,021 B flash. The final build above supersedes it and includes the reconnect and allocation-publication fixes.

Verification

All commands below passed in this task; compiler/test subprocesses and firmware builds had finite tool timeouts. No sanitizer run or hardware operation was performed.

Command	Result / Scope
python3 tests/web_cookie_auth/run.py --settings	Existing store/auth suite + 5 Settings groups: real auth/store/parser/adapter, exact extracted production handler/helper/snapshot/enum bodies, allocation failure and installed unregister; fake UART mutex/database/HTTP IO
python3 tests/web_ui_session/run.py	21 Node groups, production-C renderer/headers/failures, exact loader CSP; includes 10 Settings cycles with retained sockets/IDs, hidden output/input, malformed/oversized/errors/timeout/manual retry, cancellation/restore/identity and concurrent serial reconnect
python3 tests/web_admin_transport/server_lifecycle.py	12 groups, required/optional registration failure and stop/restart ownership; Settings allocation failure does not disable either transport
python3 tests/web_cookie_auth/run.py --admin	Store/auth plus real admin ticket-to-upgrade integration, isolation, currentness and expiry
python3 tests/web_admin_transport/run.py --tickets	25 transport / 12 ticket groups
python3 tests/web_session_store/run.py --serial	Store and serial/session integration
python3 tests/web_login_ui/run.py	Production renderer/CSP plus 8 Node groups
python3 tests/web_auth_parse/run.py	268 cases, 0 failures
python3 tests/admin_console_boundary/run.py	Shared console/currentness, deferred certificate handoff and production SSH adapter regressions
python3 tests/admin_console_boundary/lifecycle.py	Canonical lifecycle and unchanged SSH/UART0 paths
python3 tests/admin_console_boundary/accounts.py	Existing account operation-admission/persistence/isolation regressions
python3 tests/admin_ssh_policy/run.py	Existing parsed restrictions and role/transport policy
pio run	Final build above, PASS
git diff --check	PASS

The initial Settings test incorrectly reused a session after testing authoritative stale-principal invalidation; the real store correctly retired it. Fixtures now mint independent sessions for stale, database-failure and success cases. This was a test error, not an authentication-policy change.

todowrite and a task/subagent tool were unavailable in this session. Progress was tracked in commentary and this record. A separate implementer review pass found/fixed the reconnect-supersession and IDF publication issues, followed by affected suite reruns and the final build. Independent agent review was not available and is not claimed. No remaining actionable finding was identified in the implementer's final review; that is not an independent review result.

Independent Review Completion

Parent obtained a separate agent review after implementation. No actionable findings were identified in authorization/secrecy/bounds, nonblocking snapshot locking, navigation/session/expiry races, or the optional registration adapter's ownership against installed IDF 5.5.0. Reviewer reran Settings/auth (5 Settings groups), UI (21 groups plus renderer/CSP), and server lifecycle (12 groups), all passing. This supersedes the implementer-session independent-review limitation above. No production edits or additional build were needed for review. Real browser/HTTPD scheduling, hardware and runtime reserve limits remain unverified.

Pending Target Checklist

• Record flashed revision/browser(s)/direct-IP or mDNS origin. On desktop and mobile, verify layout, scrolling, keyboard/touch navigation, focus/pressed states, visibility of writer controls and CSP console cleanliness.
• Compare all eight Serial fields against UART0 serial status at a stable configuration, while stopped and running; make working-only changes via existing UART0/SSH admin controls, then Refresh. Confirm Settings never starts/stops/applies/saves/loads/resets serial or changes persisted values.
• With live serial writer and open admin, switch Serial/Admin/Settings at least 10 times and Refresh. Confirm same broker client/writer IDs, continuous serial/admin output, no input while Settings is selected, retained admin prompt/history and bounded overflow notices. Confirm explicit Disconnect and Close admin still affect only their intended route.
• Exercise normal-user direct endpoint denial; unknown query/body/method rejection; expired/logout/revoked session and account switch; delayed response/cancel, bfcache restore and explicit Refresh; unavailable/busy serial and optional route allocation failure where injectable. Confirm no old settings/terminal identity is exposed to a replacement login.
• Run UART0 recovery, native USB UART1, user/admin SSH and both browser routes concurrently, at the supported workload used for M2 (user reported up to 230400 baud). Include Settings reads while console reconfiguration is active. Record any existing admission failure by stage without asserting it is fixed or caused by this phase.
• Record settled boot, connected/full-mix and post-cycle/cleanup memory free/minimum/largest internal/DMA/PSRAM values, HTTPD and relevant task stack margins where instrumented, duration/byte/drop counts and any allocation failure. Runtime floors/budgets remain pending user approval; do not infer safety from linked RAM.

Stop at 8D.8 for target validation/user decision. 8D.9 is not started and requires a separate request after that decision. Prior M2 signoff remains accepted regardless of these new-phase pending checks.