ESP32_Serial_Swiss_Army_Knife/docs/phase8d7_implementation.md

Phase 8D.7 — Web-shell lifecycle parity and M2 acceptance

Current status: implemented scope validated

8D.7 validated by explicit user sign-off on 2026-09-07 for the implemented scope. The user explicitly requested: "Ok, mark 8D.7 as validated." This supersedes target-pending and acceptance-blocking statements below for the implemented stop/reboot, certificate and other-account slices. It does not assert full browser command parity or M2 acceptance.

User sign-off and evidence (2026-09-07)

• User reports thorough testing with no further defects except intermittent admission failures in the supported two serial WebSockets plus one admin WebSocket arrangement. These have recently not recurred and are accepted as nonblocking for this sign-off, not diagnosed or fixed. The earlier admission investigation and telemetry below remain historical evidence and a follow-up if the issue recurs.
• User verified certificate rotation and web start/stop, with lifecycle testing through UART0, SSH admin and web admin. After stopping web from the browser, restart was through another administration route; this is not a claim that browser web start is enabled.
• User reports full-client-mix operation with no broker drops at rates up to 230400 baud, with the external adapter baud corrected. This is user-reported workload evidence, not a universal zero-drop guarantee or a new measurement inferred from the earlier boot sample.
• Exact flashed revision, repetition counts, soak duration, reboot-specific results and individual account-mutation/injection checklist results were not separately supplied. Unrecorded details remain evidence limitations and regression coverage, not claims of execution or blockers reopening this user-approved validation.

Deferred scope: browser self-target/generated-password/key/legacy-credential and other owner-specific parity restrictions remain in force until separately implemented; bootstrap/recovery remain UART0-only. Numeric reserves and unmeasured stack margins remain open. M2 acceptance is not recorded or implied. No new implementation is authorized; wait for a separate request. This sign-off changes documentation only, with no production/test edits, build, upload or commit.

Third bounded account slice history

The following slice reviews, investigation and checklists preserve evidence as recorded before sign-off. Their target-pending, acceptance-blocking and next-slice instructions are superseded by the current status above; checklist items are not individually promoted to passed.

Account slice reviewed / host-tested / build-verified (2026-09-07); no actionable findings remain. Target validation and 8D.7/M2 acceptance remain pending. This supersedes the account restrictions and next-slice statements in the historical sections below, not their recorded validation limits.

Browser admins may now operate on other accounts with exact parsed user add <name> user|admin, user password <name>, user delete <name> --force, and user role <name> user|admin --force. Add/password use hidden password and confirmation prompts. Dispatcher policy and the canonical handler both enforce this bounded allowlist. Self-target mutations (even no-op role changes), generated-password output, key workflows, bootstrap and recovery remain blocked for browsers. Existing inspection is unchanged. Database final-admin and migrated-account protections remain authoritative.

Operation-admission currentness

The guarantee is operation-admission currentness, not currentness at commit. Account, originating owner/session and console token are checked after password prompts and before admitting the database API operation; delete/role also revalidate before API admission. Queued or subsequent stale operations must reject. Disconnect, logout, expiry or revocation during password derivation does not cancel or roll back an already admitted operation. Such an operation may persist and request target-only web/SSH revocation after the initiating browser is gone. Notification failure does not undo a committed mutation, and completion output is not a receipt guarantee.

After a disconnect with an uncertain result, reconnect using a current administrator and inspect user list / user show <name> before retrying. Password values cannot be inspected; verify the intended login outcome or deliberately reset the target password rather than assuming cancellation. Do not expose credentials in status or logs.

Review and verification

Independent production and regression reviews found no actionable defects. The earlier proposed precommit-currentness finding was withdrawn as inconsistent with the existing executing-handler contract. No production changes were needed during this review continuation; inherited changes were preserved.

Passed in this continuation: python3 tests/admin_console_boundary/accounts.py, python3 tests/admin_console_boundary/run.py, python3 tests/admin_ssh_policy/run.py (review agent); python3 tests/admin_console_boundary/lifecycle.py, python3 tests/web_admin_transport/run.py --tickets (25 transport / 12 ticket groups), and python3 tests/web_session_store/run.py --serial (parent). Account regressions cover admitted derivation invalidation, successful commits and target-only notification arguments, NVS open/write/commit failures, candidate cleanup, and rejection of the next stale operation.

These are deterministic handler/database and separate console/transport boundary tests, not real HTTPD disconnect/expiry, scheduler or end-to-end socket tests. Account tests double currentness, prompts, crypto, NVS and notification APIs; they do not independently prove live connection isolation. No device or sanitizer validation was performed.

pio run passed (incremental, 3.35 s): 95,580 B linked RAM / 1,648,577 B flash, 0 / +516 B versus the recorded certificate slice, 0 / +2,088 B versus 8D.6, +1,048 / +48,604 B versus 8D.0. No new task, queue depth, route, socket capacity, assets or stack size; runtime reserves and stack margins remain unmeasured.

User target evidence: admission issue (2026-09-07)

User supplied a 60-second fresh-boot sample, then reported intermittent full-client-mix admission failures. Exact flashed revision, browser/tab arrangement, failed-request timing and loaded duration were not supplied. This is target evidence, not full-mix acceptance or proof of an account-slice regression.

Heap (bytes)	Boot free / minimum / largest	Eventually connected mix free / minimum / largest
Internal 8-bit	70,860 / 59,548 / 31,744	33,428 / 5,280 / 23,552
Internal DMA	63,104 / 51,792 / 31,744	25,672 / 344 / 23,552
PSRAM	8,246,176 / 8,240,232 / 8,126,464	8,084,100 / 8,058,464 / 7,995,392

Boot: HTTPS/SSH running without reported startup errors, mDNS announced, four accounts/two admins, no cookie sessions or web/SSH clients, UART stopped at configured 230400 baud, broker empty, USB attached with host closed. SSH minimum-free stack 18,472 B. Admin backend initialized/attached, 167 B static / 240 B ticket storage / 1,552 B PSRAM payload. Supplied boot request/connection/failure counters were zero.

User could promptly connect SSH writer/admin, USB observer and browser serial; two web observers were possible, but adding browser admin repeatedly timed out. Disconnecting one observer allowed admin admission; the last observer eventually reconnected. UART logs include repeated TLS handshakes and errors -0x0050 (also on write) and -0x7280. No loaded counters or failed-allocation trace supplied. DMA's 344 B lifetime minimum is concerning, but capability heaps overlap and regional minima are conservative sums, not proof of an allocation failure or simultaneous remaining reserve.

Installed IDF 5.5.0 / Mbed TLS 3.6.3 source identifies these errors as connection reset and connection EOF, respectively, not allocation errors. Source investigation found six shared HTTPD client slots with LRU disabled (src/web_server.c); IDF stops selecting the listener when full. Ordinary idle HTTP connections can retain slots: the one-second receive wait is not an idle lifetime. TLS handshakes run synchronously on HTTPD with a configured five-second handshake timeout. Browser assets, session/status/ticket requests and WebSockets compete for slots. TLS already uses external-memory allocation; increasing sockets or enabling LRU blindly would risk memory pressure or retained serial sessions.

Investigation remains open: transient HTTPD slot occupancy is the leading hypothesis, with handshake blocking, internal-memory pressure and global descriptor capacity possible contributors. Disconnecting an observer frees both a socket and resources, so it does not isolate the cause. Next evidence: identify ticket POST versus WebSocket-open timeout using redacted browser timings/status; correlate count-only HTTPD ordinary/WS occupancy and allocation failures; repeat with serial traffic paused. Do not share cookies, CSRF values, ticket URLs or unredacted HAR. No runtime fix, device operation or build performed for this investigation; full-mix reliability must be resolved before acceptance.

Pending account target checks

1. Exercise each enabled other-account command, hidden confirmation/mismatch/cancel, and self/generated/key/bootstrap/recovery rejection. Verify final-admin protections and unchanged UART0/admin SSH behavior.
2. Confirm successful target mutations revoke only that account's web/SSH sessions, retaining unrelated browser serial/admin, USB and SSH traffic. Check stale queued commands and prompts after logout/expiry/revocation and slot reuse.
3. Where safely injectable, disconnect during admitted derivation and inspect the resulting account state after reconnect; do not expect cancellation. Check persistence failure and retry recovery without logging secrets. Unavailable timing/failure injection remains unexecuted.
4. Run the lifecycle/full-mix/soak/resource checklist below as well. Remaining self/generated/key/legacy-credential and other owner-specific parity work stays blocked until separately implemented; this review does not close M2 or start another slice.

Second bounded certificate slice history

Second slice implemented / host-tested / build-verified; independent reviewer reports no actionable findings. No hardware validation or 8D.7/M2 acceptance. The user explicitly authorized continuing to stack the next bounded slice; this is a continuation decision, not target sign-off. Next comes credential/account handling, then other owner-specific slices. Prior 8D.6 sign-off stands; numeric reserves remain open.

The second slice enables only exact parsed browser web certificate rotate --force. It changes the shared console boundary, WEB owner and canonical web handler (src/admin_ssh_console.{c,h}, src/web_admin_transport.c, src/web_console.c) plus focused tests. No new task, queue depth, route, socket/session/ticket capacity, asset, UI/settings feature or stack size. SSH and UART0 certificate/lifecycle behavior is unchanged. This documentation handoff does not change source/tests or run a build/device operation.

Second-slice execution contract

• Policy uses the same argument parser as the canonical console: exactly four parsed arguments are required; quoted equivalents are accepted, missing --force or extra arguments are not. The browser handler schedules a typed action before certificate side effects, not a command string for later replay.
• The existing request queue uses a command-line/deferred-action union, preserving capacity. The immutable owner's dispatcher_actions mask selects certificate rotation for the existing 12 KiB dispatcher, not the 4 KiB control task. Other existing actions retain control-task execution; a zero mask preserves SSH behavior.
• Control waits up to ten seconds for application-buffer drain, cancels if it does not drain, then delays 200 ms and attempts a nonblocking dispatcher handoff. A full queue fails before mutation. Pending input gating persists through queue residence and execution, not just handoff. This bounds the acknowledgement/drain stage, not browser receipt, dispatcher queue latency, or certificate/NVS/stop/start execution time; another command or prompt can delay the dispatcher.
• Dispatcher revalidates token/principal/originating session, verifies owner and pending state, reserves the slot as executing and rechecks currentness before the owner callback. WEB revalidates again before mutation. Self-detach cannot reuse the executing slot; it is wiped/released after return. Stale/revoked work fails before side effects; these checks cannot roll back an already executing action.
• WEB calls transactional web_security_rotate_certificate() → web_server_stop() → web_server_start(), outside console locks and without socket IO or handler replay. Generation/key generation/persistence failure leaves the previous committed/live material and skips stop/start. Successful commit installs the new identity before stop; later lifecycle failure does not undo it. A stop error returns immediately without start, retaining HTTPD ownership under existing server rules; auth may already be invalidated and transports detached. Start failure requires UART0/admin-SSH recovery.
• Successful restart closes both browser routes, invalidates old sessions/tickets and removes any web writer lease. Reconnect requires verifying/trusting the new certificate and logging in again. No account password or legacy recovery credential is rotated; USB UART1, UART0 and SSH are not stopped. Completion/error output after self-detach is not guaranteed to reach the browser.

Second-slice reported verification and resources

The implementer reports PASS for python3 tests/admin_console_boundary/run.py (including certificate.c), python3 tests/admin_console_boundary/lifecycle.py, python3 tests/admin_ssh_policy/run.py, python3 tests/web_admin_transport/run.py --tickets (25 transport / 12 ticket groups), python3 tests/web_admin_transport/server_lifecycle.py (11 groups), python3 tests/web_cookie_auth/run.py --admin, python3 tests/web_session_store/run.py --serial, and git diff --check. Independent review found no actionable findings. Sanitizer validation is unavailable because libasan/libubsan are missing; no sanitizer pass is claimed. Host harnesses do not prove concurrent RTOS/HTTPD/TLS behavior or target stack margins.

Parent reports final pio run PASS in 26.32 s: 95,580 B linked RAM / 1,648,061 B flash. Deltas: 0 / +1,036 B versus first slice, 0 / +1,572 B versus 8D.6, +1,048 / +48,088 B versus 8D.0. Baselines were not rebuilt. The owner mask and local deferred-request/principal scratch are real accounting considerations despite unchanged linked RAM and queue capacity; host sizeof is not target layout or stack-margin proof. Control/dispatcher path high-water marks remain unknown. No new tasks, queue depths, routes, assets or stacks; existing HTTPD/SSH stacks and transport capacities remain unchanged.

First-slice history (2026-09-06)

The following behavior, verification and resource figures record the first slice, not the second-slice final build.

First increment implemented / host-tested / build-verified; target validation pending. 8D.7 and M2 are not complete. Prior 8D.6 user sign-off stands. The user requested starting 8D.7; scope review splits the several distinct lifecycle/identity/network/account paths as the plan permits.

This increment enables browser-admin reboot and web stop using the existing deferred-control task. Allowed production files are src/admin_ssh_console.{c,h}, src/web_admin_transport.c, src/web_console.c and src/system_console.c; related host tests and documentation are updated. Starting worktree was clean. No new task, queue, socket, route, payload capacity, generated asset, UI/settings feature, upload, erase or commit.

Implemented behavior

• The canonical web stop handler recognizes browser-origin dispatch through admin_ssh_console_dispatch_is_web() and queues ADMIN_CONSOLE_DEFER_WEB_STOP before any service side effect. UART0 and admin SSH keep their existing synchronous HTTPS-stop path.
• Browser reboot now passes parsed command policy and uses the existing canonical remote-reboot deferral. The acknowledgement says console output rather than SSH output. UART0 remains synchronous; SSH reboot remains deferred.
• In the first slice, WEB owner supports SELF_CLOSE, REBOOT and WEB_STOP only. Its control-task callback revalidates originating cookie-session/principal binding and full console identity after the drain delay, outside console locks. It calls lifecycle APIs, not socket IO; HTTPD remains the socket owner. Stop API errors propagate through the existing deferred-result path. Revoked/stale work does not invoke stop/reboot.
• Existing two-entry control queue, ten-second drain wait, 200 ms final delay and five-second output backpressure remain. Drain is a best-effort application-buffer heuristic, not browser receipt confirmation, and the drain bound does not establish a hard upper bound on underlying lifecycle API execution. A failed stop may already have invalidated authentication and require UART0/SSH recovery.
• Console snapshots expose the existing deferred-pending flag. While deferral is observed, HTTPD wipes buffered/new input rather than replaying it after cancellation. The incoming frame's discard decision is latched before bounded payload reception, so cancellation during reception cannot reclassify that frame as a new command. A second frame arriving before the periodic poll does not turn deferred trailing input into a session-closing overflow. Normal invalid/oversized/fragmented frames and nondeferred buffer exhaustion still fail closed. Discard events contribute to existing input-backpressure counts.

Intentional effects: web stop closes both browser-admin and serial WebSockets and invalidates HTTPS sessions. A web writer therefore loses its broker lease; USB, UART0 and SSH are not stopped. Restart HTTPS through UART0/admin SSH using web start, then log in again. reboot restarts the entire device and loses RAM-only configuration. Normal terminal selection and admin-only exit retain their existing serial-isolation behavior.

Restrictions still in force

These remain deliberately blocked for WEB, before canonical handler side effects:

• All web forms except exact parsed web status, web stop and web certificate rotate --force. This still blocks certificate info, start, help/counters, legacy credential display/rotation and full material reset.
• Wi-Fi/mDNS commands except exact status queries.
• User mutations; allowed inspection remains bare user, user status, user list, user show <name>.
• SSH stop/disconnect/reset and host-key mutation (host-key info remains allowed).
• Bootstrap/recovery remain UART0-only for all remotes. No one-time self-generated credential workflow was enabled.

The next increment remains within 8D.7: credential/account handling, then other owner-specific restrictions in bounded slices. The user explicitly authorized stacking the next bounded slice without target sign-off. Preserve explicit rejection until each path is implemented. Do not proceed to settings or close M2 based on either slice.

First-slice executed verification

Command	Result
python3 tests/admin_console_boundary/run.py	Shared console/SSH, dispatcher-origin, pending snapshot, action bounds, queue/drain/cancellation and existing prompt/currentness regressions pass
python3 tests/admin_console_boundary/lifecycle.py	New extracted canonical-handler checks pass: browser stop deferred, SSH/UART stop unchanged, remote reboot and scheduling-failure isolation
python3 tests/admin_ssh_policy/run.py	Actual IDF parser permits only the intended new WEB forms, including quotes; remaining restrictions/SSH policy pass
python3 tests/web_admin_transport/run.py --tickets	23 transport / 12 ticket groups pass, including stale/revoked action rejection, control-task API routing/error propagation, pending input, second frame before poll and cancellation during receive
python3 tests/web_admin_transport/server_lifecycle.py	11 groups pass, including detach timeout, failed stop/retry and optional failure isolation
python3 tests/web_cookie_auth/run.py --admin	Real cookie/store/ticket/adapter/admin endpoint integration passes; lifecycle calls doubled and not invoked by endpoint tests
python3 tests/web_session_store/run.py --serial	Store/serial binding/isolation/race regressions pass
python3 tests/web_ui_session/run.py	Renderer/CSP and 17 UI groups pass; no UI changes
git diff --check	Pass
pio run	Final source build 12.44 s, finite 180-second timeout; 95,580 B linked RAM / 1,647,025 B flash

Independent review identified the second-frame-before-poll cancellation bug, then cancellation-during-receive reclassification. Both were fixed with regressions; reviewer confirmed the final correction with no remaining findings in it. An initial console test still expected web stop denial; updated it to a still-restricted command. These are not target failures. An earlier pre-review-final build was 1,646,965 B flash; the table above supersedes it.

Tests use deterministic dependencies, not real concurrent RTOS/HTTPD/TLS/device execution. Actual command handlers, console state machine, transport, and server orchestration are exercised in focused harnesses, not one fully linked concurrent end-to-end firmware harness. No sanitizer pass or on-device stop/reboot is claimed.

First-slice resources

Versus signed-off 8D.6 (95,580 / 1,646,489 B): 0 B RAM / +536 B flash. Versus recorded 8D.0 (94,532 / 1,599,973 B): +1,048 / +47,052 B. Baselines were not rebuilt.

No new static payload/state, module heap/PSRAM allocation, task, stack-size, queue-depth, HTTP handler, TLS/socket, ticket or session capacity. The snapshot adds a boolean describing already-existing console state; callers use local snapshot scratch. WEB lifecycle validation copies one secret-free principal on the existing control stack. Existing control stack is 4 KiB, dispatcher 12 KiB; runtime stack high-water marks for these new call paths remain unmeasured. HTTPD remains 10 KiB, SSH owner 20 KiB, web payload 1,552 B PSRAM-only, six HTTPD sockets and two shared console slots.

Carry forward 8D.6 loaded lifetime minima 6,516 B internal / 1,580 B DMA, noting overlapping capabilities and conservative regional sums. Numeric reserves remain open; no safe margin or new reserve violation is inferred. Do not reopen the prior sign-off solely for incomplete numeric instrumentation.

Pending target checklist — both slices

1. User-controlled flash/reload; capture exact revision, 60-second settled memory, web/SSH status and available stack telemetry. Check browser login/serial/admin, USB UART1, user/admin SSH and UART0 at the established 230400-baud workload.
2. With browser serial + admin and USB/user/admin SSH active, issue web stop from browser admin. Expect its best-effort scheduling acknowledgement, then both browser routes close; UART0/USB/SSH remain usable. Confirm no browser writer remains. Restart with UART0/admin SSH web start, reauthenticate and reconnect. Repeat five times and compare full-mix/cleanup heap and largest blocks; do not expect boot equivalence when UART/clients remain active.
3. With any RAM-only settings loss understood, issue browser reboot. Expect best-effort acknowledgement and device restart, then usable UART0 recovery and browser login. Verify no replay of the command on reconnect.
4. Exercise queued/slow output, trailing input and rapid additional input during deferral. No cross-session output, no deferred input replay after cancellation, no unrelated admin-only-close effect on serial. Logout/revoke/disconnect before execution should prevent stale actions. Timing/failure injection not practical on target remains explicitly unexecuted rather than assumed passed.
5. Check remaining forbidden commands still reject without side effects, and SSH/UART0 stop/reboot/certificate behavior remains unchanged. For browser rotation, reject missing force, extra arguments and other certificate forms without identity changes; accept only exact parsed web certificate rotate --force (including equivalent quoting).
6. With browser serial/admin plus USB and user/admin SSH active, rotate the certificate. Expect only a best-effort scheduling acknowledgement, both browser routes closing, old session/ticket rejection and loss of a web writer lease. Verify the changed public fingerprint through trusted UART0/admin SSH web certificate info, update browser trust deliberately, log in again with unchanged account credentials and reconnect. Confirm USB UART1/SSH traffic and UART0 recovery remain available; do not log private keys, passwords or session/ticket material.
7. Repeat rotation/relogin under the established full mix and compare loaded/cleanup memory and largest blocks. Exercise revocation/disconnect/slot reuse before dispatcher execution, queued prompts/slow output, trailing input and no replay. Collect control/dispatcher/HTTPD stack high-water evidence where instrumentation permits; record unknown margins rather than substituting host sizes.
8. Where safely injectable, verify generation/persistence failure skips stop/start and retains old material; stop/detach failure after commit retains ownership, skips start and may leave auth disabled; successful stop followed by start failure retains the new persisted identity. Recover through UART0/admin SSH by inspecting state and completing stop/start as appropriate, with no duplicate HTTPD start or blind rotation retry. Unavailable target failure injection remains unexecuted.

Do not run these disruptive commands automatically. The user has explicitly authorized stacking the next bounded runtime slice; no hardware checklist item is thereby passed. Final M2 acceptance remains pending and requires lifecycle parity, the full mixed-client/soak/resource checkpoint and explicit target/browser acceptance before settings.