Phase 8D.11 — SSH authorized keys

Status (2026-09-08)

Implemented on user request; host-tested and firmware-build verified. Target validation/sign-off remains pending. Accepted 8D.8–8D.10 and M2 remain accepted; this is not M3 completion or authorization for 8D.12.

User-supplied target telemetry (2026-09-08)

User supplies a fresh-boot sample at 60 seconds and a later full-client-mix sample. The last browser serial/admin connection again required several attempts. This records partial target evidence, not functional key-management acceptance or phase sign-off; exact flashed revision, browser, key algorithms, traffic duration and cleanup/soak results were not supplied.

Memory (bytes)	Boot free / minimum / largest	Loaded free / minimum / largest
Internal 8-bit	69,584 / 58,076 / 31,744	32,596 / 5,468 / 22,528
Internal DMA	61,828 / 50,320 / 31,744	24,840 / 532 / 22,528
PSRAM	8,246,356 / 8,242,108 / 8,126,464	8,087,636 / 8,056,956 / 7,995,392

Minima are conservative sums of per-region lifetime minima, not a simultaneous sample; internal/DMA capabilities overlap and must not be added. Low minima are a transient-pressure warning, not proof of allocation failure. Settled loaded free internal/DMA is comparable to the prior accepted 8D.10 sample (32,556 / 24,800), but lifetime minima are substantially lower (previously 19,228 / 11,472). Different connection attempts/workloads prevent attributing that difference to 8D.11.

Boot: SSH/HTTPS started once without reported startup failures; all supplied network request/session counters zero; no broker clients, UART stopped, USB attached but host closed. mDNS announced with ESP_OK. Five accounts/two admins, no cookie sessions. SSH owner stack 20,480 bytes, minimum-free 18,468.

Loaded: two successful SSH public-key sessions (admin console and user serial writer); two browser serial observers, USB observer and active browser admin. Broker writer 16, web observers 17/19, USB observer 10, all pending/events zero at the sample. UART running at 230400 8N1 RTS/CTS, DTR active, RX/TX queues empty; USB host open. Two cookie sessions; mDNS/SSH/HTTPS report operational, no transition/error. SSH minimum-free stack 16,276 bytes; HTTPD/dispatcher stack margins unreported. Public-key authentication works for these two sessions, but algorithms and whether keys were imported through the new UI are unspecified.

Admission evidence: HTTPS handshake attempts logged from 197688 through 379338 ms, with three errors at 377948/378298/378658 ms: mbedtls_ssl_handshake returned -0x0050, TLS session creation and HTTPD accept failure. Installed mbedTLS net_sockets.h defines this as MBEDTLS_ERR_NET_CONN_RESET, not allocation failure. It does not establish why the connection reset or whether transient heap pressure caused delays. The previously accepted intermittent last-browser admission issue has recurred and remains unresolved.

Loaded counters: SSH TCP connections/handshake success/auth attempts 2/2/2; handshake/auth/timeouts/capacity/IO failures zero. SSH broker connect 1, writer request/grant 1/1, revocations 1; admin admission 1/no failures, stream RX/accepted/rejected 7/7/0, TX 5,977, session revocations zero. Browser admin tickets issued/consumed 2/2, connected/disconnected 2/1, TX 272; rejection/capacity/protocol/backpressure/send/queue counters zero. Web requests 89 authenticated/zero auth failures, root 2/status 76/tickets 8/assets 2, response errors zero; tickets issued/consumed/expired 8/5/3. Serial WS connect/disconnect 5/3 with zero admission/service/broker failures; RX zero; TX 218 binary frames/21,200 bytes and 16 control frames/1,322 bytes; writer requests/grants/denials 5/1/4. One send failure/close, zero queue/protocol failures. Cookie login attempts 2, invalid/throttled/capacity/CSRF/logout zero. No broker drop-counter output supplied; zero pending queues is not proof of lossless traffic. Application counters do not account for every pre-HTTP TLS failure.

Documentation-only evidence update: no new build, source/config edits, upload, erase, device commands or sign-off. Next diagnostic evidence should correlate connection attempts with heap/allocation failures and browser network timing/socket occupancy rather than assume heap exhaustion or increase socket capacity blindly. Key-workflow checklist and reserve approval remain open.

Authorized admission diagnostic slice (2026-09-08)

Separately user-authorized instrumentation for the recurring third-webshell timeout, not a timeout fix or a key-workflow sign-off. Implemented in src/web_diagnostics.{c,h}, wired through web_server.c, web_console.c, console_completion.c and src/CMakeLists.txt. No generated assets, SDK patch, broad HTTPD debug logging, broker/serial change, new task/timer/queue, socket/URI capacity change, timeout change, LRU change or TLS cleanup override.

Usage and interpretation

Use UART0 (preferred during network stalls) or authenticated admin SSH:

web diagnostics clear
web diagnostics enable
web diagnostics show

Enable before reproducing the failed third connection. At failure, run web diagnostics show promptly, alongside existing memory/web counters/browser Network timing evidence; repeat the snapshot if admission remains stalled. Then web diagnostics disable freezes event retention (live occupancy continues updating); web diagnostics show prints retained history, and web diagnostics clear erases history/counters without changing live sockets or enable state. The setting is RAM-only and defaults off after boot. Existing browser-shell policy deliberately denies these commands, even though shared completion offers the fixed forms. There is no HTTP diagnostic endpoint.

Six connection records are maintained from boot even with capture disabled, so enabling on an already loaded server does not mislabel existing sockets. Each successful TLS connection gets a monotonically increasing, non-wrapping, firmware-lifetime 64-bit conn sequence independent of fd, cookie, ticket, broker or user identity. Clear, disable and HTTPS restart do not reset it. Internal TLS object identity is used only during synchronous cleanup; no pointer is exported or dereferenced by the console.
Snapshot output gives its boot-relative snapshot_us, post-TLS occupancy split ordinary/serial-WS/admin-WS and each live fd/connection sequence/open time/age. kind=0/1/2 means ordinary/serial/admin. Ordinary includes every successful TLS socket not yet observed as upgraded, including idle keep-alives, assets/login/settings and sockets not yet used for HTTP. Classification is actual public httpd_ws_get_fd_info() state after upgrade-handler return, not inferred from ESP_OK. No URI string is inspected.
The fixed 32-entry overwrite ring records successful TLS open, TLS transport-context close, and enter/result for serial/admin ticket and upgrade handlers. t_us is boot-relative observation time; result dt_us measures the underlying handler only (including its work/IO, excluding entry resource sampling); close dt_us is successful-TLS connection lifetime. Open/enter durations are zero. Open/result occupancy includes the connection; close occupancy excludes it. Event IDs survive clear; overwritten counts evicted retained records since clear. unmatched counts duplicate create/unmatched close; lost counts untrackable creates (getter failure, metadata/sequence exhaustion). Nonzero anomaly counters mean occupancy is not trustworthy as complete evidence. Counters saturate.
rc is the exact handler return, not HTTP status or ticket issuance outcome: sending a 401/403/503 can return ESP_OK. Match browser HTTP status and existing rejection/issuance counters; there is no ticket-value correlation, request ID, authentication identity, header/body/query logging, or allocation-failure attribution. Upgrade success is visible in the occupancy classification. Ring overwrite or clear/toggle during an operation can leave unpaired enter/result records; do not invent a duration for a missing result.
Each retained event samples free/largest bytes for internal 8-bit, internal DMA and PSRAM 8-bit, plus the current HTTPD task's minimum-free stack in ESP-IDF bytes. Capability scans run outside the diagnostic lock. Samples are sequential, not an atomic heap snapshot; overlapping internal/DMA pools must not be added. Stack watermark includes diagnostic call overhead and is not dispatcher margin. show does not query a live task handle: resources are historical event samples, not fresh heap values at show time. Use memory for current system heap.
Console snapshots copy only local diagnostic metadata under a short portMUX, never inspect HTTPD session internals or wait for its owner. Printing occurs outside the lock. At most 32 ID-qualified rows and six live records are printed, even during churn; concurrent clear/overwrite is reported as “no longer retained”. Capture epochs reject samples crossing enable/disable/clear, and sequence checks prevent a stale upgrade result from reclassifying a reused fd. No queued diagnostic work exists, hence no outstanding probe or stale queue lifetime to retire on restart. Minimal connection bookkeeping and upgrade classification remain active while disabled; heap scans and event recording do not. Instrumentation still has CPU/static RAM cost, not zero perturbation.

SDK audit and explicit blind spots

Audited installed PlatformIO ESP-IDF 5.5.0, components/esp_https_server/src/https_server.c (httpd_ssl_open, httpd_ssl_close), components/esp_http_server/src/httpd_sess.c (httpd_sess_delete) and httpd_main.c. HTTPS performs synchronous TLS creation before the configured open_fn and user_cb create callback. It installs a transport-context destructor; that destructor invokes the public close callback before deleting TLS/freeing the context. HTTPD's default close closes the fd, then clears contexts, then frees its session slot. The diagnostic close therefore marks a cleanup observation, not a FIN timestamp or causal close reason. It uses the stored fd, not a getter on an already-closed socket. Neither open_fn nor close_fn is replaced, preserving all existing HTTPS cleanup ownership and failure behavior. Callbacks execute synchronously under the existing HTTPD lifecycle; successful stop finishes cleanup before restart, while failed/partial stop retains remaining live metadata. No asynchronous fd-only events are consumed, avoiding event-delay/fd-reuse ambiguity.

This deliberately bounded first slice does not measure TCP connect/accept/listen backlog, pending clients when IDF stops accepting at capacity, handshake begin/duration/failure, TLS allocation failure or aggregate lwIP socket pressure. The public configured open hook is post-TLS too; adding it would not fix these blind spots. No owner-queued client-list probe is added. Occupancy is an owner-published successful-TLS lower bound, not the complete HTTPD session table while a handshake is in progress. A clean 6/6 snapshot supports established-connection saturation at that instant (three WS + three ordinary is directly distinguishable); fewer than six does not exonerate admission/TLS/global socket pressure. Correlate time with browser evidence and existing secret-free TLS errors; do not call this pre-TLS tracing or claim the root cause is proven. Existing capacities, receive/send/handshake timeouts and accepted admission issue remain unchanged.

Diagnostic validation and target follow-up

Focused host harness compiles the entire production module plus the four actual server wrappers with deterministic public-API fakes. Twelve groups cover disabled bookkeeping, routes/return preservation/actual WS state, six-slot saturation, duplicate/stale/reused fd, clear/toggle publication races, full ring/exact overwrite, formatted-output secrecy, bounded show/clear interleaving, stale upgrade completion, partial stop/restart, invalid callbacks/commands and sequence/counter exhaustion; one additional source-guard group checks forbidden APIs and callback wiring. Real TLS/network scheduling, heap/stack values and target stalls are not simulated. Server lifecycle harness additionally checks callback configuration while preserving all 16 existing groups; canonical console lifecycle and policy verify CLI dispatch and unchanged browser restrictions.

Final implementer-run validation (all PASS):

Command	Evidence
`python3 tests/web_diagnostics/run.py`	12 runtime groups + 1 production integration/secrecy guard group
`python3 tests/web_admin_transport/server_lifecycle.py`	16 lifecycle groups; callback configured, budgets/timeouts preserved
`python3 tests/admin_console_boundary/lifecycle.py`	Actual canonical diagnostic dispatch plus existing stop/certificate/reboot cases
`python3 tests/admin_ssh_policy/run.py`	Actual IDF parser; four new diagnostic forms denied to WEB, accepted by SSH
`python3 tests/admin_console_boundary/run.py`	Shared dispatcher, owner/currentness, completion, deferred certificate and SSH adapter regressions
`python3 tests/admin_console_boundary/accounts.py`	Canonical account/key transactions, parser/curve validation, sparse keys and CLI parity
`python3 tests/web_cookie_auth/run.py --admin`	Shared cookie policy and combined admin ticket-to-101 integration
`python3 tests/web_cookie_auth/run.py --accounts`	Shared cookie policy and existing phase8D11 account/key route regressions
`python3 tests/web_admin_transport/run.py --tickets`	25 transport groups + 12 ticket groups
`python3 tests/web_ui_session/run.py`	68 browser behavior groups + production renderer/HTML/CSP checks
`python3 tests/web_session_store/run.py --serial`	Store lifecycle/races and serial/session binding/isolation
`pio run`	Final production source build 10.30 s, 99,156 B RAM / 1,707,597 B flash
`git --no-pager diff --check`	PASS

Firmware delta versus recorded pre-instrumentation phase8D11 96,076 B RAM / 1,703,685 B flash: +3,080 B RAM / +3,912 B flash. xtensa-esp32s3-elf-nm -S .pio/build/esp32-s3-devkitc-1-n16r8/src/web_diagnostics.c.o verifies 2,816 B ring (32 × 88), 192 B live table (6 × 32) and 8 B portMUX; remaining static metadata/alignment contributes to the build delta. xtensa-esp32s3-elf-objdump -d on that same target object gives local entry frames: handler 144 B, record 144 B, TLS callback 160 B, show 528 B, command 32 B. These exclude callees and are not end-to-end stack margin proof. No task/stack-size increase or diagnostic dynamic allocation. Static RAM is paid even while disabled; this matters against the already low observed internal/DMA minima. Capability scans can perturb enabled timing and memory pressure correlation, so compare with disabled runs rather than treating this instrumentation as free. Initial full build also emitted installed SDK FATFS Kconfig boolean-default notes; all production builds succeeded.

Exact changed-file inventory for this diagnostic task (key implementation untouched):

Production: src/web_diagnostics.c, src/web_diagnostics.h, src/web_server.c, src/web_console.c, src/console_completion.c, src/CMakeLists.txt.
Tests: tests/web_diagnostics/run.py, tests/web_diagnostics/fakes.h, tests/web_diagnostics/test.c, tests/web_diagnostics/README.md, tests/web_admin_transport/server_lifecycle.py, tests/admin_console_boundary/lifecycle.py, tests/admin_ssh_policy/run.py.
Docs: docs/phase8d11_implementation.md, docs/agent/current-state.md, docs/agent/code-map.md, docs/agent/architecture.md. No change to durable design decisions was needed beyond documenting the new observation module in the architecture/code map.

Target pending: enable before full-mix reproduction; capture both successful and failed third admission with browser status/timing, ordinary-vs-WS occupancy and memory evidence; check live counts on disconnect/stop/restart, disabled capture and clear; retain UART0/USB recovery and one broker writer throughout. Compare enabled/disabled timing and memory under the same load. No new hardware validation, reserve approval, timeout diagnosis or phase sign-off is claimed.

Scope and contracts

Accounts settings lists selected-account key types, stable slot indices and SHA256 fingerprints; imports OpenSSH Ed25519/P-256 public-key text and explicitly deletes one slot or clears all keys.
user_database.{c,h} provides a zero-wait fingerprint-only snapshot and conditional key mutation wrappers. Username/account ID/auth generation are checked under the canonical database mutation lock. Existing CLI mutation, persistence, duplicate/capacity, curve validation and SSH authentication semantics are retained.
web_account_settings.{c,h} admits strict JSON and uses the existing account-operation slot and administration dispatcher. Successful mutations best-effort revoke only the target's web/SSH sessions; currentness checks remain authoritative. Self changes can invalidate the response/result login: 401/disconnect proves neither success nor cancellation. Inspect after relogin before retrying.
Optional POST /api/settings/accounts/keys accepts {username,user_id,auth_generation} and returns {username,user_id,auth_generation,keys:[{index,type,fingerprint}]}. This read uses POST to retain the existing bounded JSON/Origin/CSRF policy rather than introduce query parsing. Stale/absent selection returns 409; unavailable snapshot returns 503. No public-key blobs, verifier material or private keys are returned.
Existing POST /api/settings/account-operation adds key-add with public_key, key-delete with key_index (0–2), and key-clear. All carry target identity. Request bodies remain 768 bytes/four receives, public-key text at most 384 decoded bytes, decoded SSH blob at most 128 bytes, three keys per account. Envelope/base64/schema rejection occurs at admission; canonical blob/curve validation runs on the dispatcher. Results remain bounded, secret-free, replaceable and session-bound; no mutation auto-retry.
UI confirms mutations, clears pasted text on submission/context changes, uses text-only rendering and fences stale navigation/session/target work. Completion uses existing bounded polling followed by account/key refresh. Stable key slots may be sparse: option values and deletion confirmation resolve by actual index, not array position.
web_server.c registers the listing route independently through the allocation-safe optional POST helper. 24 handlers, six sockets/no LRU; no new tasks, stack-size increases, queue-depth changes or broker lease changes. Larger operation/local structures still have runtime stack costs requiring target measurement.
No private-key upload/export, SSH host-identity management, browser-shell restriction changes, UART0 recovery changes or generated asset regeneration.

Validation

Parent integrated runs passed:

python3 tests/admin_console_boundary/accounts.py: canonical key validation/authorization, Ed25519/P256, malformed/off-curve/truncated keys, zero-wait snapshots, stale ID/generation/recreation, duplicate/capacity, sparse slots, failed persistence and CLI parity. Crypto host adapters use OpenSSL; this is not target SSH validation.
python3 tests/web_cookie_auth/run.py --accounts: shared authentication policy and account/key route admission/execution tests.
python3 tests/web_ui_session/run.py: initial integrated 64 groups and subsequent sparse-slot production fix, plus renderer/HTML/CSP checks. Final test-only extension independently passed 68 browser groups (agent-run), including sparse [1]/[0,2], deletion/automatic refresh, and invalid/duplicate indices.
python3 tests/web_admin_transport/server_lifecycle.py: 16 groups, including isolated optional-route failure and restart recovery (dependencies faked).
pio run: final production build PASS, 14.85 s, 96,076 B RAM / 1,703,685 B flash; +168 B RAM / +9,448 B flash versus recorded final 8D.10. The later four UI test groups change no firmware source.
git diff --check passed for the integrated production change; the final test extension also passed its scoped check.

Backend agent additionally reports passing console boundary run.py and cookie --serial-settings. Independent source review identified the sparse-slot UI assumption; corrected production selection/validation and added dedicated regressions. No other concrete review findings were reported. Host tests are not real network/concurrency/stack-margin proof.

Pending target checklist

Import real Ed25519 and P-256 public keys, compare fingerprints with CLI/ssh-keygen, authenticate with corresponding private keys, and verify existing password authentication remains available.
Exercise maximum input, malformed/private-key text rejection, duplicates and full three-key capacity. Delete slot 0 while later slots survive; list/delete/clear sparse slots and re-add successfully.
Mutate another account through browser and CLI; verify target revocation, unrelated sessions retained, persisted keys after reboot, stale selection rejection and failed-operation recovery.
Exercise own-account key changes: immediate revocation and uncertain lost acknowledgement, relogin/inspect without automatic replay. Confirm last-key removal does not imply password removal.
Run full supported serial/admin mix, preserving one writer and isolated observers; record heap/internal-DMA largest blocks, HTTPD/dispatcher stack margins and cleanup/repeated-operation behavior.
Obtain user target sign-off separately. No upload, erase, hardware test, sanitizer run or commit was performed by this task.

21 KiB Raw Blame History Unescape Escape