Add Bounded Ordinary HTTPS Idle Cleanup

docs/agent/architecture.md

@@ -112,6 +112,8 @@ A WebSocket connection requires a one-time, principal-bound ticket with a maximu
 
 `web_diagnostics` observes public synchronous HTTPS create/close callbacks without replacing socket/transport cleanup, and wraps only the four serial/admin ticket/upgrade handlers. Six always-maintained post-TLS metadata records supply a console-safe occupancy snapshot without querying HTTPD off-owner; an opt-in 32-entry numeric ring adds timing/heap/HTTPD stack samples. Firmware-lifetime connection sequences and capture epochs survive clear/restart and fence fd reuse/toggle races. No tasks/probes/event subscriptions; no authentication/request data retained. UART0/admin SSH commands never wait for HTTPD; browser policy remains unchanged. This is successful-TLS occupancy, not preaccept or failed/in-progress handshake instrumentation; exact limits/overhead in `docs/phase8d11_implementation.md`.
 
+Ordinary HTTPS idle retention is independently enforced by `web_httpd_idle`: one persistent one-second ESP timer, at most one generation-qualified HTTPD work reservation and six owner-only rows. The private adapter observes IDF's all-route successful `req_new`/`req_delete` completion marker, checks actual WS/async flags and pending/readable input, and shuts down only the current expired ordinary fd after 15 seconds of observed idle. TLS-create resets reused-fd observations before diagnostic publication. No TLS cleanup override, LRU eviction, in-progress request interruption or diagnostics dependency. Stop fences submissions before destroying HTTPD; only successful stop retires queued state. Slow owner work and accepted-but-lost nonblocking UDP work preclude a hard wall-clock guarantee; loss stays bounded until successful restart. Timer preparation failure gates HTTPS start. SDK audit, tests and exact limits: `docs/https_idle_cleanup.md`.
+
 Web serial initialization is failure-isolated from the base HTTPS service: if the transport cannot initialize, `web_server_init()` can still succeed and serve authenticated non-WebSocket routes.
 
 `web_ui.c` contains authored index/application strings and response policy; it validates `/api/session` before connect/restore, adds explicit Sign out, and cancels stale work on 401/logout/page exit. `web_login_ui` is a standalone public page without protected-asset dependencies. Both authentication documents and app script are no-store. Its restrictive CSP contains a hard-coded hash of the inline loader, so those two must change atomically; preserve same-origin connections, no-referrer behavior, frame denial, and the existing cache policy. `web_assets_data.c` contains checked-in generated arrays for vendored compressed xterm assets and the logo. Normal builds compile these arrays directly; they do not regenerate assets.

docs/agent/code-map.md

@@ -63,6 +63,7 @@ This is a semantic map, not a complete file inventory. Start here, then read the
 **Responsibility:** serve authenticated HTTPS UI/API, issue WebSocket tickets, and adapt browser serial sessions to broker clients.
 
 - Files: `src/web_server.{h,c}`, `src/web_serial_transport.{h,c}`, `src/web_ui.{h,c}`, `src/web_console.{h,c}`
+- Ordinary HTTPS idle cleanup: `src/web_httpd_idle.{c,h}`, owner sweep in `web_httpd_adapter.{c,h}`, lifecycle/TLS composition in `web_server.c`; `tests/web_httpd_idle/run.py`. Independent of diagnostics/optional transports: 15-second observed idle, one-second timer/one queued probe, six rows, actual WS/async/pending exemptions, safe current-owner shutdown and stop/restart fencing. No LRU/socket/timeout/stack increase. SDK queue/owner-delay limits and target checklist: `docs/https_idle_cleanup.md`.
 - Opt-in admission diagnostics: `src/web_diagnostics.{c,h}`, `tests/web_diagnostics/run.py`. Public synchronous HTTPS create/close callbacks publish six post-TLS connection records; four ticket/upgrade wrappers feed a 32-entry numeric ring. UART0/admin SSH `web diagnostics enable|disable|show|clear`; no queue/task/cleanup override or capacity change. Full bounds, SDK semantics and preaccept/TLS blind spots: `docs/phase8d11_implementation.md`.
 - Security files: `src/web_security.{h,c}`, `src/web_cookie_auth.{h,c}`, `src/web_session_store.{h,c}`, `src/web_auth_parse.{h,c}`. Private IDF boundary: `src/web_httpd_adapter.{h,c}`.
 - Asset files: authored/generated boundary in `src/web_assets_data.{h,c}`, `web_assets/SOURCES.md`, `web_assets/generate_embedded_assets.py`

docs/agent/current-state.md

@@ -4,6 +4,8 @@ This file is working memory. Update it during active work and before handoff; do
 
 ## Development state
 
+- **User-authorized bounded ordinary HTTPS idle cleanup complete (2026-09-08), host-tested/build-verified; target pending:** Preserved existing 8D.11 key/diagnostic implementation (initial Git status was clean). User capture: post-TLS occupancy 6/6, ordinary4/serial2/admin0, ordinary connection ages 50–74 s; admin ticket returned in 14 ms, no TLS/upgrade observed and ticket unconsumed. Installed IDF stops selecting listener at capacity with LRU disabled. `web_httpd_idle.{c,h}` plus sole-private-boundary sweep in `web_httpd_adapter`: six owner-only rows, 15-second observed-idle window (three five-second status polls), one-second ESP timer/at most one queued owner probe, current fd shutdown (no reusable-pointer queued close), all-route SDK request-plus-purge completion marker and input readiness/WS/async checks, TLS-create fd reset, submit fence/nonwrapping restart generations. Diagnostics does not gate policy; socket/timeout/LRU/task/stack/asset settings unchanged. Implementation `pio run` PASS 57.55 s, final confirmation PASS 3.13 s; **99,316 B RAM / 1,708,981 B flash (+160/+1,384 versus diagnostic baseline)**. Rows144 B/module static167 B before placement; timer32 B internal heap before overhead; sweep local frame80 B, runtime margins unmeasured. Final idle **18 + SDK guards**, server **18**, diagnostics **12+1** PASS; cookie accounts/admin/settings/serial-settings, admin transport25/tickets12, store/serial, UI68+CSP and canonical console/account/key/lifecycle/policy suites PASS; diff check PASS. Cookie test-double missing `<stdint.h>` fixed and all variants rerun. Exact contracts/audit/tests/resources/target checklist: `docs/https_idle_cleanup.md`. Ordinary ages are connection ages, not proved idle duration. No hard wall-clock/admission guarantee: owner-blocking handshake/parser/send delays cleanup; continuously active ordinary sockets are not evicted. Reported queue errors retry, but accepted-and-lost nonblocking UDP work stays one reserved probe until successful stop/restart (tested), rather than accumulating unsafe delayed work. Failed stop remains detached/owned until retry. No hardware/upload/erase/commit, target acceptance or reserve approval; next is the documented target reproduction/soak, not capacity changes.
+
 - **Authorized 8D.11 admission diagnostic slice complete (2026-09-08), host-tested/build-verified; target pending:** `web_diagnostics.{c,h}` uses public synchronous HTTPS create/close callbacks and four ticket/upgrade wrappers, six fixed metadata slots and a 32-record opt-in numeric ring. Console `web diagnostics enable|disable|show|clear` via UART0/admin SSH; browser policy unchanged. No close/open override, queue probe/task, capacity/timeout/log-level/assets/SDK change. Post-TLS owner-published occupancy only; TLS failures/in-progress handshakes and preaccept backlog remain invisible. Final `pio run` PASS **10.30 s, 99,156 B RAM / 1,707,597 B flash**, **+3,080/+3,912 B** versus recorded key-slice baseline. Ring/table target symbols 2,816/192 B; diagnostic handler/record/TLS callback/show local frames 144/144/160/528 B excluding callees, runtime margins unmeasured. PASS diagnostic **12+1**, server lifecycle **16**, canonical console lifecycle/policy/boundary, canonical account/key transactions, cookie/admin and cookie/accounts, admin transport **25**/tickets **12**, browser **68 plus renderer/CSP**, store/serial and diff check. Exact files/commands/contracts and reproduction checklist in `docs/phase8d11_implementation.md`. Existing key work preserved; no device/commit/upload/erase, timeout diagnosis, target acceptance or reserve approval claimed. Next is user target capture, not capacity changes or 8D.12.
 
 - **8D.11 partial target evidence (2026-09-08):** User supplied 60-second fresh-boot/full-mix telemetry; no functional/sign-off claim. Full mix eventually established (two SSH public-key sessions, SSH writer + USB/two web observers, both admins), but last browser admission again required retries. Three TLS errors `-0x0050` verified against installed mbedTLS as NET_CONN_RESET, not allocation failure; cause unknown. Loaded internal/DMA free 32,596/24,840 B, lifetime minima 5,468/532 B, largest 22,528 B; per-region minima are conservative/non-simultaneous. Settled free is similar to prior 8D.10, minima much lower; no causal regression attribution. SSH stack minimum-free 18,468 boot/16,276 loaded B. Web send failure/close 1, expired tickets 3; SSH handshake/auth failures zero; no broker drop counters. Exact memory/counters/evidence limits in `docs/phase8d11_implementation.md`. Previously accepted admission issue recurred, not fixed; investigate timing/socket occupancy/allocation correlation before capacity changes. Key algorithms/new UI provenance, mutation checklist, HTTPD/dispatcher margins, cleanup/soak and target sign-off remain pending. Documentation-only update; no source/build/device action.

docs/agent/design-decisions.md

@@ -128,6 +128,8 @@ The following first-slice exclusions are historical and superseded by slice 2:
 
 **HTTPD boundary:** `web_httpd_adapter` alone includes private ESP-IDF 5.5.0 structures. Public request getters expose only the first field, so the adapter validates bounded parsed headers/rejects duplicates. `/ws/serial` is an ordinary GET until authenticated transport admission explicitly sends 101 and installs the frame handler; automatic HTTPD upgrades happen before URI handlers. Preserve right-aligned unread pending bytes when wiping request memory. Two Set-Cookie calls append pointer-backed fields, whose distinct buffers must survive through send. HTTPD logs above ERROR are compiled out to prevent header/ticket exposure.
 
+**Ordinary HTTPS idle lifecycle:** Keep six sockets and LRU disabled. `web_httpd_idle` queues at most one owner sweep each second; `web_httpd_adapter` alone reads the installed successful request-plus-purge completion marker. Fifteen seconds of observed ordinary idle (three normal five-second browser polls), current SDK WS/async exemption, pending/readable-input checks and synchronous TLS-create fd invalidation authorize current-owner `shutdown`, never queued `sock_db *` close. Do not use response events, diagnostic wrappers or connection age as the completion/idle boundary. All ordinary response work must finish synchronously or retain the SDK async exemption. Submission fencing precedes HTTPD stop; only successful stop retires queued state and admits a nonreused generation. No tracing dependency, forced per-response close, hard request deadline, arbitrary admission eviction or capacity increase. A reported queue failure retries; accepted-but-lost UDP work stays reserved until successful restart rather than accumulating potentially delayed probes. Exact safety/liveness limits and target checklist: `docs/https_idle_cleanup.md`.
+
 **Consequence:** The version guard is not a source-hash guarantee. Re-audit layout, scratch/pending ownership, logging and handshake/frame dispatch on SDK changes; do not scatter private accesses through application code or assume host doubles prove real socket behavior. No SDK patch is currently applied. See `docs/phase8d3_implementation.md` for verification and target gates.
 
 **Relevant files:** `src/web_cookie_auth.{c,h}`, `src/web_session_store.{c,h}`, `src/web_httpd_adapter.{c,h}`, `src/web_server.c`, `src/web_serial_transport.c`, `src/CMakeLists.txt`.