ESP32_Serial_Swiss_Army_Knife/docs/web_admission_diagnostics.md

Web admission diagnostics

Current default-disabled instrumentation, not an admission fix or an allocation-failure detector. src/web_diagnostics.{c,h} observes public synchronous HTTPS create/close callbacks and the four serial/admin ticket/upgrade handlers. It adds no owner task, queued probe or socket-capacity change. See ordinary idle cleanup for the independent cleanup policy and acceptance evidence for user reports.

Usage and interpretation

Use UART0 (preferred during network stalls) or authenticated admin SSH:

web diagnostics clear
web diagnostics enable
web diagnostics show

Enable before reproducing the failed third connection. At failure, run web diagnostics show promptly, alongside existing memory/web counters/browser Network timing evidence; repeat the snapshot if admission remains stalled. Then web diagnostics disable freezes event retention (live occupancy continues updating); web diagnostics show prints retained history, and web diagnostics clear erases history/counters without changing live sockets or enable state. The setting is RAM-only and defaults off after boot. Existing browser-shell policy deliberately denies these commands, even though shared completion offers the fixed forms. There is no HTTP diagnostic endpoint.

• Six connection records are maintained from boot even with capture disabled, so enabling on an already loaded server does not mislabel existing sockets. Each successful TLS connection gets a monotonically increasing, non-wrapping, firmware-lifetime 64-bit conn sequence independent of fd, cookie, ticket, broker or user identity. Clear, disable and HTTPS restart do not reset it. Internal TLS object identity is used only during synchronous cleanup; no pointer is exported or dereferenced by the console.
• Snapshot output gives its boot-relative snapshot_us, post-TLS occupancy split ordinary/serial-WS/admin-WS and each live fd/connection sequence/open time/age. kind=0/1/2 means ordinary/serial/admin. Ordinary includes every successful TLS socket not yet observed as upgraded, including idle keep-alives, assets/login/settings and sockets not yet used for HTTP. Classification is actual public httpd_ws_get_fd_info() state after upgrade-handler return, not inferred from ESP_OK. No URI string is inspected.
• The fixed 32-entry overwrite ring records successful TLS open, TLS transport-context close, and enter/result for serial/admin ticket and upgrade handlers. t_us is boot-relative observation time; result dt_us measures the underlying handler only (including its work/IO, excluding entry resource sampling); close dt_us is successful-TLS connection lifetime. Open/enter durations are zero. Open/result occupancy includes the connection; close occupancy excludes it. Event IDs survive clear; overwritten counts evicted retained records since clear. unmatched counts duplicate create/unmatched close; lost counts untrackable creates (getter failure, metadata/sequence exhaustion). Nonzero anomaly counters mean occupancy is not trustworthy as complete evidence. Counters saturate.
• rc is the exact handler return, not HTTP status or ticket issuance outcome: sending a 401/403/503 can return ESP_OK. Match browser HTTP status and existing rejection/issuance counters; there is no ticket-value correlation, request ID, authentication identity, header/body/query logging, or allocation-failure attribution. Upgrade success is visible in the occupancy classification. Ring overwrite or clear/toggle during an operation can leave unpaired enter/result records; do not invent a duration for a missing result.
• Each retained event samples free/largest bytes for internal 8-bit, internal DMA and PSRAM 8-bit, plus the current HTTPD task's minimum-free stack in ESP-IDF bytes. Capability scans run outside the diagnostic lock. Samples are sequential, not an atomic heap snapshot; overlapping internal/DMA pools must not be added. Stack watermark includes diagnostic call overhead and is not dispatcher margin. show does not query a live task handle: resources are historical event samples, not fresh heap values at show time. Use memory for current system heap.
• Console snapshots copy only local diagnostic metadata under a short portMUX, never inspect HTTPD session internals or wait for its owner. Printing occurs outside the lock. At most 32 ID-qualified rows and six live records are printed, even during churn; concurrent clear/overwrite is reported as “no longer retained”. Capture epochs reject samples crossing enable/disable/clear, and sequence checks prevent a stale upgrade result from reclassifying a reused fd. No queued diagnostic work exists, hence no outstanding probe or stale queue lifetime to retire on restart. Minimal connection bookkeeping and upgrade classification remain active while disabled; heap scans and event recording do not. Instrumentation still has CPU/static RAM cost, not zero perturbation.

SDK audit and explicit blind spots

Audited installed PlatformIO ESP-IDF 5.5.0, components/esp_https_server/src/https_server.c (httpd_ssl_open, httpd_ssl_close), components/esp_http_server/src/httpd_sess.c (httpd_sess_delete) and httpd_main.c. HTTPS performs synchronous TLS creation before the configured open_fn and user_cb create callback. It installs a transport-context destructor; that destructor invokes the public close callback before deleting TLS/freeing the context. HTTPD's default close closes the fd, then clears contexts, then frees its session slot. The diagnostic close therefore marks a cleanup observation, not a FIN timestamp or causal close reason. It uses the stored fd, not a getter on an already-closed socket. Neither open_fn nor close_fn is replaced, preserving all existing HTTPS cleanup ownership and failure behavior. Callbacks execute synchronously under the existing HTTPD lifecycle; successful stop finishes cleanup before restart, while failed/partial stop retains remaining live metadata. No asynchronous fd-only events are consumed, avoiding event-delay/fd-reuse ambiguity.

This deliberately bounded first slice does not measure TCP connect/accept/listen backlog, pending clients when IDF stops accepting at capacity, handshake begin/duration/failure, TLS allocation failure or aggregate lwIP socket pressure. The public configured open hook is post-TLS too; adding it would not fix these blind spots. No owner-queued client-list probe is added. Occupancy is an owner-published successful-TLS lower bound, not the complete HTTPD session table while a handshake is in progress. A clean 6/6 snapshot supports established-connection saturation at that instant (three WS + three ordinary is directly distinguishable); fewer than six does not exonerate admission/TLS/global socket pressure. Correlate time with browser evidence and existing secret-free TLS errors; do not call this pre-TLS tracing or claim the root cause is proven. Existing capacities, receive/send/handshake timeouts and accepted admission issue remain unchanged.

Regression entry points and limits

python3 tests/web_diagnostics/run.py covers bounded metadata/ring behavior and SDK guards; python3 tests/web_admin_transport/server_lifecycle.py covers lifecycle composition. These are reusable host commands, not a new execution claim. Doubled TLS/scheduling and host tests cannot establish target admission latency, reserve floors or fault causes. Capture on UART0 alongside memory, web counters, browser timings and non-consuming broker/serial counters, without cookies/tickets/headers. Generic TLS NET_RECV_FAILED (-0x004C) and NET_CONN_RESET (-0x0050) are not OOM diagnoses.