Add broker and web throughput diagnostics

This commit is contained in:
2026-09-08 22:40:58 +02:00
parent 36d41be422
commit 042499e4d6
20 changed files with 899 additions and 5 deletions
+2
View File
@@ -84,6 +84,8 @@ UART RX -> serial-service RX stream -> broker task
The broker drains serial input even with no clients. A full client output stream drops only that client's copy and updates drop counters; it does not block UART reception or other clients.
Active-client counter snapshots expose ID/type/pending/HWM/UART/queued/read/dropped. HWM is maintained independently of web tracing and counter clear seeds current occupancy. Read means transport handoff, not peer receipt. Disconnect removes the client row while global totals retain traffic and unread-output discards. Independent default-disabled web performance capture uses two fixed slot records and nonwrapping epoch/generation fences; toggles fence in-flight samples, disable freezes aggregates, clear preserves enable state. Binary-only timestamps bound reservation-entry to callback-entry before locking, synchronous send calls and completion to broker-read return; idle and intervening work prevent scheduler-only/backlog-at-completion claims. No new instrumentation allocations or scheduling/buffer changes. Exact fields, overhead and capture contracts: `docs/web_throughput_diagnostics.md`.
### Clients to RS-232
```text
+2
View File
@@ -43,6 +43,7 @@ This is a semantic map, not a complete file inventory. Start here, then read the
- Data path: `transport -> broker -> serial service -> UART1`; reverse data is fanned out per client.
- Ownership: client IDs are slot/generation-safe; events are advisory and can drop, so use snapshots as authority.
- Lifecycle: one permanent task and eight preallocated client slots; slow output drops only for the affected client.
- Diagnostics: `broker counters` adds active-client ID/type/pending/HWM/UART/queued/read/dropped rows; clear seeds HWM from pending, disconnect removes rows but retains global discard accounting. `tests/session_broker_diagnostics/run.py`; capture before disconnect, never use consuming `broker read` as a probe. Semantics/recipe: `docs/web_throughput_diagnostics.md`.
## Native USB CDC
@@ -66,6 +67,7 @@ This is a semantic map, not a complete file inventory. Start here, then read the
- Files: `src/web_server.{h,c}`, `src/web_serial_transport.{h,c}`, `src/web_ui.{h,c}`, `src/web_console.{h,c}`
- Ordinary HTTPS idle cleanup: `src/web_httpd_idle.{c,h}`, owner sweep in `web_httpd_adapter.{c,h}`, lifecycle/TLS composition in `web_server.c`; `tests/web_httpd_idle/run.py`. Independent of diagnostics/optional transports: 15-second observed idle, one-second timer/one queued probe, six rows, actual WS/async/pending exemptions, safe current-owner shutdown and stop/restart fencing. No LRU/socket/timeout/stack increase. SDK queue/owner-delay limits and target checklist: `docs/https_idle_cleanup.md`.
- Independent throughput diagnostics: `web_serial_transport.{c,h}` owns two fixed per-slot binary-TX aggregates and epoch fences; `web_console.c` exposes default-disabled `web performance enable|disable|show|clear`. Queue-entry/callback-entry, synchronous-send and completion/drain-return estimates, not peer receipt or scheduler-only latency. `tests/web_serial_performance/run.py`; resource/evidence limits and UART0 paired capture: `docs/web_throughput_diagnostics.md`.
- Opt-in admission diagnostics: `src/web_diagnostics.{c,h}`, `tests/web_diagnostics/run.py`. Public synchronous HTTPS create/close callbacks publish six post-TLS connection records; four ticket/upgrade wrappers feed a 32-entry numeric ring. UART0/admin SSH `web diagnostics enable|disable|show|clear`; no queue/task/cleanup override or capacity change. Full bounds, SDK semantics and preaccept/TLS blind spots: `docs/phase8d11_implementation.md`.
- Legacy removal **user-signed-off 2026-09-08** (unchanged certificate fingerprint, preexisting users usable, full-mix evidence): `user_database` persists missing storage empty and preserves valid v1 user bytes; private derived `v1_admin_marker`, no public bootstrap/migration/sync APIs. `web_security` privately migrates v1 1392-byte material to TLS-only v2 1340-byte material, exact identity/generation retained, commit before publish, fail closed without fallback overwrite. Credential commands removed; user generated passwords and TLS rotation remain. Contracts, downgrade and evidence limits: `docs/legacy_credential_removal.md`.
- Security files: `src/web_security.{h,c}`, `src/web_cookie_auth.{h,c}`, `src/web_session_store.{h,c}`, `src/web_auth_parse.{h,c}`. Private IDF boundary: `src/web_httpd_adapter.{h,c}`.
+2
View File
@@ -4,6 +4,8 @@ This file is working memory. Update it during active work and before handoff; do
## Development state
- **Web throughput diagnostics handoff (2026-09-08), target unmeasured:** Broker active-client ID/type/pending/HWM/UART/queued/read/dropped rows and independent default-disabled `web performance enable|disable|show|clear` now documented against source. Two fixed web slots, epoch/generation fencing, binary-only queue-entry/callback, synchronous-send and completion/drain timing aggregates; nonempty intervals include idle and are not scheduler-only or proof of completion-time backlog. No CPU160MHz/scheduling/priority/4096-byte broker/512-byte web buffer tuning or new instrumentation allocations. User burst71,292 × four=285,168 expected,265,852 queued,19,316 dropped; browser142,584123,268 matches, without reported UART loss/disconnect/new login. One-browser no-drop report is not a measured paired sample. Parent broker7/performance suite, console boundary/lifecycle/policy (new SSH-allowed/browser-denied command tests), transport25/tickets12, server lifecycle21, admission diagnostics12+guard, cookie Network+shared PASS; review no confirmed actionable bugs. Parent pio run PASS9.07s,100,100 B RAM/1,748,161 B flash (+552/+3,836 vs Settings presentation), diff check PASS. Fixed extracted console fixture missing new handler double; no production behavior fix. Host sanitizer libraries missing, no sanitizer pass. Real scheduling/overhead/stack margins unmeasured. UART0 quiet/settled full-mix and one-browser capture recipe, before-disconnect attribution, enabled/disabled comparison, 177.8ms retention scale and limits: `docs/web_throughput_diagnostics.md`. Documentation-only pass; no source/tests/assets/device/build/commit action.
- **8D.12/8D.13 + Settings presentation functionally signed off (2026-09-08):** User explicitly accepts after 60-second boot/full-mix telemetry. Web writer8, web17/SSH10/USB11 observers, both admins, two cookies at230400 8N1 RTS/CTS DTR active. Web send/queue/protocol/close-failure0, SSH handshake/auth/IO failures0; retain SSH rejected1 byte, web rejected1 frame/1 byte, serial connect3/disconnect1, cookie login3/logout1, authfailures2 boot/3 loaded without causal diagnosis. Loaded free internal/DMA/PSRAM31,512/23,756/8,112,140 B; minima2,276/156/8,070,736 B; largest20,480/20,480/7,995,392 B. SSH stack min18,468 boot/16,276 loaded. Exact samples/counters/limits: `docs/phase8d12_13_implementation.md`. Supersedes target-pending/no-signoff wording below for implemented phases and visual refinements. Very low lifetime internal/DMA minima remain transient-headroom follow-up (conservative non-simultaneous region sums, not proof allocation failed); HTTPD/dispatcher floors/peak correlation and numeric reserves unapproved, not reopening functional acceptance. No invented individual checklist/soak/full-M3 pass, no 8D.14 authorization. Documentation only, no source/config/build/test/device/assets/commit action.
- **Settings presentation unified (2026-09-08), user-requested visual refinement:** Accounts/Network now use Serial-style 600px label/value definition lists, shared form styling, compact muted help, consistent Refresh/result labels and action grouping. Public-key textarea/generated-password fields styled; checkboxes intrinsic-width. Safe DOM text replaces preformatted summaries; readable ASCII SSID/hex fallback retained, pre-wrap preserves significant spaces (review finding fixed with rendered-width regression). IDs/events/auth/mutations/secret cleanup/terminal ownership unchanged; no backend/assets changes. Parent UI100 behavior groups plus HTML/renderer/CSP and headless Chromium geometry/whitespace checks at320/600/1200px PASS; pio run PASS23.69s, 99,548 B RAM / 1,744,325 B flash (+0/+1,744 vs preceding ASCII-summary build). Diff check PASS. Fixtures/browser layout checks are not target visual sign-off or live Wi-Fi validation. No upload/erase/commit.
+2
View File
@@ -28,6 +28,8 @@ Only constraints supported by implementation or current project documentation be
**Consequence for future changes:** Do not replace fan-out with a blocking shared queue. Any added transport must tolerate partial/no-progress reads and expose drop/backpressure counters.
**Throughput observation, not tuning:** Preserve CPU160MHz, scheduling/priorities and 4096/512-byte broker/web buffers while gathering per-client HWM/drop attribution and independent opt-in web binary-TX timing. Fixed-slot epoch/generation-fenced aggregates avoid stale attribution; no new runtime allocations. Clear preserves queued data and seeds broker HWM; disconnected rows disappear while global discard counts remain. Callback timestamps precede the transport lock; synchronous send return is not peer receipt. Completion-to-read intervals include broker/mutex/control work and possible idle, even when the first read is nonempty; never label them pure scheduling latency or proof of backlog at completion. Compare enabled/disabled target captures before drawing overhead conclusions. Contracts and reproduction: `docs/web_throughput_diagnostics.md`.
**Relevant files:** `src/session_broker.c`, `src/session_broker.h`, `docs/roadmap.md`
## Physical UART ownership and logical writer ownership remain separate