Commander1024/ESP32_Serial_Swiss_Army_Knife
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Consolidate Phase 8 documentation

Browse Source 
Mark web administration complete, centralize current contracts and
acceptance evidence, and remove superseded slice records. Update
roadmap,
architecture notes, and test references without changing firmware
sources.
This commit is contained in:
Commander1024
2026-09-13 22:27:10 +02:00
parent 1608641d50
commit 91267b371e
39 changed files with 376 additions and 3489 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/web_throughput_diagnostics.md
+7 -9
View File
@@ -1,10 +1,10 @@
# Web serial throughput diagnostics
## Scope and evidence (2026-09-08)
## Current contract and accepted fix
This is instrumentation, not throughput tuning or a confirmed scheduler diagnosis. UART0 recovery, network-independent native USB, binary-transparent UART1 transport and the broker's one-writer/isolated-observer model remain unchanged. CPU stays at 160 MHz; scheduling, priorities, 4,096-byte broker output buffers and 512-byte web TX payloads are unchanged.
Default-disabled instrumentation measures broker fan-out and binary WebSocket TX, not CPU usage, peer receipt or pure scheduler latency. The user separately accepted the combined binary header/payload-send fix at **160 MHz / 230400 baud with full mix including browser admin**. [Acceptance evidence](web_administration_acceptance.md) distinguishes that report from the latest capture and unmeasured reserves. The old frequency-only experiment is not the retained solution.
The motivating user capture reported a 71,292-byte UART burst with four fan-outs: expected 285,168 bytes, queued 265,852, dropped 19,316. The two browsers' expected 142,584 minus actual 123,268 is exactly 19,316. No UART loss, disconnect or new login was reported during this burst. This is consistent with browser-copy loss at broker fan-out, but the old aggregate counters cannot attribute loss to either browser or identify the cause. The user reports no drops with one browser; that is not a measured paired baseline.
`web_httpd_ws_send_binary` is an IDF-5.5.0-pinned owner-only session-override send with a bounded 516-byte header/payload scratch copy and 512-byte payload. Generation validation and one outstanding item per slot remain. A non-full send installs a reject-only override before close/shutdown, preventing SDK automatic PONG/CLOSE from reentering TLS with different arguments after incomplete output, even if shutdown fails. No retry/replay; HTTPD owns TLS destruction. Text/control/admin retain the SDK sender. One API call does not promise one TLS record/packet or peer receipt. Keep CPU160, priorities, scheduling and 4096-byte broker/512-byte web payload bounds unless separately changing them with evidence.
## Broker accounting
@@ -68,12 +68,10 @@ Each timing reports count, sum in microseconds, integer estimated average (`sum/
5. Only after saving evidence, change to one browser, keeping the other full-mix participants and serial settings equivalent. Quiet/settle and repeat the complete reset/burst/drain/stop/capture sequence. Do not compare a fresh capture to lifetime counters.
6. Separately compare equivalent performance-enabled and performance-disabled bursts with fresh ordinary counter resets. Keep performance disabled for the latter; do not treat retained performance aggregates as that run's measurements. Keep admission tracing state equal and record it.
At 230400 baud, 8N1, 4,096 bytes represents approximately **177.8 ms** of continuous input retention from an empty broker queue, ignoring flow-control pauses. Compare per-client drops/HWM with latency maxima against this scale, not as a hard deadline or proof of cause; occupancy, idle gaps and timings cover different boundaries. Target follow-up is per-client attribution and these controlled comparisons, not buffer, priority, scheduler or CPU tuning.
At 230400 baud, 8N1, 4,096 bytes represents approximately **177.8 ms** of continuous input retention from an empty broker queue, ignoring flow-control pauses. Compare per-client drops/HWM with latency maxima against this scale, not as a hard deadline or proof of cause; occupancy, idle gaps and timings cover different boundaries. For future regressions, gather per-client attribution and controlled comparisons before changing buffers, priorities, scheduling or CPU.
## Resources and validation handoff
## Resources and regression references
Two fixed web-slot diagnostic records plus epoch/timestamp/state fields and per-client broker HWM fields add bounded storage; there are no new allocations, tasks, queues or payload buffers for this instrumentation. Disabled is not zero overhead: relaxed gate loads/branches remain, and broker occupancy sampling/HWM maintenance is always active. Enabled timestamping and short locked aggregate updates are bounded. Target runtime overhead, stack margins and throughput effects remain unmeasured.
Two fixed diagnostic records plus nonwrapping epoch/timestamp/slot fences and broker HWM fields add bounded storage, no new instrumentation allocations/tasks/queues/payloads. Disabled is not zero overhead: gate branches remain and broker occupancy/HWM is always active. Enabled timing and short aggregate locks have runtime costs not established by host tests. The combined-send local scratch adds HTTPD stack use; actual reserve remains a target measurement.
Parent integrated validation PASS: `python3 tests/session_broker_diagnostics/run.py` (7 groups), `python3 tests/web_serial_performance/run.py` (production queue/send/drain paths with host doubles, epochs, idle, retirement and saturation; suite-level PASS), console boundary `run.py`/`lifecycle.py`, `tests/admin_ssh_policy/run.py`, admin transport `--tickets` (25 transport/12 ticket groups), server lifecycle (21), admission diagnostics (12+guard) and cookie `--network`. Parent updated the extracted console fixture for the new command and added explicit SSH-allowed/browser-denied performance command policy regressions. Independent review found no confirmed actionable production bugs. Synthetic reproduction is not target measurement; real scheduling/critical-section overhead and delayed original-pointer reuse still rely on existing owner lifecycle guarantees.
Final parent `pio run` **PASS, 9.07 s**, **100,100 B RAM / 1,748,161 B flash**, **+552 B RAM / +3,836 B flash** against the recorded Settings-presentation build. These are whole-build figures, not isolated record sizes or runtime heap measurements. Parent diff check passed; subsequent policy-test additions change no production source and their reruns passed. Host sanitizer libraries are unavailable; no sanitizer pass is claimed. No upload, erase, commit, generated assets, target acceptance, measured enabled/disabled overhead, lossless paired baseline or tuning result.
`tests/session_broker_diagnostics/run.py` covers counters and generation-safe management; `tests/web_serial_performance/run.py` covers production send/drain paths, epochs, retirement, SDK wire compatibility and failed-send/control-reply isolation. `tests/admin_ssh_policy/run.py` covers SSH-allowed/browser-denied tracing commands. Prior focused/broad host validation and firmware builds passed; no tests were run during documentation consolidation. These harnesses do not prove real TLS/scheduler overhead, peer-byte integrity or long-soak behavior. Use the capture procedure above for future attribution, retaining explicit workload/duration/counter provenance.