Commander1024/ESP32_Serial_Swiss_Army_Knife
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Send binary WebSocket frames in one write

Browse Source
This commit is contained in:
Commander1024
2026-09-08 23:52:00 +02:00
parent 042499e4d6
commit 60d9c54bb4
10 changed files with 334 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/agent/current-state.md
+12
View File
@@ -4,6 +4,18 @@ This file is working memory. Update it during active work and before handoff; do
## Development state
- **Throughput fix signed off at160MHz (2026-09-08):** User explicitly validates drop-free230400-baud operation with the full client mix, including browser admin, at160MHz. Current defaults, active sdkconfig and generated sdkconfig.h independently confirm160MHz selection. This supersedes earlier frequency uncertainty and pending functional throughput validation below. Retain combined binary header/payload TLS send and bounded failure isolation; no further tuning needed for this issue. Latest full-mix160MHz result is user-reported without additional raw counters or duration; no invented long-soak, peer-byte comparison or HTTPD-stack reserve validation. Documentation-only acceptance update; no source/config/build/upload/erase/commit action.
- **Combined binary WS send functionally validated by user (2026-09-08):** User confirms no drops with browser admin also connected and explicitly signs off. Prior detailed two-browser/SSH/USB capture (browser admin absent) had UART71,292 B, all four clients read71,292/drop0, global285,168, web430 frames/142,584 B, HWM2,499/2,563; send averages4,215/4,522us and callback5,691/6,253us, no transport errors/disconnects. Full-mix follow-up is user-reported, no additional counters/duration supplied. Supersedes target-pending functional status below; no sustained-soak/HTTPD-stack reserve approval inferred. User also reports commenting out CPU240 default and assumes160MHz; inspection shows tracked default commented but active sdkconfig and generated sdkconfig.h STILL select240MHz. Thus full-mix functional sign-off stands, but160MHz operation is not verified. Preserve user config edit; no automatic rollback/build/upload. Next optional step is explicit160MHz selection in defaults and active config, rebuild/generated-setting verification, then target retest. Documentation-only sign-off update.
- **Combined binary WS send experiment implemented (2026-09-08), target pending:** User authorized after CPU240 target run still dropped6,318 browser bytes. `web_httpd_adapter` now assembles FIN/binary/unmasked header+payload in516 B owner-local scratch, sends once through the existing HTTPS session override; serial binary only, controls/admin remain SDK path. IDF5.5.0-pinned owner/current-context/upgraded/nonclosing/nonasync checks,512 B payload/one pending slot/generation fencing retained. Full-length return required; short/zero/error installs reject-only send override BEFORE ws_close/shutdown so SDK automatic PONG/CLOSE cannot reenter TLS with different arguments before deferred cleanup, even if shutdown fails. Independent review found that race in initial implementation; implementer fixed, parent inspected correction and reran performance suite PASS. No retry/replay; normal HTTPD owns TLS destruction. One TLS API call is not a guarantee of one TLS record/packet or peer receipt. No priority/affinity/TCP/buffer/task/config/assets changes; CPU240 retained. Implementer final pio run PASS22.08s,100,100 B RAM/1,748,513 B flash (+352 vs CPU-only baseline); helper local stack frame560 B, runtime HTTPD margin unmeasured. Implementer tests PASS performance (513 installed-SDK wire comparisons, short/zero/TLS errors, actual SDK automatic PING/CLOSE after failure including shutdown failure, fd reuse/slot isolation/diagnostic epochs), cookie base/settings/admin, idle18, lifecycle21, store serial. Parent performance/diff check PASS. Host TLS/scheduler doubles are not target timing/fault proof. No upload/erase/commit. Next: same settled full-mix capture and dmesg burst at240MHz, memory and HTTPD stack margin, then sustained traffic; no throughput fix claimed.
- **CPU240-only target result (2026-09-08):** UART71,311 B/no loss/errors; SSH/USB each71,311 read/drop0, HWM704/449. Web10 read68,504/drop2,807; web11 read67,800/drop3,511; both HWM4096, pending0. Global278,926 read+6,318 dropped=4×71,311; web277 frames/136,304 B equals browser reads. No connect/disconnect or send/queue/protocol errors;17 authenticated status requests. Callback wait avg11,704/11,939us, send8,292/8,547us, first drain3,332/3,257us. DCD116/RI4 transitions retained without diagnosis. Roughly68% fewer dropped copies than160MHz paired run, but still overflowing: frequency-only not a fix or proof CPU is dominant. User authorized combined-header/payload experiment above; supersedes prior target-pending/next-flash CPU entry below.
- **240 MHz-only throughput experiment prepared (2026-09-08), user authorized; target pending:** Set `CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_240=y` in tracked `sdkconfig.defaults` and updated the existing ignored `sdkconfig.esp32-s3-devkitc-1-n16r8` selection/compatibility aliases so the current build actually changes from160 to240MHz. Generated `.pio/build/esp32-s3-devkitc-1-n16r8/config/sdkconfig.h` confirms CPU240 and FreeRTOS100Hz. No transport, priority, affinity, buffer, TCP, asset or task changes. `pio run` PASS56.42s,100,100 B RAM/1,748,161 B flash (unchanged); existing nonfatal SDK FATFS_PRINT_FLOAT bool-default warning. No upload/erase/commit or target throughput-fix claim. Higher power/temperature possible. Next: flash normally, establish and settle full mix including two web serial clients and both admins, then repeat quiet UART0 performance capture and same dmesg burst; compare drops/HWM/send/wait timings and memory, then longer sustained output. Defaults alone do not override an existing sdkconfig on other worktrees; explicitly update its CPU choice too. Rollback requires160MHz in both defaults and active config, then rebuild.
- **Paired 160 MHz target throughput evidence (2026-09-08):** Full mix UART71,317 B, serial loss/errors0; SSH/USB each read71,317/drop0/HWM704/640, web10 read61,801/drop9,516 and web11 read61,234/drop10,083, both HWM4096. Global queued/read265,669/drop19,599; web253 frames/123,035 B matches browser reads, send/queue/protocol errors0; no disconnects,14 status requests only. Web callback wait averages12,954/13,278us, sends9,315/9,134us, first drain3,422/3,624us. Paired one-browser run UART71,292 B, SSH/USB/web11 each read71,292/drop0, HWM706/449/1536; global213,876/drop0, web222 frames/71,292 B, wait3,338us/send7,604us/first drain2,709us, errors0. One browser disconnected after capture began,20 status requests and one denied writer request: not perfectly settled, but byte accounting exact. Both admin routes retained per user. Loss localized to browser broker queues under shared HTTPD service deficit; elapsed send time is not CPU usage and successful sends are not peer receipt. Idle-inclusive maxima (~0.95s two-browser/~2.59s one-browser) are not scheduler-stall evidence. CPU saturation/TCP blocking not distinguished; single-variable frequency experiment now authorized, superseding diagnostic-only/no-target-evidence wording below. Do not enlarge queues or alter affinity/priority/TCP alongside it.
- **Web throughput diagnostics handoff (2026-09-08), target unmeasured:** Broker active-client ID/type/pending/HWM/UART/queued/read/dropped rows and independent default-disabled `web performance enable|disable|show|clear` now documented against source. Two fixed web slots, epoch/generation fencing, binary-only queue-entry/callback, synchronous-send and completion/drain timing aggregates; nonempty intervals include idle and are not scheduler-only or proof of completion-time backlog. No CPU160MHz/scheduling/priority/4096-byte broker/512-byte web buffer tuning or new instrumentation allocations. User burst71,292 × four=285,168 expected,265,852 queued,19,316 dropped; browser142,584−123,268 matches, without reported UART loss/disconnect/new login. One-browser no-drop report is not a measured paired sample. Parent broker7/performance suite, console boundary/lifecycle/policy (new SSH-allowed/browser-denied command tests), transport25/tickets12, server lifecycle21, admission diagnostics12+guard, cookie Network+shared PASS; review no confirmed actionable bugs. Parent pio run PASS9.07s,100,100 B RAM/1,748,161 B flash (+552/+3,836 vs Settings presentation), diff check PASS. Fixed extracted console fixture missing new handler double; no production behavior fix. Host sanitizer libraries missing, no sanitizer pass. Real scheduling/overhead/stack margins unmeasured. UART0 quiet/settled full-mix and one-browser capture recipe, before-disconnect attribution, enabled/disabled comparison, 177.8ms retention scale and limits: `docs/web_throughput_diagnostics.md`. Documentation-only pass; no source/tests/assets/device/build/commit action.
- **8D.12/8D.13 + Settings presentation functionally signed off (2026-09-08):** User explicitly accepts after 60-second boot/full-mix telemetry. Web writer8, web17/SSH10/USB11 observers, both admins, two cookies at230400 8N1 RTS/CTS DTR active. Web send/queue/protocol/close-failure0, SSH handshake/auth/IO failures0; retain SSH rejected1 byte, web rejected1 frame/1 byte, serial connect3/disconnect1, cookie login3/logout1, authfailures2 boot/3 loaded without causal diagnosis. Loaded free internal/DMA/PSRAM31,512/23,756/8,112,140 B; minima2,276/156/8,070,736 B; largest20,480/20,480/7,995,392 B. SSH stack min18,468 boot/16,276 loaded. Exact samples/counters/limits: `docs/phase8d12_13_implementation.md`. Supersedes target-pending/no-signoff wording below for implemented phases and visual refinements. Very low lifetime internal/DMA minima remain transient-headroom follow-up (conservative non-simultaneous region sums, not proof allocation failed); HTTPD/dispatcher floors/peak correlation and numeric reserves unapproved, not reopening functional acceptance. No invented individual checklist/soak/full-M3 pass, no 8D.14 authorization. Documentation only, no source/config/build/test/device/assets/commit action.