# Web serial throughput diagnostics

## Current contract and accepted fix

Default-disabled instrumentation measures broker fan-out and binary WebSocket TX, not CPU usage, peer receipt or pure scheduler latency. The user separately accepted the combined binary header/payload-send fix at **160 MHz / 230400 baud with full mix including browser admin**. [Acceptance evidence](web_administration_acceptance.md) distinguishes that report from the latest capture and unmeasured reserves. The old frequency-only experiment is not the retained solution.

`web_httpd_ws_send_binary` is an IDF-5.5.0-pinned owner-only session-override send with a bounded 516-byte header/payload scratch copy and 512-byte payload. Generation validation and one outstanding item per slot remain. A non-full send installs a reject-only override before close/shutdown, preventing SDK automatic PONG/CLOSE from reentering TLS with different arguments after incomplete output, even if shutdown fails. No retry/replay; HTTPD owns TLS destruction. Text/control/admin retain the SDK sender. One API call does not promise one TLS record/packet or peer receipt. Keep CPU160, priorities, scheduling and 4096-byte broker/512-byte web payload bounds unless separately changing them with evidence.

## Broker accounting

`broker counters` retains global totals and adds one row per active client: generation-safe ID, type, pending bytes, output high-water mark (HWM), UART bytes considered for that client, queued bytes, read bytes and dropped bytes. `broker clients` helps map the IDs to active transports; `web performance show` supplies each web slot's broker ID.

- HWM is peak output occupancy, bounded by 4,096 bytes. `broker clear-counters` clears totals but seeds each active HWM with current pending occupancy, not zero. It does not drain queues or change ownership.
- `read` means handed to a transport, not received or rendered by a peer. Queued counts successful copies; UART counts bytes considered while the client was connected.
- Disconnected per-client rows disappear and slot/generation reuse resets them. Global counters retain disconnected traffic until cleared, including accepted-but-unread output discarded on disconnect. Thus global drops need not mean only overflow, and queued plus dropped need not equal UART fan-out after disconnect discards.
- Capture before disconnect. **Do not use `broker read` for observation:** it consumes data intended for that transport. Snapshot/counter commands do not consume serial data.

## Independent web performance capture

Use physical UART0 for the least intrusive capture. Commands are also available through the authenticated admin SSH registry. `web performance enable|disable|show|clear` is separate from admission tracing (`web diagnostics enable|disable|show|clear`) and from ordinary `web counters|clear-counters`.

Performance capture defaults disabled. Enable resumes aggregates; disable freezes aggregates; clear resets aggregates while preserving enable state. Each control operation advances a nonwrapping epoch and fences in-flight samples. Exhaustion fails closed. Slot generation and broker identity also fence reuse; this is not a disconnected-session archive.

Each active one of two fixed slots shows fd, generation, broker ID, pending/measured-pending/executing state and current-epoch `pending_age_us`. Age starts at reservation-path entry and includes an executing send. If `measured_pending=0`, zero age is unavailable, not evidence of immediate service. Disabling invalidates current-epoch pending age; capture a live `show` only if investigating a stuck queue, accepting its perturbation.

Only serial **binary TX** is measured. Text/control frames are not samples, though they can delay subsequent binary work. No payloads, passwords, keys, cookies, tickets, verifier material or other authentication secrets are retained or printed.

| Printed timing | Exact interpretation |
|---|---|
| `queue->callback-entry` | Reservation-path entry to HTTPD callback entry, timestamped before the callback takes the transport lock. Includes submission-path work, not just HTTPD queue residence; owned callbacks retired without sending can count. |
| `send-call` | Time around the actual synchronous HTTPD-owner send API call. Return is not peer acknowledgement, browser receipt or rendering. |
| `completion->first-drain-attempt-return` | Successful send-call completion to return of the first subsequent broker read attempt, including empty/error attempts. |
| `completion->next-nonempty-drain-return (includes idle)` | Completion to the next successful nonempty broker read return, possibly after empty attempts and source idle time. |
| `completion->first-attempt-nonempty-return` | Subset where the first subsequent read returns data. Excludes observed empty attempts, but does not prove backlog existed at send completion. |

Each timing reports count, sum in microseconds, integer estimated average (`sum/count`, zero for no samples) and maximum. Completion intervals are not scheduler-only: they include transport work, broker read/mutex time and intervening control-frame work, and may include idle time even in the first-attempt-nonempty subset. They do not isolate a particular task, TLS operation or network cause.

`queued_frames/bytes` counts transport reservations, including reported queue failures; `queue_errors` records those failures. `sent_frames/bytes` counts successful send-call returns. `send_errors` includes owner-context rejection, whereas send-call timings require an actual API call. `retired` records measured callbacks retired without sending. Saturating aggregates set `saturated=1`; totals and averages then cease to be reliable. Epoch fencing intentionally prevents old work contaminating a new capture, so do not demand equality across toggles or in-flight clears.

## Reproducible UART0 capture

1. Record firmware/build identity, baud/framing/flow control, source burst size, browser versions, client IDs/roles and full-mix topology. Establish all serial clients and both admin routes first. Quiet the UART source and allow queues and connection activity to settle. Do not reconnect/login during the trace.
2. Optionally stop independent admission tracing before the timed capture. Then reset in this order, while quiet:

   ```text
   web diagnostics disable
   web performance clear
   web performance enable
   broker clear-counters
   serial clear-counters
   web clear-counters
   ```

   The first command is optional; record whether admission tracing was enabled. Resets are sequential, not an atomic cross-service snapshot.
3. Send the known burst once, stop the source and wait for transport queues to drain. Avoid repeated console/status printing during the burst. A quiet final broker pending count of zero alone does not prove browser receipt; retain peer byte counts too.
4. Stop performance capture first, then capture all counters **before disconnecting any client**:

   ```text
   web performance disable
   web performance show
   broker counters
   serial counters
   web counters
   broker clients
   ```

   Disable already prints the performance snapshot; `show` is an explicit repeatable frozen-aggregate record. Keep connection counts and source/peer totals with the sample.
5. Only after saving evidence, change to one browser, keeping the other full-mix participants and serial settings equivalent. Quiet/settle and repeat the complete reset/burst/drain/stop/capture sequence. Do not compare a fresh capture to lifetime counters.
6. Separately compare equivalent performance-enabled and performance-disabled bursts with fresh ordinary counter resets. Keep performance disabled for the latter; do not treat retained performance aggregates as that run's measurements. Keep admission tracing state equal and record it.

At 230400 baud, 8N1, 4,096 bytes represents approximately **177.8 ms** of continuous input retention from an empty broker queue, ignoring flow-control pauses. Compare per-client drops/HWM with latency maxima against this scale, not as a hard deadline or proof of cause; occupancy, idle gaps and timings cover different boundaries. For future regressions, gather per-client attribution and controlled comparisons before changing buffers, priorities, scheduling or CPU.

## Resources and regression references

Two fixed diagnostic records plus nonwrapping epoch/timestamp/slot fences and broker HWM fields add bounded storage, no new instrumentation allocations/tasks/queues/payloads. Disabled is not zero overhead: gate branches remain and broker occupancy/HWM is always active. Enabled timing and short aggregate locks have runtime costs not established by host tests. The combined-send local scratch adds HTTPD stack use; actual reserve remains a target measurement.

`tests/session_broker_diagnostics/run.py` covers counters and generation-safe management; `tests/web_serial_performance/run.py` covers production send/drain paths, epochs, retirement, SDK wire compatibility and failed-send/control-reply isolation. `tests/admin_ssh_policy/run.py` covers SSH-allowed/browser-denied tracing commands. Prior focused/broad host validation and firmware builds passed; no tests were run during documentation consolidation. These harnesses do not prove real TLS/scheduler overhead, peer-byte integrity or long-soak behavior. Use the capture procedure above for future attribution, retaining explicit workload/duration/counter provenance.