97 lines
17 KiB
Markdown
97 lines
17 KiB
Markdown
# Phase 8D.16 — Broker visibility and confirmed writer assignment
|
||
|
||
## Status and scope (2026-09-13)
|
||
|
||
User-authorized implementation complete; focused host regressions, broader regressions and production build pass. **Independent review complete with no actionable findings; target validation and user phase sign-off remain pending.** Reviewer independently reran broker management/wrap, cookie Broker and Display, dispatcher, lifecycle25, UI119+C/HTML/CSP, throughput and diff checks, all PASS. Parent final `pio run` confirmation PASS6.93s,100,300 B RAM/1,782,613 B flash; diff check PASS. Host doubles do not establish real multicore timing or target margins.
|
||
|
||
Settings → Broker is admin-only and uses the existing Serial/Display label/value and action styles. It shows up to eight connected clients with ID, type, name, writer/observer role, pending output bytes, output high-water and dropped bytes. Selection is explicit and Refresh clears it. Only Assign writer followed by confirmation submits a mutation. Viewing, selecting, refreshing, switching settings/terminals and reading results do not change the lease or close either terminal.
|
||
|
||
The initial Git status was clean. Preserve the previously signed-off combined binary WebSocket send and CPU160 configuration; neither was edited. No 8D.15 diagnostics UI (diagnostics remain shell-only), 8D.17/18 popovers/icon changes, disconnect/revoke-only UI, serial persistence/configuration change, new generic runner/task/timer/mutex allocation, queue depth/item or stack-size increase, generated assets, SDK patches, upload, erase, branch or commit. No full M3 or resource-reserve acceptance claim.
|
||
|
||
## Production files and ownership
|
||
|
||
- `src/session_broker.{c,h}` owns the atomic non-consuming compact management projection, conditional writer assignment and non-reused client IDs. All serial mediation, event queues and data buffers remain broker-owned.
|
||
- `src/web_broker_settings.{c,h}` owns HTTP authorization/parsing/encoding, one originating-login-bound operation/result slot and typed execution adapter.
|
||
- `src/admin_ssh_console.{c,h}` queues only a numeric operation ID on the existing four-entry request queue. The same dispatcher executes the broker API outside its critical sections, never through a constructed command string.
|
||
- `src/web_server.c`, `src/CMakeLists.txt` compose three optional routes and the new source.
|
||
- `src/web_ui.c` supplies authored HTML/JS, native select/confirmation, bounded result polling and uncertainty recovery. Loader/CSP and generated/vendored assets are unchanged.
|
||
|
||
## Authoritative broker contract and wrap analysis
|
||
|
||
`session_broker_get_management_snapshot()` takes the existing mutex with **zero wait**, copying writer ID, lease generation and all active compact rows in one lock acquisition. It never reads UART payloads or pops events. Busy/uninitialized snapshots are unavailable, not empty-success snapshots. Pending/HWM/drop fields describe broker output, not peer receipt; counters cover the connection or last counter clear. Disconnected clients disappear; global historical accounting remains unchanged.
|
||
|
||
`session_broker_assign_writer_current(target, generation)` requires a nonzero target and generation. It shares the existing force-writer implementation, with the generation comparison and connected-target lookup **inside the same broker mutex, before any lease/counter/event effects**. Stale generations return `ESP_ERR_INVALID_STATE`; absent/reused targets return `ESP_ERR_NOT_FOUND` when the generation is otherwise current. Both map to a typed `conflict` result. Assignment to the already-current target is a no-op only after validation; UI disables this redundant action. There is no compare-unlock-force race.
|
||
|
||
The existing unconditional force/release APIs and transport request/release policy remain available. At most one connected client owns the lease; assignment makes the former writer an observer. Already-accepted serial TX bytes are not recalled or discarded. Events remain advisory: transport reconciliation still uses authoritative broker state.
|
||
|
||
Two independent stale-reference problems required small broker changes:
|
||
|
||
1. **Client IDs:** three slot bits plus 29 generation bits; generations previously wrapped from `536870911` to `1`. Free slots at their maximum generation are now skipped/retired until reboot. The final generation is still usable, including ID `UINT32_MAX` in slot 7. Exhausting all eight slots returns the existing `ESP_ERR_NO_MEM` without DTR/client publication. No previously-issued ID is reused within a boot; no wider transport protocol ID or per-client allocation is introduced. Reboot resets broker IDs but invalidates all originating web sessions, so an old authorized confirmation cannot span boots.
|
||
2. **Lease confirmation:** a separate 32-bit generation starts at 1, survives counter clear and advances under the mutex for each granted/released/revoked event, **before attempting advisory delivery**. All current lease transitions emit these events; denied requests and same-writer no-ops do not advance it. A forced transfer can advance twice (revoke plus grant), so this is an opaque version, not a transition count. Release/reacquire by the same writer invalidates old confirmation (ABA). At `UINT32_MAX` it saturates permanently; management snapshots remain readable but conditional assignment rejects and UI disables assignment. Ordinary transport requests, release/disconnect, console force and UART0/USB recovery remain available. Counter/event-sequence wrap cannot resurrect a confirmation because neither is used as its token.
|
||
|
||
New observer connections/disconnections do not invalidate unrelated confirmations. The selected target's non-reused ID still prevents reassignment to its replacement; any writer disconnect changes the lease generation. The central grant/release/revoke emission invariant must be preserved by future lease mutations.
|
||
|
||
## HTTP and operation contract
|
||
|
||
All routes require current cookie/principal authentication and administrator role. Normal users cannot read management details/results or transfer. GET retains the established bodyless/queryless/header/Origin policy. POST additionally requires canonical same-origin Origin, CSRF and exact supported JSON Content-Type, with no query or transfer encoding. Responses use no-store/nosniff/no-referrer; scratch cleanup and unread-body close behavior follow Display.
|
||
|
||
| Route | Method | Bound / response |
|
||
|---|---|---|
|
||
| `/api/settings/broker` | GET | 2,048-byte response buffer; exactly `generation`, `writer`, `clients`. 503 when broker unavailable/busy. |
|
||
| `/api/settings/broker-operation` | POST | Nonempty, at most 256 bytes/four receive calls. Exactly `action:"assign"`, `generation` (1–4294967294), `target` (1–4294967295). 202 means admitted, not executed. |
|
||
| `/api/settings/broker-operation` | GET | 96-byte response buffer; exactly `id`, `action`, `state`, latest result for this originating login only. |
|
||
|
||
Each client row has exactly `id`, `type`, `name_hex`, `pending`, `high_water`, `dropped`. Type is the existing enum (0 Console, 1 USB, 2 Web, 3 SSH, 4 Internal). Names are at most 23 bytes encoded as at most 46 lowercase hex characters; browser decoding and DOM `textContent` avoid JSON/HTML injection. Invalid UTF-8 displays replacement characters, while IDs remain the authoritative identity. `dropped` is an unsigned decimal **string**, preserving all 64 bits in JavaScript. Writer/observer role is derived from the same snapshot's writer ID. No passwords, tickets, verifiers, private keys or UART bytes are included.
|
||
|
||
The narrow flat parser rejects duplicates, missing/unknown fields, escaped names/actions, strings in numeric fields, nested values, leading zeros, negatives, fractions, exponents, integer overflow, embedded NUL and trailing garbage. Exact 256-byte whitespace-padded requests are supported. Excessive fragmentation/receive errors fail rather than retry/drain indefinitely.
|
||
|
||
One static slot is shared across administrators, but results are isolated by non-reused **web-session ID**, not username. Another login sees `{id:0,action:"none",state:"idle"}`. Pending/full/not-ready dispatcher or exhausted operation IDs returns 503/Retry-After 1. IDs never wrap; completed results are replaceable, not durable history or idempotency records.
|
||
|
||
The dispatcher checks original session/principal/admin and a **30-second dequeue admission deadline** before calling the conditional broker API. No timer or hard cancellation is added: a blocked dispatcher can retain the bounded pending record beyond the deadline until dequeue. Work admitted before logout/expiry/stop may finish; no delivery guarantee is made. Auth stop/restart invalidates queued originating identities. Principal material is wiped at completion and local copies are wiped on exit.
|
||
|
||
States: `idle` (no retained result for this login, not proof of cancellation), `pending` (queued/executing), `ok` (assignment/no-op completed at execution time, not a future lease guarantee), `conflict` (stale/exhausted generation, absent target or uninitialized broker; no change by this operation), `failed` (other owner failure), `cancelled` (currentness/dequeue deadline denied before owner admission).
|
||
|
||
Optional registration orders snapshot → result GET → mutation POST. Failed snapshot/result registration prevents later routes; failed POST removes result GET, unless unregister itself fails, leaving reads only. Other settings, authentication, serial/admin transports and recovery survive optional failures. Budget: **30 → 33 URI handlers**, six sockets/no LRU unchanged.
|
||
|
||
## Browser behavior
|
||
|
||
- Admin-only full-page Settings navigation; no empty placeholder, contextual popover, icon redesign or new ordinary-user status detail.
|
||
- No default selected target; no transfer on opening/selecting. Refresh always clears selection rather than silently renewing a previous confirmation.
|
||
- Confirmation identifies target ID/type/name and snapshot writer ID, explains observer transfer and already-queued UART bytes. The exact selected target/generation is captured before asynchronous session revalidation and submission; concurrent changes are rejected at the broker lock.
|
||
- One explicit POST, no automatic mutation retry. At most ten one-second result GETs and a 15-second overall automatic-check deadline, including delayed session work. Manual Check Result remains available after uncertainty.
|
||
- Lost acknowledgement, another-tab result replacement, invalid response and navigation preserve explicit uncertain-outcome messaging. Known terminal results refresh the snapshot once; stale/conflict requires fresh selection/confirmation.
|
||
- Navigation, pagehide, logout and expiry abort/fence stale browser work and clear selected identities/rows. This is not backend cancellation. Both terminals retain their existing hidden-output draining and selected-input policy.
|
||
|
||
## Validation actually run
|
||
|
||
All commands below passed in this implementation session with bounded terminal runtimes:
|
||
|
||
- `python3 tests/session_broker_diagnostics/run.py`: existing seven diagnostic groups plus two management/wrap groups, actual broker/console under deterministic RTOS/serial doubles. Covers atomic/non-consuming projection, zero-wait contention, USB/SSH/Web interleavings, exactly-one-writer transfer, stale/reused targets, release/reacquire ABA, counter clear, local force-release, writer disconnect, saturation and final client IDs.
|
||
- `python3 tests/web_cookie_auth/run.py --broker`: six new groups plus shared auth/store/installed-IDF boundary tests; actual HTTP policy/parser/operation module with broker dependency double. Bounds, safe eight-row encoding, session-isolated results, queue failure/replay, stale-owner result mapping, currentness/deadline/expiry/revocation/stop-restart and lost acknowledgement.
|
||
- `python3 tests/admin_console_boundary/run.py`: actual dispatcher, including new Broker ID routing, zero/full/not-ready admission, unchanged four-slot capacity and no command-runner invocation; existing certificate and SSH adapters also pass.
|
||
- `python3 tests/web_admin_transport/server_lifecycle.py`: 25 groups, including six Broker descriptor/name allocation positions, failed result unregister, failed stop/retry/restart and unrelated route isolation. Orchestration uses HTTPD/TLS doubles, not actual TLS allocation faults.
|
||
- `python3 tests/web_ui_session/run.py`: **119 browser groups** (eight new Broker groups) plus actual C renderer/HTML structure/shared styles/no-store/loader-CSP checks. Snapshot validation, safe names/64-bit counters, no view/selection mutation, confirmation/cancel, stale completion, polling bounds, lost/replaced results, navigation/401/pagehide/logout fences. No real browser geometry or target visual pass claimed.
|
||
- Cookie variants `--display`, `--network`, `--accounts`, `--serial-settings`, `--settings`, `--admin`; canonical `tests/web_network_settings/run.py`; `tests/web_auth_parse/run.py` (294 cases); `tests/admin_console_boundary/accounts.py`, `lifecycle.py`; `tests/admin_ssh_policy/run.py`; `tests/web_serial_performance/run.py` (including 513 installed-SDK wire comparisons); `tests/web_httpd_idle/run.py`; `tests/web_admin_transport/run.py --tickets`; `tests/web_session_store/run.py --serial`; `tests/web_diagnostics/run.py`; `tests/web_login_ui/run.py`.
|
||
- `git --no-pager diff --check` passed.
|
||
|
||
Tests initially caught and fixed a six-field Broker row incorrectly checked as seven fields. Harness integration corrections: static helper collision with extracted console code, HTTP test output bound, and lifecycle route-count expectations. No remaining known focused test failures. Self-review inspected broker transitions/ID generation, HTTP/session policy, optional registration and browser confirmation/result flow; subsequent independent review found no actionable findings (see status above).
|
||
|
||
### Build/resource evidence
|
||
|
||
Actual pre-edit `pio run`: PASS **7.14 s**, **100,196 B RAM / 1,765,233 B flash**. Post-integration build: PASS82.93s. Final confirmation after formatting/review: PASS21.20s, unchanged sizes. Final substantive-source build: PASS **20.54 s**, **100,300 B RAM / 1,782,613 B flash**: **+104 B RAM / +17,380 B flash** versus baseline, within the existing 4 MiB application partition. Authored response sizes: HTML28,929 bytes; app JS109,731 bytes after copy-text cleanup (not generated assets). Existing nonfatal SDK `FATFS_PRINT_FLOAT` bool-default warning occurred in the integration build; no SDK/config workaround applied.
|
||
|
||
CPU160 verified in `sdkconfig.defaults`, active `sdkconfig.esp32-s3-devkitc-1-n16r8` and generated `config/sdkconfig.h`; PlatformIO's generic board banner still says240MHz and is not the configured frequency. No CPU/scheduling/throughput tuning was performed.
|
||
|
||
Static image delta excludes runtime URI allocation overhead and peak nested stack usage. Snapshot handler uses bounded 2,048-byte response plus compact eight-row snapshot and auth locals on the existing HTTPD stack; operation admission uses256-byte request/96-byte result buffers and one slot, no payload heap allocation. HTTPD/dispatcher live margins, full-mix internal/DMA/PSRAM minima and scheduling overhead are **unmeasured**. A build pass does not establish safe runtime reserves.
|
||
|
||
## Pending target checklist — not executed
|
||
|
||
- [ ] Compare admin Broker rows with UART0 broker snapshots/counters for zero, one and full supported client mix; verify ID/type/name/role/pending/HWM/drop semantics without consuming serial data. Normal user direct GET/POST requests must be denied.
|
||
- [ ] Open Settings/Broker, select/cancel/Refresh, switch terminals/views and confirm no lease/socket change; exercise keyboard/touch selection and native confirmation on narrow/wide screens.
|
||
- [ ] Assign among USB, role-user SSH and both web serial clients while both admin routes remain open. Verify exactly one writer, observers still receive binary output, former writer input is rejected, no in-band controls and already-accepted TX semantics.
|
||
- [ ] Hold confirmation while target disconnects/reconnects; confirm old target fails without changing the current writer. Repeat with competing USB/SSH/browser writer requests, admin-shell force, local expected-writer release and same-writer release/reacquire. Refresh/reselect/confirm explicitly after conflict.
|
||
- [ ] Test two independent admin logins and two tabs on one login: pending capacity, isolated/latest replaced results, lost POST/result response, bounded polling/manual recovery and no duplicate transfer. Logout/expiry/role revoke/HTTPS stop-restart while queued must reject stale admission; already-admitted work may finish.
|
||
- [ ] With full mix at signed-off160MHz/230400 baud, repeat settled non-consuming throughput captures and verify binary transparency/drop isolation. Measure boot/full-mix/operation internal/DMA/PSRAM free/minimum/largest blocks and HTTPD/dispatcher stack minima, including maximum rows and concurrent HTTPS load.
|
||
- [ ] Validate optional route allocation failures and failed stop/restart where a safe target fault-injection mechanism is available; unavailable settings must not remove UART0 or native USB recovery. Host failure injection is not target proof.
|
||
- [ ] Record exact revision/build/client mix, observations and limits; independent review is complete, but explicit user 8D.16 sign-off remains required. No erase or identity/configuration migration is required.
|