ESP32_Serial_Swiss_Army_Knife/docs/phase8d12_13_implementation.md

Phase 8D.12/8D.13 — Typed Network settings

Status and scope (2026-09-08)

The user authorized both phases together. Backend and admin-only Settings/Network UI are implemented: 8D.12 delivers secret-free STA/AP/profile projections, non-secret edits, explicit persistence and mDNS; 8D.13 adds explicit Wi-Fi password replacement/disabled-STA clear and manager-owned connection controls. This supersedes older wait-for-8D.12 statements, not previous scoped acceptance. No 8D.14 work, M3 completion, target sign-off or numeric memory reserve approval is claimed.

Source authority: src/web_network_settings.{c,h}, wifi_manager.{c,h}, wifi_config.{c,h}, mdns_service.{c,h}, mdns_config.{c,h}, admin_ssh_console.{c,h}, integration in web_server.c/src/CMakeLists.txt, and authored web_ui.c. Contract/test details: tests/web_network_settings/README.md, tests/web_ui_session/network.cjs, cookie Network tests and server lifecycle tests. This documentation handoff changes no source, tests, generated assets or commands. Browser-shell Wi-Fi/mDNS restrictions are unchanged; typed routes do not grant general command execution.

Routes, authorization and isolation

Method	Path	Purpose
GET	/api/settings/network	Secret-free working/runtime snapshot
GET	/api/settings/network-operation	Latest result for initiating login
POST	/api/settings/network-operation	One typed operation

All require a current admin cookie/principal. Normal users are denied, including direct API access. Existing duplicate-header, framing and Fetch-Metadata protections apply. GET is bodyless; all routes reject queries. GET permits absent Origin but rejects a supplied mismatch. POST requires matching Origin, CSRF and exactly application/json or application/json; charset=utf-8. JSON responses are no-store, nosniff and no-referrer. There is no credential-export endpoint.

The three added method/path handlers bring the configured budget to 27 handlers, with six sockets, LRU policy unchanged. Registration is optional and staged: snapshot failure skips Network operation registration; operation GET failure skips POST; POST failure unregisters operation GET while retaining the snapshot. This preserves unrelated Settings, login/status, serial and admin routes rather than making Network a base-service startup dependency. Snapshot reads do not depend on successful secret-timer admission. Optional transport failures retain their existing independence. Host lifecycle coverage is not proof of live low-memory behavior.

Complete bounded snapshot

The JSON object has exactly these domains/fields:

Object	Fields and meanings
wifi	generation (nonzero uint32), enabled_at_boot (boolean), ap, profiles
wifi.ap	policy (off, fallback, always), channel (1..11), ssid (byte string), password_configured (boolean)
each wifi.profiles entry	index (stable 0..3), enabled (boolean), priority (0..255), security (mixed, wpa3), ssid, password_configured
runtime	started (boolean), state, active_profile (-1 means none, otherwise 0..3), ip (dotted IPv4 string), ap_running (boolean), ap_clients (count), last_error (numeric esp_err_t)
mdns	generation (nonzero uint32), suffix, hostname (without .local), announced (boolean), last_error (numeric esp_err_t)

All four profiles are always present. Runtime states are stopped, starting, connecting, waiting-ip, online, backoff, ap-only, error, with unknown fallback. mixed means WPA2-or-stronger, not open or a WPA2-only guarantee. announced is expected STA announcement state, not client-verified DNS.

Wi-Fi working configuration and runtime are copied together under the Wi-Fi mutex; mDNS is a separate consistent projection, not an atomic cross-domain snapshot. Both acquisitions are zero-wait. Contention/unavailability returns 503 snapshot_unavailable, never inferred partial values. HTTPD performs no driver/NVS call or secret-bearing configuration read. Neither projection structs nor JSON contain saved PSKs or PSK lengths; only password_configured is exposed to support staging/enabling validation.

SSIDs are reversible bytes, not JSON Unicode text

SSID limits are 0..32 decoded bytes. AP and enabled STA SSIDs must be nonempty. Empty STA SSID requires disabled status and no password.

The wire codec accepts printable ASCII, standard single-character JSON escapes (\", \\, \/, \b, \f, \n, \r, \t) and case-insensitive \u00HH; each decoded codepoint is one byte. Raw non-ASCII, non-byte Unicode, surrogates and malformed escapes are rejected. Snapshot encoding uses \u00hh for nonprintable/non-ASCII bytes, quote and backslash. Thus "A\u0000\u00ff" represents 41 00 ff, including embedded NUL and non-UTF-8 bytes.

The UI offers UTF-8 text and literal hex byte pairs. It UTF-8-encodes text before byte-preserving JSON serialization; it does not submit raw JS Unicode strings as SSIDs. Existing bytes enter text mode only after fatal UTF-8 decoding and exact re-encoding (including BOM preservation), with control bytes excluded; otherwise hex is selected. Failed conversion preserves the original input. Hex accepts byte pairs with optional single spaces; the decoded limit remains 32 bytes. Summaries display printable ASCII SSIDs as quoted text (empty SSID as ""), with exact hex fallback when any byte is outside ASCII 0x20..0x7e. Quotes/backslashes are escaped for unambiguous display; rendering uses DOM text, not HTML. No silent replacement decoding, double encoding or truncation is intended.

Complete POST contract

One flat JSON object, at most 13 distinct keys; unknown/duplicate fields are rejected. No nested config, arrays, nulls, signed/fractional/exponent integers or leading-zero numbers. Booleans are JSON booleans. Every optional patch field preserves the current value when omitted; patches require at least one patch field. A request selects one domain/target only.

action	Required fields besides action	Optional fields
wifi-patch	Wi-Fi generation	enabled_at_boot, ap_policy (off/fallback/always), channel (1..11), ssid, password, clear_password:true
profile-patch	Wi-Fi generation, profile (0..3)	enabled, priority (0..255), security (mixed/wpa3), ssid, password, clear_password:true
wifi-save, wifi-load	Wi-Fi generation	none
start, stop, reconnect, next-profile	none	none
mdns-set	mDNS generation, suffix	none
mdns-save, mdns-load, mdns-defaults	mDNS generation	none

Generation is the selected domain's nonzero uint32 snapshot value. Replacement password is 8..63 printable ASCII bytes; empty replacement is invalid. Omission means Keep, never clear. Replacement and clear cannot coexist; clear_password:false is rejected. A disabled STA password can be cleared, including a single patch that disables and clears. Enabled STA requires a valid password. AP clear is canonically invalid even with AP policy off; no open-AP path exists. Syntactically admitted but canonically invalid requests can return 202 followed by invalid.

Ownership, concurrency and persistence

HTTPD validates/adopts a bounded request; only its operation ID enters the existing administration dispatcher. The dispatcher rechecks initiating session/principal/admin currentness and dequeue deadline, then invokes canonical APIs. The existing Wi-Fi manager task remains the radio/event/mDNS-transition owner; no second driver owner or generic job executor is added.

Wi-Fi patch checks generation, merges omitted fields against current secret bytes and validates the whole candidate under the configuration mutex. Required restart queue admission precedes publication; queue failure leaves RAM unchanged. Generations do not wrap/reuse. CLI applies and local Start/Stop participate, so stale browser edits cannot undo newer state. Save holds the selected generation stable under the mutex during canonical persistence. Load reads only the existing canonical stored blob and conditionally installs it; missing, invalid/incompatible or failed storage does not generate/install a new AP secret or change RAM.

Edits are RAM-only until explicit Save. Disabled-profile-only edits do not restart the radio; enabling/disabling and enabled-profile/AP changes follow canonical asynchronous restart policy. enabled_at_boot alone changes next-boot policy, not immediate radio state. Start/Stop also change RAM enabled_at_boot; Save persists that choice. Reconnect/Next are no-ops when stopped. Next profile means the next enabled profile in canonical priority order, wrapping. The UI profile selector chooses the configuration to edit, not the profile to connect to; it labels the connection action Next profile rather than promising explicit-index selection.

mDNS has its own mutex/generation and conditional Set/Save/Load/Defaults. Suffix is 1..55 lowercase ASCII letters/digits/hyphens with no leading/trailing hyphen; hostname is sak-<suffix>. Set/Load/Defaults change RAM and request manager-owned reannouncement; Save persists. Load may select deterministic MAC-derived defaults and reports that outcome. Offline edits are applied to an already-initialized responder on the next STA IP. mDNS is STA-only and failure is nonfatal. A RAM change followed by reannouncement queue failure is not rolled back. Existing NVS remains unencrypted; logical clear/replacement is not secure flash erasure.

Admission, result states and secret lifetime

Successful POST returns HTTP 202; GET returns HTTP 200. Both contain exactly id, action, state, error, for example {"id":42,"action":"profile-patch","state":"pending","error":0}. Only the initiating login can retrieve the slot. Other logins/no retained result see {"id":0,"action":"none","state":"idle","error":0}. No query ID or history exists: UI compares acknowledged ID/action. Later admission replaces the previous result. IDs never wrap; exhaustion denies admission until reboot.

State	Meaning
idle	No result retained for this login
pending	Queued or executing
accepted	RAM apply/owner queue request accepted; not association, DHCP, online, completed radio transition or verified DNS
ok	Explicit Wi-Fi/mDNS Save succeeded
failed	Canonical/owner/storage failure
cancelled	Queued expiry or session/currentness/dequeue deadline denial before canonical admission
stale	Selected generation mismatched
invalid	Canonical configuration rejected patch/load
loaded_defaults	mDNS Load selected deterministic RAM defaults and queued reannouncement
applied_not_queued	mDNS RAM changed but reannouncement queue failed; refresh, do not assume rollback

error is numeric esp_err_t, not arbitrary input/error-text echo or a state override; cancellation can have zero error. Later runtime errors appear in fresh snapshots, not by rewriting accepted.

Existing HTTP errors include 400 framing/query/body errors, 401 authentication, 403 Origin/CSRF/admin denial, and 503 auth unavailable; unsupported handler methods return 405. Backend errors are 400 invalid_network_request, 503 timer_unavailable, 503 busy with Retry-After: 1, and 503 snapshot_unavailable. Malformed input never queues; unread-body/receive failures close rather than drain.

One static session-bound pending/result slot has an executing reservation under a short portMUX. One firmware-lifetime one-second ESP timer cancels and wipes non-executing inputs at 30 seconds plus scheduling latency. Shared input wipes on dequeue before auth checks; dispatcher-local inputs wipe on every return. HTTP body/parser/operation inputs wipe on rejection and before response IO. Already-admitted work may finish after logout/disconnect/deadline: no hard cancellation, transactional session-liveness or hard wall-clock erasure guarantee. Expired IDs cannot execute a replacement operation. Queue entries never carry credentials.

UI behavior and connection-loss safety

Network is an admin-only Settings subview with strict snapshot/result shape validation, independent request ownership and stale/session/navigation fencing. Apply submits changed fields for the selected target. Refresh discards drafts; Save persists device working state, not unsubmitted browser inputs. Stale/unavailable snapshots disable mutation instead of inferring values.

Passwords are never fetched/prefilled: explicit Keep/Replace/Clear, with Clear restricted to disabled STA. Replacement input has a 60-second context-bound browser lifetime and best-effort clearing on expiry, context/navigation/session change, refresh, submission and rejection. This does not promise secure erasure of immutable JS/browser copies. AP clear is unavailable in UI and denied by canonical validation.

Disruptive Start/Stop/Reconnect/Next/Load, AP changes and enabled-profile changes require explicit confirmation and recovery warnings; mDNS Load/Defaults confirm replacement of working state. Persistence and hostname consequences remain explicit. No automatic mutation replay. After an acknowledged POST, checks run at one-second intervals, at most ten GETs/15 seconds, with session checks; known terminal results refresh the snapshot. Manual Check Result/Refresh handles pending, replaced or uncertain results. A lost acknowledgement may leave the latest result attributable to an earlier request/another tab; an unknown ID must not be treated as proof of completion.

There is no same-response delivery guarantee: HTTPS, SSH and both browser WebSockets can disconnect before the POST acknowledgement or result arrives. accepted, a lost response, 401 or disconnect proves neither online nor cancellation. Reconnect through the available STA/AP address and inspect state before retrying. Changed hostname requires DNS verification and browser trust/login review at the new origin; host-only cookies do not move with the name. UART0 remains administrative recovery; native USB remains network-independent UART1 access, not a replacement admin console.

Settings navigation itself does not close terminals, release writer ownership or reconfigure UART1. Hidden terminal draining and selected-keyboard rules remain. Actual network disruption can close network transports and consequently release their broker client/lease; it does not intentionally stop the serial service or USB. Do not claim uninterrupted network serial delivery across a radio restart.

Resources and evidence

• POST maximum 768 bytes, at most four receives, 13 keys, 64-byte parser value scratch; no heap JSON tree.
• Snapshot buffer 2,048 bytes; backend maximum escaped fixture has 1,877 payload bytes (fixture bound, not runtime heap measurement).
• Result buffer 128 bytes; one slot and one small persistent timer.
• 27 handlers/six sockets; no task count, task stack size, dispatcher item size, queue depth or persisted schema growth. Added state/timer/buffers are not zero-cost: runtime timer heap, internal/DMA/PSRAM floors, allocation overhead and HTTPD/dispatcher stack margins remain pending.

Reported evidence, not reruns by this documentation agent:

• Backend agent: Network backend and cookie Network suites PASS; its contract README records additional cookie/settings/account/admin and canonical console regressions. Backend P3 queue-drop-counter finding fixed, preserving failed queue-admission observability.
• UI agent: 97 groups plus renderer/CSP checks PASS, review PASS.
• Integration/lifecycle agent: 21 groups PASS.
• Backend sanitizer attempt could not link because host ASan/UBSan libraries were missing; no sanitizer pass claimed.
• Final parent integrated validation PASS: python3 tests/web_network_settings/run.py (five production-path groups), cookie --network (five Network groups plus shared auth), --accounts, --serial-settings, --admin; canonical console boundary run.py and accounts.py; server lifecycle 21; browser UI 97 plus renderer/CSP; idle cleanup 18 + SDK guards; admin transport 25/tickets 12; session-store --serial; diagnostics 12 + integration/secrecy; git diff --check. The backend queue-drop projection finding is fixed and covered in these reruns. Independent backend and UI reviews reported no other actionable findings.
• Parent pio run PASS, 24.99 s, 99,548 B RAM / 1,742,437 B flash, +288 B RAM / +36,656 B flash versus accepted legacy-cleanup build (99,260 / 1,705,781). Earlier integration-only build was 99,548 / 1,718,721 before final UI; it emitted a nonfatal FATFS_PRINT_FLOAT boolean-configuration warning. The final parent incremental build did not emit it. No unrelated configuration change was made.
• UI agent measured authored rendered HTML 23,184 B (+5,245) and app.js 86,535 B (+23,843). These are uncompressed renderer sizes, not separate target heap measurements. Generated embedded vendor assets were not regenerated.
• No upload, erase, hardware validation, commit, heap reserve or target sign-off. Real association/DHCP/AP transitions, mDNS announcement, concurrent radio-owner behavior and HTTPD/dispatcher stack floors remain pending; host manager tests exercise extracted production paths with driver/scheduler/storage doubles rather than a full real radio loop.

Pending target checklist

Record revision/browser/client mix and only nonsecret evidence. Prepare UART0 and native USB before deliberate network disruption; use disposable profile changes with an explicit recovery plan. This is a procedure, not completed validation.

1. Verify admin-only UI and direct-route normal-user denial, missing/wrong Origin/CSRF, expiry/logout, unavailable snapshot and malformed/boundary fields. Check no PSK/value/length leakage through JSON, UI summaries, logs, completion or local display.
2. Round-trip printable, UTF-8, BOM, control/NUL, non-UTF-8 and maximum 32-byte SSIDs in text/hex; verify failed conversions preserve drafts. Test Keep/Replace/disabled-STA Clear, combined disable/clear, enabled-STA and AP-clear denial, expiry/context changes and failed submissions without stored-secret prefill.
3. Race browser generations against CLI/local Start/Stop and another tab. Exercise queue saturation/drop accounting, stale patches and save/load. Confirm failed admission leaves RAM unchanged. Test stored-only Wi-Fi Load with missing/invalid/read/commit failures without default-secret generation; distinguish working edits, explicit Save and reboot persistence.
4. Exercise Start/Stop/Reconnect/Next and AP policies, stopped no-ops, canonical priority/wrap, and editing versus connection selection. Cancel confirmations. Deliberately lose acknowledgements/results, revisit Settings and use Check Result/Refresh without replay. Verify STA/AP recovery and UART0/USB availability.
5. Exercise live and offline mDNS Set/Save/Load/Defaults, stale generation, queue failure (applied_not_queued), responder init/live failures and next-STA-IP reconciliation. Verify actual client DNS withdrawal/reannouncement, changed-hostname trust/login and separate IP/name origins; announced alone is insufficient.
6. Keep USB, two browser serial clients, SSH serial, and both admin routes active where possible. Check one writer/isolated observers, hidden output draining and no navigation-induced serial disruption. Separate expected losses from actual network changes from unrelated serial/broker regression; capture broker drops and transport errors, not merely UI responsiveness.
7. Exercise optional route registration/allocation failure and stop/restart isolation on target where fault injection is available; retain base login/status, serial/admin and other settings. Confirm timer-unavailable admission fails safely and no queued stale ID mutates newer work. Delayed dispatcher/scheduler behavior is not a hard-cancellation test guarantee.
8. Measure settled boot/full-mix internal/DMA/PSRAM free/minimum/largest blocks, memory floor during TLS/admission and Network reads/writes, timer/slot overhead, repeated-operation cleanup and soak. Capture HTTPD and administration-dispatcher stack high-water margins, not SSH alone; no stack/task/queue increase is authorized by this checklist. Completed parent build/tests and pending user target acceptance remain separate evidence.

Explicit exclusions

No specific-index connection selection (only canonical Next), Wi-Fi reset/default generation, AP-open mode, secret export/fetch, durable operation history/idempotency, cancellation endpoint, generic jobs/command runner, scans or new diagnostics workflow. No display settings/8D.14, M3 completion, browser-shell policy widening, new commands, generated-asset changes, schema migration, task/stack/queue expansion, factory erase or new security hardening. Accepted legacy-cleanup startup correction is documentary only: main.c independently gates SSH on Wi-Fi plus SSH security/runtime readiness, not HTTPS identity readiness.