Commander1024/ESP32_Serial_Swiss_Army_Knife
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Restructure project documentation

Browse Source 
Separate operator guides from development contracts, relocate
implementation notes under `docs/devel/`, and update cross-references
and
hardware tooling paths. Add getting-started, user administration, and
firmware update guides while preserving firmware behavior.
This commit is contained in:
Commander1024
2026-09-21 15:24:42 +02:00
parent 655f26a611
commit ab6a8010ba
43 changed files with 959 additions and 364 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/agent/architecture.md
+16 -16
View File
@@ -25,7 +25,7 @@ SSH role=admin ------> shared administration dispatcher <------ UART0
## Typed Network settings
`web_network_settings` admits bounded current-admin operations to the existing ID-only dispatcher; `wifi_manager` remains radio/event/reannouncement owner and `mdns_service` owns independent hostname persistence. Zero-wait secret-free projections never copy saved PSKs onto HTTPD. Conditional compare/merge preserves omitted secrets, queue admission precedes Wi-Fi publication, and edits require explicit Save. Wi-Fi Load is stored-only; mDNS may load deterministic defaults. SSIDs remain byte-reversible; AP clear is denied. One login-bound slot and one-second timer bound queued secret retention to 30 seconds plus scheduling latency, not cancellation after owner admission. `accepted` is not online/DNS completion. See [Network contracts](../web_administration.md#network).
`web_network_settings` admits bounded current-admin operations to the existing ID-only dispatcher; `wifi_manager` remains radio/event/reannouncement owner and `mdns_service` owns independent hostname persistence. Zero-wait secret-free projections never copy saved PSKs onto HTTPD. Conditional compare/merge preserves omitted secrets, queue admission precedes Wi-Fi publication, and edits require explicit Save. Wi-Fi Load is stored-only; mDNS may load deterministic defaults. SSIDs remain byte-reversible; AP clear is denied. One login-bound slot and one-second timer bound queued secret retention to 30 seconds plus scheduling latency, not cancellation after owner admission. `accepted` is not online/DNS completion. See [Network contracts](../devel/web_administration.md#network).
## Startup and initialization
@@ -80,7 +80,7 @@ The broker drains serial input even with no clients. A full client output stream
Binary serial WebSocket output uses the IDF-5.5.0-pinned `web_httpd_ws_send_binary` adapter: one owner-only session-override send of a bounded header+payload copy (516 bytes of local scratch, 512-byte payload). Existing generation validation and one outstanding work item per slot remain required. Non-full sends immediately replace that session's send override with a reject-only guard, mark it closing and shut down the socket before deferred cleanup; this prevents automatic SDK control replies from reentering TLS after incomplete output. HTTPD retains TLS destruction ownership. Text/control and admin output retain the SDK sender. One send call does not imply one TLS record, packet or peer receipt; the user signed off drop-free230400-baud full-client-mix operation at160MHz. Runtime stack-margin validation remains a separate follow-up; evidence and acceptance scope are in `current-state.md`.
Active-client counter snapshots expose ID/type/pending/HWM/UART/queued/read/dropped. HWM is maintained independently of web tracing and counter clear seeds current occupancy. Read means transport handoff, not peer receipt. Disconnect removes the client row while global totals retain traffic and unread-output discards. Independent default-disabled web performance capture uses two fixed slot records and nonwrapping epoch/generation fences; toggles fence in-flight samples, disable freezes aggregates, clear preserves enable state. Binary-only timestamps bound reservation-entry to callback-entry before locking, synchronous send calls and completion to broker-read return; idle and intervening work prevent scheduler-only/backlog-at-completion claims. No new instrumentation allocations or scheduling/buffer changes. Exact fields, overhead and capture contracts: [throughput diagnostics](../web_throughput_diagnostics.md).
Active-client counter snapshots expose ID/type/pending/HWM/UART/queued/read/dropped. HWM is maintained independently of web tracing and counter clear seeds current occupancy. Read means transport handoff, not peer receipt. Disconnect removes the client row while global totals retain traffic and unread-output discards. Independent default-disabled web performance capture uses two fixed slot records and nonwrapping epoch/generation fences; toggles fence in-flight samples, disable freezes aggregates, clear preserves enable state. Binary-only timestamps bound reservation-entry to callback-entry before locking, synchronous send calls and completion to broker-read return; idle and intervening work prevent scheduler-only/backlog-at-completion claims. No new instrumentation allocations or scheduling/buffer changes. Exact fields, overhead and capture contracts: [throughput diagnostics](../devel/web_throughput_diagnostics.md).
### Clients to RS-232
@@ -106,31 +106,31 @@ TinyUSB callbacks enqueue/copy data and state; the transport task owns broker li
`web_server` owns HTTPS on port 443 with a persisted self-signed P-256 identity and IDF's dual-stack listener. Shared Host/Origin parsing accepts canonical bracketed IPv6 literals (no interface zones) while preserving same-origin/session binding; DNS discovery does not confer certificate trust. `web_serial_transport` mediates two fixed WebSocket slots through the broker; HTTPD owns socket sends/close, the transport task owns broker IO. Four outstanding serial tickets, four cookie sessions, one optional admin WebSocket and six total HTTPD sockets are distinct limits; LRU is disabled. Current handler capacity is 40. Base HTTPS can serve authenticated non-WebSocket routes if optional serial/admin transport initialization fails.
Cookie login/logout replaces Basic/cache. Digest-only records carry copied principals, CSRF state, absolute expiry and nonreused originating-session IDs. Strict same-origin/CSRF mutations and session/principal checks gate admission; logout invalidates its session before transport cleanup, account mutations invalidate only the affected account, and ongoing currentness is authoritative. Authentication initialization failure gates HTTPS; failed start/accepted stop wipes records. RNG/SHA/database calls run outside short spinlocks with post-call epoch/identity revalidation. [Authentication contract](../web_administration.md#authentication-and-admission).
Cookie login/logout replaces Basic/cache. Digest-only records carry copied principals, CSRF state, absolute expiry and nonreused originating-session IDs. Strict same-origin/CSRF mutations and session/principal checks gate admission; logout invalidates its session before transport cleanup, account mutations invalidate only the affected account, and ongoing currentness is authoritative. Authentication initialization failure gates HTTPS; failed start/accepted stop wipes records. RNG/SHA/database calls run outside short spinlocks with post-call epoch/identity revalidation. [Authentication contract](../devel/web_administration.md#authentication-and-admission).
`web_httpd_adapter` is the sole private IDF 5.5.0 boundary for duplicate headers, admission-before-101, consumed-scratch wiping, staged optional URI registration, combined binary sends and owner-only idle sweeps. Re-audit its version guard on SDK upgrades. HTTPD debug logging must not expose headers/tickets. `web_diagnostics` independently observes public post-TLS callbacks using six metadata records and a default-disabled 32-event ring; it cannot see preaccept/in-progress/failed TLS. [Admission diagnostics](../web_admission_diagnostics.md).
`web_httpd_adapter` is the sole private IDF 5.5.0 boundary for duplicate headers, admission-before-101, consumed-scratch wiping, staged optional URI registration, combined binary sends and owner-only idle sweeps. Re-audit its version guard on SDK upgrades. HTTPD debug logging must not expose headers/tickets. `web_diagnostics` independently observes public post-TLS callbacks using six metadata records and a default-disabled 32-event ring; it cannot see preaccept/in-progress/failed TLS. [Admission diagnostics](../devel/web_admission_diagnostics.md).
`web_httpd_idle` uses one one-second timer, six rows and at most one queued owner probe. Current-owner shutdown follows 15 seconds of observed ordinary idle, exempting actual WebSockets/async/pending input. Owner delays prevent hard timeout guarantees; accepted-but-lost work stays reserved until successful destruction, failed stop retains ownership. [Idle lifecycle contract](../https_idle_cleanup.md).
`web_httpd_idle` uses one one-second timer, six rows and at most one queued owner probe. Current-owner shutdown follows 15 seconds of observed ordinary idle, exempting actual WebSockets/async/pending input. Owner delays prevent hard timeout guarantees; accepted-but-lost work stays reserved until successful destruction, failed stop retains ownership. [Idle lifecycle contract](../devel/https_idle_cleanup.md).
`web_ui`/`web_login_ui` own authored documents/scripts and hash-bound CSP loaders; authentication documents/app are no-store. Checked-in generated xterm/logo assets are compiled, not regenerated by ordinary builds. Browser Serial/Admin/Settings navigation changes view/input only, preserving serial client/lease and hidden output draining. Session-identity changes require a clean document; pagehide/restore revalidates before exposing buffers. One shared quick-settings host/controller preserves drafts, stale selections and pending uncertainty. [Terminal and console contract](../web_administration.md#terminal-and-console-ownership).
`web_ui`/`web_login_ui` own authored documents/scripts and hash-bound CSP loaders; authentication documents/app are no-store. Checked-in generated xterm/logo assets are compiled, not regenerated by ordinary builds. Browser Serial/Admin/Settings navigation changes view/input only, preserving serial client/lease and hidden output draining. Session-identity changes require a clean document; pagehide/restore revalidates before exposing buffers. One shared quick-settings host/controller preserves drafts, stale selections and pending uncertainty. [Terminal and console contract](../devel/web_administration.md#terminal-and-console-ownership).
### Browser administration and HTTPS lifecycle
`web_admin_transport`/`web_admin_tickets` add one optional admin socket, two session-bound tickets and admission to the same two remote-console slots shared with SSH, never a broker client. HTTPD owns a 1,552-byte PSRAM-only payload and IO; a 20 ms timer queues at most one poll. Current-owner shutdown avoids queued reusable socket pointers. Detach fences submissions and only successful HTTPD stop retires old work. Console/owner checks enforce currentness before sensitive work; unsupported parsed shell commands reject before effects. [Browser-shell policy](../web_administration.md#browser-shell-policy).
`web_admin_transport`/`web_admin_tickets` add one optional admin socket, two session-bound tickets and admission to the same two remote-console slots shared with SSH, never a broker client. HTTPD owns a 1,552-byte PSRAM-only payload and IO; a 20 ms timer queues at most one poll. Current-owner shutdown avoids queued reusable socket pointers. Detach fences submissions and only successful HTTPD stop retires old work. Console/owner checks enforce currentness before sensitive work; unsupported parsed shell commands reject before effects. [Browser-shell policy](../devel/web_administration.md#browser-shell-policy).
`web_lifecycle_settings` uses one original-login slot and send-return → nonreused-ID HTTPD callback → existing dispatcher for self-cutting HTTPS/reboot actions. Two-second ACK and 30-second dequeue bounds precede admission, not receipt/completion. Accepted-but-lost callbacks retain one reservation through failed stop; only callback/successful destruction retires it. Conditional stop/restart reserve saturated lifecycle generation; restart retains ownership through stop/start. Conditional reboot invokes canonical `esp_restart()` outside locks, never HTTPD self-stop or console-cleanup waits.
`web_server_replace_identity` reserves service before identity and retains both through commit → reserved stop/start. Direct security and canonical CLI/browser-shell paths share task-bound nonreused identity reservations. Crypto/NVS run outside short security/service locks; commit precedes publication/wipe. Precommit failure leaves identity/HTTPD/logins unchanged; postcommit lifecycle failure never rolls back identity and can leave served/stored fingerprints different. Failed stop skips start and retains canonical recovery. Public service/security projections are separate observations, not authorization. [HTTPS ownership, generation and recovery contract](../web_administration.md#https-and-reboot).
`web_server_replace_identity` reserves service before identity and retains both through commit → reserved stop/start. Direct security and canonical CLI/browser-shell paths share task-bound nonreused identity reservations. Crypto/NVS run outside short security/service locks; commit precedes publication/wipe. Precommit failure leaves identity/HTTPD/logins unchanged; postcommit lifecycle failure never rolls back identity and can leave served/stored fingerprints different. Failed stop skips start and retains canonical recovery. Public service/security projections are separate observations, not authorization. [HTTPS ownership, generation and recovery contract](../devel/web_administration.md#https-and-reboot).
### Admin firmware upload
`web_firmware_update` streams a raw admin-only application image through HTTPD into the inactive OTA slot using standard ESP-IDF APIs. Existing Origin/CSRF/session checks precede body IO; raw length is bounded before HTTPD's narrowed length is trusted, SDK image validation and final principal currentness precede boot selection. One 4KiB buffer and a transient reboot task bound application storage; NVS and partition layout are untouched. HTTPD is occupied during upload, so normal web traffic can stall; deadlines bound receive progress, not flash-operation latency.
The server transition and identity reservations exclude competing lifecycle work. A separate atomic gate excludes ordinary reboot paths without depending on HTTPS initialization, preserving UART0 recovery. Successful response schedules delayed reset while retaining reservations; failed response after selection retains the selected-image latch but releases resources for manual reboot. A subsequent upload is refused until reset. No automatic retry, rollback, signing infrastructure or dependency patching. [Contract and validation limits](../roadmap.md#phase-10--simple-admin-web-firmware-upload).
The server transition and identity reservations exclude competing lifecycle work. A separate atomic gate excludes ordinary reboot paths without depending on HTTPS initialization, preserving UART0 recovery. Successful response schedules delayed reset while retaining reservations; failed response after selection retains the selected-image latch but releases resources for manual reboot. A subsequent upload is refused until reset. No automatic retry, rollback, signing infrastructure or dependency patching. [Contract and validation limits](../devel/roadmap.md#phase-10--simple-admin-web-firmware-upload).
### SSH
Typed SSH settings use the existing ID dispatcher and original-login result slot, never HTTPD wolfSSH calls or owner waits. Conditional lifecycle/session controls compare a saturated service generation and exact nonreused session ID under canonical locks. `ssh_transport_replace_identity` reserves service then identity before stop, retaining the command mutex across stop → commit → conditional restart. Failed stop skips mutation/start; failed persistence may follow disconnection; committed identity is never rolled back after restart failure. Only the SSH owner frees context after all slots retire, and start rejects orphan handles. Direct security/CLI/deferred SSH callers share task-bound identity reservations; crypto/NVS run outside security locks. HTTPS remains available, so no self-cutting HTTP ACK gate is needed. [SSH contracts](../web_administration.md#ssh).
Typed SSH settings use the existing ID dispatcher and original-login result slot, never HTTPD wolfSSH calls or owner waits. Conditional lifecycle/session controls compare a saturated service generation and exact nonreused session ID under canonical locks. `ssh_transport_replace_identity` reserves service then identity before stop, retaining the command mutex across stop → commit → conditional restart. Failed stop skips mutation/start; failed persistence may follow disconnection; committed identity is never rolled back after restart failure. Only the SSH owner frees context after all slots retire, and start rejects orphan handles. Direct security/CLI/deferred SSH callers share task-bound identity reservations; crypto/NVS run outside security locks. HTTPS remains available, so no self-cutting HTTP ACK gate is needed. [SSH contracts](../devel/web_administration.md#ssh).
`ssh_transport` uses wolfSSH on port 22 with one explicitly dual-stack IPv6 wildcard listener and two fixed session/handshake slots. Peer formatting preserves IPv6 interface scope; IPv4 shares the listener, not a second socket. Initialization calls `wolfSSH_Init()` in the caller before task creation; after that, one owner task pinned to core 1 exclusively owns runtime contexts/sessions and wolfSSH calls. It enforces bounded handshakes, authentication attempts, receive work, and session buffers.
@@ -160,9 +160,9 @@ NVS is not encrypted. Password verifiers improve password storage, but Wi-Fi cre
## Typed Accounts and Serial settings
`web_account_settings` owns one login-bound slot for create/password/role/delete/key operations; HTTPD reads compact zero-wait metadata and queues only IDs. The database compares target username/account-ID/auth-generation inside its mutation lock and shares canonical invariant/commit logic. Successful commands target-revoke, including self; result loss is uncertain. A one-second timer wipes queued non-executing credentials after 30 seconds plus scheduling latency; admitted work wipes locals on return. Generated password delivery is a separate no-retained-retrieval POST, not mutation; key listing is fingerprint-only with stable sparse indices. [Accounts contracts](../web_administration.md#accounts-and-authorized-keys).
`web_account_settings` owns one login-bound slot for create/password/role/delete/key operations; HTTPD reads compact zero-wait metadata and queues only IDs. The database compares target username/account-ID/auth-generation inside its mutation lock and shares canonical invariant/commit logic. Successful commands target-revoke, including self; result loss is uncertain. A one-second timer wipes queued non-executing credentials after 30 seconds plus scheduling latency; admitted work wipes locals on return. Generated password delivery is a separate no-retained-retrieval POST, not mutation; key listing is fingerprint-only with stable sparse indices. [Accounts contracts](../devel/web_administration.md#accounts-and-authorized-keys).
`web_serial_settings` queues bounded typed operations to the same dispatcher, retaining one original-login result and a 30-second dequeue check. Apply/Defaults are RAM-only; Save persists device working state. Reconfiguration can discard serial-service pending bytes while broker clients/lease/output remain. Snapshot reads are zero-wait and consistent; `/api/status` reports unavailable running state as null. Settings navigation preserves both terminals; bounded completion checks and manual uncertainty recovery never replay mutations. [Typed API/lifetime](../web_administration.md#typed-settings-api-and-operation-lifetime).
`web_serial_settings` queues bounded typed operations to the same dispatcher, retaining one original-login result and a 30-second dequeue check. Apply/Defaults are RAM-only; Save persists device working state. Reconfiguration can discard serial-service pending bytes while broker clients/lease/output remain. Snapshot reads are zero-wait and consistent; `/api/status` reports unavailable running state as null. Settings navigation preserves both terminals; bounded completion checks and manual uncertainty recovery never replay mutations. [Typed API/lifetime](../devel/web_administration.md#typed-settings-api-and-operation-lifetime).
## Console architecture
@@ -185,7 +185,7 @@ For SSH, standard output/error is redirected to the invoking session's bounded o
Admin SSH `exit`, remote reboot, SSH stop/disconnect, and host-key rotate/reset use deferred control. The control task waits up to ten seconds for command state plus administration and transport application buffers to clear, then adds a short delay; this is a bounded best-effort heuristic, not peer-delivery confirmation. UART0 invokes these actions synchronously. User mutations and their revocations are not part of this mechanism. UART0 linenoise and the SSH editor consume the same manually maintained completion matcher and candidate formatter, so the two administration routes cannot drift in offered or displayed ambiguous completions; the hints can still drift from command registration and are not an authorization list.
Browser stop/reboot uses this same owner-adapter control path. Exact forced certificate rotation instead uses the typed queue union and immutable `dispatcher_actions` mask to hand off after drain/200 ms to the existing 12 KiB dispatcher, not the 4 KiB control stack. Pending input is discarded through execution and an executing slot remains reserved across self-detach. Canonical shared service/identity replacement preserves commit/stop/start failure semantics. Parsed other-account interactive add/password and forced role/delete are allowed. Canonical Wi-Fi commands are also allowed, including hidden secret prompts and disruptive changes; these use the ordinary owner queue and may cut off the browser before output arrives, without a deferred acknowledgement guarantee. Browser self/generated/key/recovery, mDNS mutations and restricted SSH shell actions remain denied; typed Settings has separate permissions. Post-prompt currentness is operation admission, not an atomic session-liveness/NVS guarantee. [Browser policy and recovery](../web_administration.md#browser-shell-policy).
Browser stop/reboot uses this same owner-adapter control path. Exact forced certificate rotation instead uses the typed queue union and immutable `dispatcher_actions` mask to hand off after drain/200 ms to the existing 12 KiB dispatcher, not the 4 KiB control stack. Pending input is discarded through execution and an executing slot remains reserved across self-detach. Canonical shared service/identity replacement preserves commit/stop/start failure semantics. Parsed other-account interactive add/password and forced role/delete are allowed. Canonical Wi-Fi commands are also allowed, including hidden secret prompts and disruptive changes; these use the ordinary owner queue and may cut off the browser before output arrives, without a deferred acknowledgement guarantee. Browser self/generated/key/recovery, mDNS mutations and restricted SSH shell actions remain denied; typed Settings has separate permissions. Post-prompt currentness is operation admission, not an atomic session-liveness/NVS guarantee. [Browser policy and recovery](../devel/web_administration.md#browser-shell-policy).
### Shared IPv4/IPv6 diagnostics
@@ -217,9 +217,9 @@ Configuration modules generally choose RAM defaults without erasing incompatible
## Typed Display and Broker settings
`web_display_settings` queues IDs; `local_status_ui` owns a nonwrapping configuration generation and zero-wait writer reservation shared with CLI/legacy Apply. NVS runs outside critical sections. Save stabilizes RAM, Load preserves fallback behavior, Reset commits before RAM publication. Buttons/diagnostic holds change independent activity state, not config generation. Settings need an available UI task, not an attached OLED. [Display contract](../web_administration.md#serial-and-display).
`web_display_settings` queues IDs; `local_status_ui` owns a nonwrapping configuration generation and zero-wait writer reservation shared with CLI/legacy Apply. NVS runs outside critical sections. Save stabilizes RAM, Load preserves fallback behavior, Reset commits before RAM publication. Buttons/diagnostic holds change independent activity state, not config generation. Settings need an available UI task, not an attached OLED. [Display contract](../devel/web_administration.md#serial-and-display).
`web_broker_settings` exposes compact zero-wait rows and confirmed writer assignment through one login-isolated slot. The broker atomically snapshots clients/writer/lease version and compares selected target/version inside the force-writer lock before effects. Three-bit slot/29-bit client generations retire rather than wrap; a separate saturated 32-bit lease generation advances before advisory event delivery and survives counter clear. Saturation blocks conditional assignment, not ordinary request/release/disconnect/recovery force. Contextual refresh never silently rebases explicit selections or clears sticky stale/absence latches; deliberate reselection is required. [Broker/context contracts](../web_administration.md#broker-and-contextual-controls).
`web_broker_settings` exposes compact zero-wait rows and confirmed writer assignment through one login-isolated slot. The broker atomically snapshots clients/writer/lease version and compares selected target/version inside the force-writer lock before effects. Three-bit slot/29-bit client generations retire rather than wrap; a separate saturated 32-bit lease generation advances before advisory event delivery and survives counter clear. Saturation blocks conditional assignment, not ordinary request/release/disconnect/recovery force. Contextual refresh never silently rebases explicit selections or clears sticky stale/absence latches; deliberate reselection is required. [Broker/context contracts](../devel/web_administration.md#broker-and-contextual-controls).
## Local UI and hardware boundaries
@@ -247,4 +247,4 @@ Hardware diagnostics are synchronous console commands. RS-232 tests own the phys
`user_database_init(load_result)` has no credential input. Missing storage is persisted empty; `user_database_recover_empty()` is the unavailable-only destructive recovery API. Valid v1 user bytes load without rewriting or account changes. The private `v1_admin_marker` retains its byte position and is derived from administrator count during mutations; it is not a public bootstrap state, new role or schema change. No user migration/bootstrap/synchronization API remains.
`web_security` owns TLS only. A private reader validates 1,392-byte v1 `web_sec/material`, copies exact key/certificate DER, fingerprint and generation into 1,340-byte v2, commits, then publishes. Temporary v1 credential-bearing input is wiped; no public legacy credential type/getter/rotation remains. Malformed/unknown records and read/validation/commit failures fail closed, with no fallback regeneration or overwrite of rejected records. Missing material may be generated; explicit reset replaces TLS only. Downgrade to v1-only firmware is incompatible. Logical NVS replacement is not secure flash erasure. Contracts/evidence: [legacy compatibility](../roadmap.md#phase8-legacy-credential-compatibility).
`web_security` owns TLS only. A private reader validates 1,392-byte v1 `web_sec/material`, copies exact key/certificate DER, fingerprint and generation into 1,340-byte v2, commits, then publishes. Temporary v1 credential-bearing input is wiped; no public legacy credential type/getter/rotation remains. Malformed/unknown records and read/validation/commit failures fail closed, with no fallback regeneration or overwrite of rejected records. Missing material may be generated; explicit reset replaces TLS only. Downgrade to v1-only firmware is incompatible. Logical NVS replacement is not secure flash erasure. Contracts/evidence: [legacy compatibility](../devel/roadmap.md#phase8-legacy-credential-compatibility).