Add admin firmware upload support
Implement authenticated HTTPS OTA uploads with bounded streaming, image validation, reboot coordination, and lifecycle exclusion. Add the admin UI, regression tests, and Phase 10 acceptance documentation.
This commit is contained in:
@@ -104,7 +104,7 @@ TinyUSB callbacks enqueue/copy data and state; the transport task owns broker li
|
||||
|
||||
### HTTPS, WebSocket, and web serial
|
||||
|
||||
`web_server` owns HTTPS on port 443 with a persisted self-signed P-256 identity. `web_serial_transport` mediates two fixed WebSocket slots through the broker; HTTPD owns socket sends/close, the transport task owns broker IO. Four outstanding serial tickets, four cookie sessions, one optional admin WebSocket and six total HTTPD sockets are distinct limits; LRU is disabled. Current handler capacity is 39. Base HTTPS can serve authenticated non-WebSocket routes if optional serial/admin transport initialization fails.
|
||||
`web_server` owns HTTPS on port 443 with a persisted self-signed P-256 identity. `web_serial_transport` mediates two fixed WebSocket slots through the broker; HTTPD owns socket sends/close, the transport task owns broker IO. Four outstanding serial tickets, four cookie sessions, one optional admin WebSocket and six total HTTPD sockets are distinct limits; LRU is disabled. Current handler capacity is 40. Base HTTPS can serve authenticated non-WebSocket routes if optional serial/admin transport initialization fails.
|
||||
|
||||
Cookie login/logout replaces Basic/cache. Digest-only records carry copied principals, CSRF state, absolute expiry and nonreused originating-session IDs. Strict same-origin/CSRF mutations and session/principal checks gate admission; logout invalidates its session before transport cleanup, account mutations invalidate only the affected account, and ongoing currentness is authoritative. Authentication initialization failure gates HTTPS; failed start/accepted stop wipes records. RNG/SHA/database calls run outside short spinlocks with post-call epoch/identity revalidation. [Authentication contract](../web_administration.md#authentication-and-admission).
|
||||
|
||||
@@ -122,6 +122,12 @@ Cookie login/logout replaces Basic/cache. Digest-only records carry copied princ
|
||||
|
||||
`web_server_replace_identity` reserves service before identity and retains both through commit → reserved stop/start. Direct security and canonical CLI/browser-shell paths share task-bound nonreused identity reservations. Crypto/NVS run outside short security/service locks; commit precedes publication/wipe. Precommit failure leaves identity/HTTPD/logins unchanged; postcommit lifecycle failure never rolls back identity and can leave served/stored fingerprints different. Failed stop skips start and retains canonical recovery. Public service/security projections are separate observations, not authorization. [HTTPS ownership, generation and recovery contract](../web_administration.md#https-and-reboot).
|
||||
|
||||
### Admin firmware upload
|
||||
|
||||
`web_firmware_update` streams a raw admin-only application image through HTTPD into the inactive OTA slot using standard ESP-IDF APIs. Existing Origin/CSRF/session checks precede body IO; raw length is bounded before HTTPD's narrowed length is trusted, SDK image validation and final principal currentness precede boot selection. One 4KiB buffer and a transient reboot task bound application storage; NVS and partition layout are untouched. HTTPD is occupied during upload, so normal web traffic can stall; deadlines bound receive progress, not flash-operation latency.
|
||||
|
||||
The server transition and identity reservations exclude competing lifecycle work. A separate atomic gate excludes ordinary reboot paths without depending on HTTPS initialization, preserving UART0 recovery. Successful response schedules delayed reset while retaining reservations; failed response after selection retains the selected-image latch but releases resources for manual reboot. A subsequent upload is refused until reset. No automatic retry, rollback, signing infrastructure or dependency patching. [Contract and validation limits](../roadmap.md#phase-10--simple-admin-web-firmware-upload).
|
||||
|
||||
### SSH
|
||||
|
||||
Typed SSH settings use the existing ID dispatcher and original-login result slot, never HTTPD wolfSSH calls or owner waits. Conditional lifecycle/session controls compare a saturated service generation and exact nonreused session ID under canonical locks. `ssh_transport_replace_identity` reserves service then identity before stop, retaining the command mutex across stop → commit → conditional restart. Failed stop skips mutation/start; failed persistence may follow disconnection; committed identity is never rolled back after restart failure. Only the SSH owner frees context after all slots retire, and start rejects orphan handles. Direct security/CLI/deferred SSH callers share task-bound identity reservations; crypto/NVS run outside security locks. HTTPS remains available, so no self-cutting HTTP ACK gate is needed. [SSH contracts](../web_administration.md#ssh).
|
||||
|
||||
@@ -88,10 +88,16 @@ This is a semantic map, not a complete file inventory. Start here, then read the
|
||||
### Browser admin backend
|
||||
|
||||
- Files: `src/web_admin_transport.{c,h}`, `src/web_admin_tickets.{c,h}`, protected registration/lifecycle in `web_server.c`, revocation through `web_serial_transport_revoke_*`, diagnostics in `web_console.c`.
|
||||
- Routes: CSRF-protected admin-only `POST /api/admin/ws-ticket`; ordinary `GET /ws/admin` with cookie/Origin/ticket/shared-console admission before explicit 101. Admin UI entry is explicit; no admin broker client. One socket, two tickets, existing two shared console slots; six total HTTPD sockets, LRU disabled; current overall capacity is 39 URI handlers.
|
||||
- Routes: CSRF-protected admin-only `POST /api/admin/ws-ticket`; ordinary `GET /ws/admin` with cookie/Origin/ticket/shared-console admission before explicit 101. Admin UI entry is explicit; no admin broker client. One socket, two tickets, existing two shared console slots; six total HTTPD sockets, LRU disabled; current overall capacity is 40 URI handlers.
|
||||
- Currentness/policy: `admin_ssh_console_open_available()` shares two slots with runtime SSH; transport-qualified tokens and owner adapters revalidate outside console locks before commands/prompts. Parsed browser policy remains narrower than typed Settings; [shell contract](../web_administration.md#browser-shell-policy). Tests: `tests/admin_console_boundary/{run,accounts,lifecycle}.py`, `tests/admin_ssh_policy/run.py`, `tests/web_admin_transport/run.py --tickets`, `tests/web_cookie_auth/run.py --admin`.
|
||||
- Ownership: 20 ms ESP timer queues at most one HTTPD poll, no new task; HTTPD owns 1,552 B PSRAM-only payload and IO. Closure uses HTTPD-owned `shutdown`, not IDF's reusable-pointer queued close. Detach fences submitters; only successful HTTPD stop retires queued state before restart. Session/principal currentness and generation checks protect all sensitive boundaries.
|
||||
|
||||
## Firmware upload
|
||||
|
||||
- `src/web_firmware_update.{c,h}`: admin raw `POST /api/firmware`, cookie/Origin/CSRF admission via `web_cookie_auth`, registration in `web_server`, UI in `web_ui`. Standard SDK OTA into inactive app only; bounded4KiB buffer, validated raw length/header/final image and principal before boot selection. No NVS/layout writes or vendor patches.
|
||||
- HTTPD handles streaming synchronously; upload is deliberately disruptive to web traffic. Server/identity reservations fence lifecycle changes; atomic ordinary-reboot gate also covers console/SSH/browser/button reset paths. Success-response schedules delayed reset; response failure after selection latches uploads until deliberate manual reboot. No blind retries.
|
||||
- Tests: `tests/web_firmware_update/run.py` (active-build SDK5.5.0 headers and actual begin/abort contract), `tests/web_ui_session/run.py`, existing auth/lifecycle tests. [Usage, acceptance limits and reusable regression checks](../roadmap.md#phase-10--simple-admin-web-firmware-upload). Phase 10 complete by explicit user acceptance on 2026-09-18: upload works and normal operation verified; no specific fault/NVS-comparison/power-loss/recovery pass implied. Initial install by wire; subsequent upload is application `firmware.bin`, not full-flash image.
|
||||
|
||||
## Typed settings source and regression map
|
||||
|
||||
HTTPD reads zero-wait projections and queues only IDs to the existing dispatcher. One original-login slot per domain; canonical owners compare/reserve at execution. [API/lifetime and failure contracts](../web_administration.md#typed-settings-api-and-operation-lifetime).
|
||||
|
||||
@@ -2,17 +2,22 @@
|
||||
|
||||
Working memory, not an implementation timeline. Source is authoritative; begin with [code map](code-map.md), [architecture](architecture.md) and [decisions](design-decisions.md).
|
||||
|
||||
## Phase 10 plan — simplified by user, 2026-09-18
|
||||
## Phase 10 COMPLETE — explicit user acceptance, 2026-09-18
|
||||
|
||||
- Plan only: admin HTTPS file picker/upload for a locally built ESP32-S3 application firmware.bin, existing auth/same-origin/CSRF, standard ESP-IDF OTA APIs and bounded streaming into inactive4MiBslot, SDK image/target/size validation before boot selection and controlled reboot.
|
||||
- Preserve NVS and all other data partitions; only inactiveapp/otadata writes. No wholeflash/bootloader/partitiontable uploads or erase. Storage bytes preserved, but user-selected firmware must remain schema-compatible. Wired USB-to-UART recovery if a valid image is nonfunctional.
|
||||
- Explicitly no signature/key infrastructure, antirollback/version rules, automatic rollback/healthconfirmation, remote downloadservice or dependency patches. This replaces the old broader OTA plan, not an implementation authorization. [Roadmap](../roadmap.md#phase-10--simple-admin-web-firmware-upload) is authoritative.
|
||||
- User confirmed after firmware upload implementation and the concise-UI fix: “That works perfectly. And the usual operation is also verified.” Acceptance establishes that upload works and normal operation is verified. Do not infer specific fault-injection, NVS before/after comparisons, power-loss or recovery passes. The roadmap's compact regression guidance is reusable, not an acceptance blocker.
|
||||
- Latest concise-UI change was copy-only; reported UI regression **169 groups PASS**. No rebuild after that text change. The integration build below is historical, not validation of a newly rebuilt UI or this documentation update.
|
||||
|
||||
- User authorized simple OTA implementation; initial Git clean. New web_firmware_update module with standard SDK APIs, raw POST /api/firmware, admin cookie/Origin/CSRF and final principal currentness. Settings → HTTPS / Reboot has File/XHR upload progress, confirmation, session fencing and no automatic retries. Only inactive app/otadata writes; NVS/layout untouched. No signatures/antirollback/automaticrollback, dependencies or generated assets changed. Consolidated procedure, contract and regression guidance: [Phase 10](../roadmap.md#phase-10--simple-admin-web-firmware-upload); standalone guide removed.
|
||||
- 4KiB internal buffer + transient2048B-stack reboot owner allocated before erase; 10s stall/120s receive-loop budget, not totalflashdeadline. HTTPD synchronously blocks other web work during upload; networkserial maystall/drop, reboot disruptsall. No task/request/socket lifetime capture after handler. Service/identity reservation and atomic ordinary-reboot gate cover UI/UART0/SSH/browser/localbutton paths. Failed response after bootselect schedules noautomaticreset; selected latch rejects further uploads409, manual reboot available. Successful response schedules500ms reboot retaining reservations.
|
||||
- Review fixed two actualSDK5.5.0 edge cases: failed esp_ota_begin maypublishlivehandle beforeeraseerror (abortthat handle); rawContentLength64 canwrapHTTPDsize_t32 (overflow-safe actualslotbound/strictdecimal/equality check beforebody/erase). End consumes handle evenerror. SDKvalidation followed by exactparsedimage length includingSHA; basic header requiresS3appdescriptor/hash. Unrelated old/privateSDK code unpatched.
|
||||
- Parent final pio PASS **94,220 B RAM / 1,847,645 B flash**, +24RAM/+19,080flash vsPhase9, not runtimeheadroom. Parent newbackend88cases+actualSDKbeginfailurecontract, UI169groups+CSP, serverlifecycle44, admin25, consolelifecycle, SSHruntime, cookielifecyclePASS. Additionalbase/admin/display/lifecyclecookie, idle18, SSHmanagement/runtime/security agentPASS after adding missing rebootfake to adminfixture (no productionchange). Independent review final noactionablefindings; realbuilt firmware parsed with SDKmetadata bothOTAoffsets (notdeviceflashproof).
|
||||
- **Next:** Phase 11 BLE remains planned, not authorized by this acceptance update. Preserve small scope and existing uncommitted implementation/UI work; do not resurrect Phase 9 patches. This handoff changed documentation only; no build, test, upload, erase, device operation or commit was performed.
|
||||
|
||||
## Accepted state — 2026-09-18
|
||||
|
||||
- **Reduced Phase 9 complete by explicit user sign-off.** User waived a new whole-phase device check based on prior Phase 8 validation. Application code was unchanged, but no-core-dump/silent-panic defaults changed. Do not record the waived check as executed or claim a new panic/hardware pass.
|
||||
- Small scope: standard `sdkconfig.defaults` options, source-reviewed [operational checklist](../security_operations.md), README/roadmap guidance. No dependency patches, crypto policy replacement, allocator hooks, SDK migration, encryption, eFuse or partition changes. User abandoned the extensive earlier Phase 9 and restored baseline `f40c09c`; do not resurrect it.
|
||||
- Last actual build: normal `pio run` PASS on PlatformIO6.12.0 / IDF5.5.0 / original20241119 toolchains, **94,196 B linked RAM / 1,828,565 B flash**. Generated configuration confirmed no dumps/silent panic; compilation inputs had no abandoned overlays/crypto guard. Existing generated config already selected these options. Defaults do not override saved sdkconfig; README explains verification. No new build is implied by this documentation consolidation.
|
||||
- Historical Phase 9 build: normal `pio run` PASS on PlatformIO6.12.0 / IDF5.5.0 / original20241119 toolchains, **94,196 B linked RAM / 1,828,565 B flash**. Generated configuration confirmed no dumps/silent panic; compilation inputs had no abandoned overlays/crypto guard. Existing generated config already selected these options. Defaults do not override saved sdkconfig; README explains verification. No new build is implied by this documentation consolidation.
|
||||
- **Phase 8 complete:** 8A–C target validated; explicit 8D.22 user acceptance on 2026-09-13. Completion/telemetry and legacy compatibility now live in [roadmap acceptance](../roadmap.md#phase8-acceptance-evidence) and [storage compatibility](../roadmap.md#phase8-legacy-credential-compatibility). Separate acceptance/legacy history documents removed; active [web contracts](../web_administration.md) and [regression procedures](../user_administration_tests.md) retained.
|
||||
- Post-acceptance baseline includes PSRAM-only ping payload/user snapshot allocations and refined web quick panels. Prior focused host/geometry/build checks passed; those are not additional hardware claims. Preserve lazy-allocation failure isolation and unchanged serial hot path.
|
||||
|
||||
@@ -21,7 +26,7 @@ Working memory, not an implementation timeline. Source is authoritative; begin w
|
||||
- Previously accepted combined binary WS send: CPU160MHz / 230400 baud full mix including browser admin. Latest recorded telemetry has very low internal/DMA lifetime minima (2,052/460 B); these are nonblocking headroom follow-ups, not approved reserves or proof of simultaneous allocation failure. Full table, capture workload and counter limits are preserved in the roadmap.
|
||||
- TLS `-0x004C` means generic NET_RECV_FAILED, not OOM. Historical authentication/admission symptoms do not establish a cause. Do not invent fault, soak, timing or power-loss passes.
|
||||
- Credentials remain unencrypted; old flash contents are not erased. Intermittent trusted-network operation reduces exposure, not physical-extraction risk. Upstream upgrades are separate deliberate tasks, not an endless local backport programme.
|
||||
- Phase 10 is planned, not automatically authorized by acceptance. No device operations, branch/reset, commits or dependency upgrades are part of this documentation task.
|
||||
- Phase 10 is complete by the explicit acceptance above; detailed unreported regression scenarios remain unevidenced, not completion blockers. Device operations, branch/reset, commits and dependency upgrades remain outside this documentation task.
|
||||
|
||||
## Contracts to preserve
|
||||
|
||||
|
||||
@@ -11,9 +11,11 @@ Browser `web` allows only status/stop/exact forced certificate rotation; `wifi`/
|
||||
| Command | Description |
|
||||
|---|---|
|
||||
| `memory` | Show free memory, minimum free memory, and largest blocks for internal RAM, DMA-capable RAM, and PSRAM. |
|
||||
| `reboot` | Drain console output briefly and restart the ESP32. |
|
||||
| `reboot` | Drain console output briefly and restart the ESP32; refused while a firmware upload or another reboot owns exclusion. |
|
||||
| `exit` | Close the current administrative SSH or browser session after its acknowledgement drains; unavailable on UART0. Browser `exit` leaves serial connected. Ctrl+D on an empty administrative command line does the same. |
|
||||
|
||||
Firmware upload is an admin **Settings → HTTPS / Reboot** action, not a shell command. Ordinary UART0/admin-SSH/browser-shell/local-display and typed reboot paths are gated against an active upload. If upload boot selection succeeded but its response failed, no automatic restart is scheduled: the selected image remains, another admissible upload returns 409 until reset, and a deliberate manual `reboot` is permitted after reservations release. Inspect first; a lost response is not cancellation. See [firmware update and wired recovery](roadmap.md#phase10-update-and-recovery). Do not erase for an ordinary update; native USB is UART1 access, not recovery administration.
|
||||
|
||||
## Role-based users
|
||||
|
||||
| Command | Description |
|
||||
|
||||
+36
-13
@@ -40,7 +40,7 @@ These constraints apply across all phases:
|
||||
| 7 | Local display and button interface | **Complete** |
|
||||
| 8 | Role-based users and administrative access | **Complete** |
|
||||
| 9 | Small intermittent-use security baseline | **Complete (user signoff 2026-09-18; new hardware check waived)** |
|
||||
| 10 | Simple admin web firmware upload | **Planned** |
|
||||
| 10 | Simple admin web firmware upload | **Complete (explicit user acceptance 2026-09-18; upload and normal operation verified)** |
|
||||
| 11 | BLE serial transport and provisioning evaluation | **Planned** |
|
||||
| 12 | Advanced network integration | **Under evaluation** |
|
||||
| 13 | Optional filesystem-backed features | **Under evaluation** |
|
||||
@@ -251,26 +251,49 @@ Small implementation:
|
||||
|
||||
**Acceptance (2026-09-18):** the user signed off this small Phase 9 and explicitly waived a new whole-phase hardware check of boot, UART0 recovery, native USB serial, HTTPS/SSH login and normal serial sharing. Application code is unchanged, **but crash-diagnostic defaults changed**; this is a waiver, not evidence of target or panic-path validation. The prior build/configuration evidence above remains the validation record; no new hardware test was performed. A separately controlled panic check with synthetic secrets remains optional, not a completion gate; do not add a production panic endpoint or erase flash. Disabling dumps does not erase old contents.
|
||||
|
||||
## Current and planned phases
|
||||
|
||||
**Phases 8 and 9 are complete** for their accepted scopes; Phase 9 completion includes the explicit new-hardware-check waiver above. Phase 10 is planned, not implemented; later work remains planned or under evaluation. Optional features must not weaken completed serial and recovery paths. General release gates below guide future work, not claims that every fault, soak or reserve measurement was performed for Phase 8 or that Phase 9 received new target validation.
|
||||
|
||||
### Phase 10 — Simple admin web firmware upload
|
||||
|
||||
**Plan only.** Add a file picker and upload button in the existing admin web interface. The owner builds the firmware locally and chooses the application `firmware.bin`; there is no release server, automatic download or update service. Treat an authenticated administrator as authorized to replace the application, as the physical owner can through wired flashing.
|
||||
**Complete — explicit user acceptance on 2026-09-18: firmware upload works and normal operation is verified.** The admin **Settings → HTTPS / Reboot** firmware card uploads the locally built `.pio/build/esp32-s3-devkitc-1-n16r8/firmware.bin`. Install this OTA-enabled firmware by wire first, then use web application uploads. There is no release server, automatic download or update service. [Firmware update and recovery](#phase10-update-and-recovery) below covers operation; [regression guidance](#phase10-regression-guidance) covers future checks.
|
||||
|
||||
Planned work:
|
||||
Implemented scope:
|
||||
|
||||
- Reuse existing HTTPS admin authentication, same-origin and CSRF protections; ordinary `user` accounts cannot upload firmware. Require an explicit upload/reboot confirmation and show progress, success or an actionable error. Never automatically retry a possibly completed update.
|
||||
- Use standard ESP-IDF OTA APIs (`esp_ota_begin`, `esp_ota_write`, `esp_ota_end`, `esp_ota_set_boot_partition`) to stream one upload at a time through a bounded buffer into the inactive application slot. Use the existing two 4 MiB slots and `otadata`; no custom flash protocol or whole-image RAM buffer.
|
||||
- Accept an ESP32-S3 application binary for this board/layout, not an ELF, ZIP, merged full-flash image, bootloader or partition-table image. Check nonempty/complete upload, target/header compatibility and actual destination capacity; use SDK image validation before selecting the new boot partition. A `.bin` filename alone is not validation. Basic format/integrity checks do not prove authenticity or that the application will work.
|
||||
- On interrupted, invalid or failed uploads, abort the OTA operation and leave the current boot selection unchanged. Select the new slot only after successful final validation, report completion and reboot in a controlled way. Upload may disrupt network/serial activity; reboot interrupts every transport, so do not promise uninterrupted operation.
|
||||
- Raw `POST /api/firmware` requires exact `Content-Type: application/octet-stream`, known exact `Content-Length`, cookie, same-origin Origin and `X-CSRF-Token`; no multipart/JSON, Basic authentication or ordinary `user` access. Reject queries, ambiguous headers, chunked transfer and `Expect`; UI code leaves browser-managed headers alone. Admission precedes body/flash work; session/principal currentness is rechecked before boot selection. The card confirms upload/reboot and shows progress/results without automatic retries. HTTPD capacity is 40 method/path handlers.
|
||||
- Standard ESP-IDF OTA APIs (`esp_ota_begin`, `esp_ota_write`, `esp_ota_end`, `esp_ota_set_boot_partition`) stream one upload through a 4 KiB buffer into the inactive slot (two 4 MiB app slots). A reboot task with a 2 KiB stack is preallocated before erase. Lifecycle/identity reservations and ordinary software-reboot gating exclude competing work; no custom flash protocol or whole-image RAM buffer.
|
||||
- Accepts an ESP32-S3 application binary for this board/layout, not an ELF, ZIP, merged full-flash image, bootloader or partition-table image. Browser hints require a nonempty `.bin` of at most 4 MiB; the server bounds raw length against actual destination capacity before trusting HTTPD's narrowed length. Checks completeness, target/header and exact SDK image length/integrity, including mandatory appended SHA-256, before selection. The digest is not publisher authentication; neither it nor filename checks prove board compatibility or a working application.
|
||||
- Before selection, rejected/incomplete uploads and receive/write/validation failures abort any live OTA handle without selecting the candidate; the inactive slot may be erased/partially written. Boot-metadata failure (`firmware_commit_failed`) needs inspection and carries SDK transactional uncertainty. `200 {"ok":true,"rebooting":true}` means validation/selection succeeded; successful synchronous send schedules restart after 500 ms, retaining reservations, but proves neither browser receipt nor boot success. Response failure after selection schedules no restart, releases reservations for manual reboot and latches further admissible uploads to 409 `firmware_selected_reboot_required`; the latch survives HTTPS stop/start until device reset.
|
||||
- Synchronous HTTPD receive/flash blocks other HTTPD work: browser sessions can stall/drop. The 120-second total receive-loop and ten-second stall checks use the existing one-second socket timeout, not an absolute deadline: synchronous SDK erase/write/validation and scheduling are not preempted. The browser's 180-second timeout cannot cancel committed work. UART0/native USB remain independent paths, not guarantees of uninterrupted serial timing during flash; reboot interrupts all transports. Software exclusion cannot prevent physical reset, power loss or panic.
|
||||
- **Preserve NVS:** write only the inactive application slot and the OTA selection metadata. Do not erase the chip, rewrite the partition table/bootloader, or touch `nvs`, `nvs_key`, PHY, storage or other data partitions. Existing users, passwords, Wi-Fi/serial settings and HTTPS/SSH identities remain stored, as with an application-only wired update without erase. This preserves stored bytes; the uploaded firmware must still understand the existing schemas and must not itself erase/migrate them incompatibly.
|
||||
- Keep wired USB-to-UART flashing documented as recovery if the uploaded application does not boot or no longer serves the web UI. Basic image validation cannot prevent a valid but broken application from requiring wired recovery.
|
||||
|
||||
**Not in scope:** secure boot, image signatures/signing-key management, anti-rollback/version-downgrade enforcement, automatic rollback/post-boot health-confirmation machinery, remote release discovery, partition migration or NVS backup/restore. Use upstream OTA support without dependency patches. The administrator is responsible for selecting trusted, compatible firmware, including when deliberately installing an older build.
|
||||
|
||||
**Acceptance:** test a successful application upload/reboot, invalid/wrong-target/oversized and interrupted uploads, authorization rejection, and retained configuration/identities after update. Confirm wired recovery remains possible. These are future tests, not execution claims; no new cryptographic certification or exhaustive fault campaign is required.
|
||||
**Historical integration build and host evidence (parent-reported):** `pio run` **PASS**, **94,220 B linked RAM / 1,847,645 B flash**. Against Phase 9's 94,196 B / 1,828,565 B, this is **+24 B RAM / +19,080 B flash**. Final backend 88 cases plus the pinned SDK begin/abort failure-contract test, UI 169 groups, server lifecycle 44, admin transport 25, console lifecycle, SSH runtime and cookie lifecycle checks passed. Additional cookie base/admin/display/lifecycle, HTTPD idle cleanup 18, and SSH management/runtime/security checks passed; the extra cookie `--admin` run initially lacked a reboot symbol in its fixture, corrected in test-only code before passing. Host doubles are not target evidence. The later concise-UI fix was copy-only; its latest reported UI regression passed all **169 groups**, with no rebuild after the text change. The build figures above remain historical; this documentation update ran no build or tests.
|
||||
|
||||
**Acceptance (2026-09-18):** after the firmware upload implementation and concise-UI fix, the user confirmed: “That works perfectly. And the usual operation is also verified.” Phase 10 is complete by this explicit acceptance of working upload and verified normal operation. Do not infer specific fault-injection, NVS before/after comparisons, power-loss or wired-recovery passes. The [regression guidance](#phase10-regression-guidance) below is reusable future guidance, not an acceptance blocker.
|
||||
|
||||
#### Phase10 update and recovery
|
||||
|
||||
1. Keep **USB-to-UART** available for UART0 administration/flashing; native USB CDC is network-independent UART1 access, not administration. Install the updater by wire first: older firmware without the route cannot install its own first web updater. The custom OTA layout is required; [one-time old-layout migration](../README.md#one-time-migration-from-the-default-partition-table) is separate and destructive.
|
||||
2. Build trusted, schema-compatible `esp32-s3-devkitc-1-n16r8` firmware with `pio run`; choose **`.pio/build/esp32-s3-devkitc-1-n16r8/firmware.bin`**, never a merged image or a renamed non-application file. Observe the [downgrade warning](../README.md#legacy-credential-removal).
|
||||
3. Save desired RAM-only settings and record nonsecret configuration/public HTTPS/SSH fingerprints. Arrange a maintenance window and stable power/network; sign in as `admin` over trusted HTTPS, open **Settings → HTTPS / Reboot → Firmware update**, choose the file and confirm **Upload and reboot…**.
|
||||
4. Wait for validation/reboot, not just 100% transmitted bytes. Restore network reachability, reload and explicitly sign in; verify the running application, saved configuration and identities before another upload.
|
||||
5. A lost response, timeout, page close, sign-out or browser abort proves neither cancellation nor failure. The UI locks uncertain outcomes without replay; reload clears only browser locks, not the device latch. Inspect through UART0 and deliberately reboot if appropriate: any later reset can boot an already selected image.
|
||||
6. For initial wired installation or recovery when boot/HTTPS fails, connect USB-to-UART, select a known-good compatible checkout, run `pio run --target upload`, then `pio device monitor -b 115200`. **Do not erase for ordinary updates or recovery**; there is no automatic recovery flashing/rollback.
|
||||
|
||||
Replies: **400** malformed/incompatible/incomplete input; **401/403** authentication/Origin/CSRF or `admin_required`; **408** `firmware_timeout`; **409** selected-image latch; **413/415** destination capacity/content type; **500** `firmware_write_failed`/`firmware_commit_failed`; **503** busy/unavailable/resources. Unread rejected bodies close rather than drain; bounded secret-free JSON may never reach a disconnected browser.
|
||||
|
||||
#### Phase10 regression guidance
|
||||
|
||||
These are reusable checks, **not recorded passes or outstanding acceptance gates**. Record only performed outcomes; host flash/network/scheduling doubles are not hardware evidence. Focused host suites: `python3 tests/web_firmware_update/run.py` and `python3 tests/web_ui_session/run.py` (the SDK begin/abort case covers a live handle published on erase failure).
|
||||
|
||||
- Exercise wired-first install, known-good upload/reboot/new build and explicit reconnect; compare saved users/roles, password/key login, Wi-Fi/serial/display/hostname settings and public HTTPS/SSH fingerprints without recording secrets.
|
||||
- Interrupt/cancel receive; reject corrupt, truncated, wrong-target, non-application and oversized inputs without unintended selection/restart, then deliberately perform a valid update. Reject unauthenticated/ordinary-user, wrong-Origin and missing/wrong-CSRF requests before flash. Never attempt power interruption at commit without wired recovery ready.
|
||||
- Exercise competing upload, HTTPS lifecycle/identity and ordinary reboot exclusion; HTTPD requests may wait rather than promptly return busy. With controlled post-selection response-failure injection, verify no automatic restart, subsequent 409 (also after HTTPS restart), then manual reboot; an arbitrary disconnect does not establish this fault case.
|
||||
- Observe web stalls/drops and serial/network recovery, single-writer isolation, UART0 recovery and native USB UART1 access without networking; demonstrate known-good wired recovery without erase and recheck configuration/identities. Do not claim uninterrupted traffic.
|
||||
|
||||
## Current and planned phases
|
||||
|
||||
**Phases 8, 9 and 10 are complete** for their accepted scopes. Phase 9 includes the explicit new-hardware-check waiver above; Phase 10 includes explicit user acceptance of upload and normal operation. Phase 11 remains planned; later work is under evaluation. Optional features must not weaken completed serial and recovery paths. General release gates below guide future work, not claims that every fault, soak, recovery or reserve measurement was performed for completed phases.
|
||||
|
||||
### Phase 11 — BLE
|
||||
|
||||
@@ -332,7 +355,7 @@ The following are not implemented merely because flash partitions or library sup
|
||||
|
||||
- NVS, flash, or PSRAM encryption.
|
||||
- Secure boot or production eFuse provisioning.
|
||||
- Automatic OTA downloads, image signing, post-boot health confirmation and automatic rollback. The simple admin upload is planned in Phase 10.
|
||||
- Automatic OTA downloads, image signing, post-boot health confirmation and automatic rollback. The simple admin upload is complete in Phase 10; these advanced update features remain deferred.
|
||||
- Core-dump collection or secret-safe core-dump processing.
|
||||
- Filesystem mounting.
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Web administration contracts
|
||||
|
||||
Current, accepted firmware behavior. Phase status and executed-evidence limits belong in the [roadmap](roadmap.md#phase-8--role-based-users-and-administrative-access--complete) and [acceptance record](roadmap.md#phase8-acceptance-evidence), not in implementation timelines. [Regression procedures](user_administration_tests.md#integrated-web-administration-regression-procedure) describe checks, not results. Source is authoritative; start with the [code map](agent/code-map.md).
|
||||
Current implementation contracts; [Phase 10 firmware upload is complete by explicit user acceptance on 2026-09-18](roadmap.md#phase-10--simple-admin-web-firmware-upload), confirming upload and normal operation, not specific fault or recovery checks. Phase status and executed-evidence limits belong in the [roadmap](roadmap.md#phase-8--role-based-users-and-administrative-access--complete) and [acceptance record](roadmap.md#phase8-acceptance-evidence), not in implementation timelines. [Regression procedures](user_administration_tests.md#integrated-web-administration-regression-procedure) describe checks, not results. Source is authoritative; start with the [code map](agent/code-map.md).
|
||||
|
||||
## Authentication and admission
|
||||
|
||||
@@ -9,7 +9,7 @@ Current, accepted firmware behavior. Phase status and executed-evidence limits b
|
||||
- Mutation admission requires current cookie/principal, strict Origin and CSRF validation; administration additionally requires current `admin`. Normal users retain serial/status but cannot invoke administration directly. Authentication POST fetches use CORS mode with fixed same-origin URLs and same-origin credentials: do not accept Origin `null` to compensate for browser no-referrer behavior.
|
||||
- Session-store initialization is part of admitted HTTPS start; authentication failure gates HTTPS. Failed start/accepted stop disables and wipes session state. Logout invalidates only the originating session before socket cleanup; account mutation invalidates that account's sessions/tickets, including deletion/recreation, without revoking unrelated accounts. Currentness checks remain authoritative if best-effort notifications fail.
|
||||
- Four serial tickets and two admin tickets are digest-only, single-use, 30-second, session/principal-bound records. Cookie/Origin/ticket/currentness and transport admission precede explicit WebSocket 101. Store RNG/SHA/database calls run outside short spinlocks; IDs/expiry/epochs fence stale publication without nested store/transport locks.
|
||||
- HTTPD remains bounded to six sockets, two serial WebSockets, one admin WebSocket and 39 method/path handlers; LRU eviction is disabled. Sessions, sockets, tickets and the two shared remote-console slots are separate capacity limits. Optional settings/admin failures preserve unrelated routes where their initialization contract permits; UART0 and native USB remain independent of web readiness.
|
||||
- HTTPD remains bounded to six sockets, two serial WebSockets, one admin WebSocket and 40 method/path handlers; LRU eviction is disabled. Sessions, sockets, tickets and the two shared remote-console slots are separate capacity limits. Optional settings/admin failures preserve unrelated routes where their initialization contract permits; UART0 and native USB remain independent of web readiness.
|
||||
|
||||
`web_httpd_adapter` alone accesses private IDF 5.5.0 HTTPD state. It rejects duplicate/ambiguous headers, postpones 101 until admission, and wipes consumed header scratch while preserving right-aligned unread bytes. Optional Settings registration stages descriptor/name allocations before publishing either, avoiding the pinned public registration failure path. Re-audit these private boundaries on SDK upgrades and same-version SDK patches: the version guard does not detect patches that retain the same version number. HTTPD response headers are pointer-backed, not copied; both `Set-Cookie` value buffers must remain valid and distinct through response send. Do not reuse or wipe those buffers before sending completes. Do not enable header/ticket debug logging. Auth documents, scripts and sensitive responses are no-store with CSP/no-referrer/frame-denial protections; authored loader changes require matching CSP hashes. Generated assets are not a normal documentation/build output.
|
||||
|
||||
@@ -111,6 +111,14 @@ HTTPS ordering is **commit → stop → restart**. Precommit generation/RNG/stor
|
||||
|
||||
Save drafts; rotation/restart invalidates all web logins and closes both browser routes. Inspect `web certificate info` through trusted UART0, verify fingerprint before renewing trust, then reload/sign in freshly. Accepting a warning alone is not trusted verification. Use canonical UART0/admin SSH `web stop` / `web start` for retained-server recovery. Network/SSH/USB are not stopped by HTTPS-only operations; whole-device reboot interrupts all transports and loses unsaved RAM.
|
||||
|
||||
### Application firmware upload
|
||||
|
||||
The admin-only firmware card is in **Settings → HTTPS / Reboot**. It sends a raw `POST /api/firmware` with `application/octet-stream`, known length, session cookie, same-origin Origin and `X-CSRF-Token`; it is not a JSON Settings operation or dispatcher/result-slot workflow. Standard SDK OTA APIs stream through a 4 KiB buffer to the inactive application slot and select it only after validation/currentness checks. Only that slot and `otadata` are written; NVS/data partitions are untouched.
|
||||
|
||||
One upload reserves HTTPS lifecycle/identity and excludes ordinary software reboot. The synchronous HTTPD handler can stall/drop browser serial/admin sessions. Its 120-second total receive-loop and ten-second stall checks are not preemptive flash deadlines or uninterrupted-traffic guarantees. Successful response send schedules a delayed reboot, not proof of peer receipt. Failed response after commit leaves the image selected, schedules no automatic reboot, releases reservations for manual reboot, and latches subsequent admissible uploads to 409 `firmware_selected_reboot_required` until reset. Never automatically retry an uncertain outcome.
|
||||
|
||||
See [firmware update](roadmap.md#phase-10--simple-admin-web-firmware-upload) for wired-first installation, exact image selection, manual recovery, acceptance limits and reusable future hardware regression checks. Phase 10 is accepted; there are no signature/version policies or automatic rollback.
|
||||
|
||||
### SSH
|
||||
|
||||
GET `ssh` supplies service/session state plus identity generation, fixed P-256 algorithm, unpadded OpenSSH `SHA256:` base64 fingerprint and rotatable flag. Service actions use exactly `action`, `generation`, `target`; rotate adds `identity_generation` and requires target zero. Start/stop and exact-session disconnect use published state, saturated service generation and the command mutex; exhausted SSH session slots retire rather than wrap. Disconnect success is an owner close request, not completed teardown. HTTPD never calls wolfSSH or waits for the SSH task.
|
||||
@@ -125,4 +133,4 @@ SSH changes leave invoking HTTPS available, so they use the ordinary ID-dispatch
|
||||
|
||||
See [admission diagnostics](web_admission_diagnostics.md), [ordinary HTTPS idle cleanup](https_idle_cleanup.md), [throughput diagnostics](web_throughput_diagnostics.md) and [legacy storage compatibility](roadmap.md#phase8-legacy-credential-compatibility). Broker read means transport handoff, not peer receipt; capture non-consuming counters before disconnect. TLS `-0x004C` is generic NET_RECV_FAILED, not evidence of OOM. Resource minima and counter observations require attribution, not inferred causes.
|
||||
|
||||
Phase 8D.15's dedicated typed network-diagnostics UI/API was removed: diagnostics remain shell-based, subject to frontend policy. The unimplemented 8D.19 ordinary browser-session/native-USB control expansion was removed; existing SSH controls remain. No full shell parity, browser identity recovery/reset/export, encryption, secure boot or OTA is implied by acceptance. UART0 is the administrative recovery authority; native USB is binary-transparent, network-independent UART1 access. Neither permits bypassing the broker's single writer or recalling already-admitted work.
|
||||
Phase 8D.15's dedicated typed network-diagnostics UI/API was removed: diagnostics remain shell-based, subject to frontend policy. The unimplemented 8D.19 ordinary browser-session/native-USB control expansion was removed; existing SSH controls remain. No full shell parity, browser identity recovery/reset/export, encryption or secure boot is implied by Phase 8 acceptance. Phase 10 application upload is complete by explicit user acceptance of upload and normal operation; unreported fault, NVS-comparison, power-loss and recovery checks are not implied. UART0 is the administrative recovery authority; native USB is binary-transparent, network-independent UART1 access. Neither permits bypassing the broker's single writer or recalling already-admitted work.
|
||||
|
||||
Reference in New Issue
Block a user