Migrate to IDF 5.5.3 candidate
Pin PlatformIO packages and toolchains, rebase protected SDK overrides, and add WebSocket receive regression coverage. Document isolated candidate validation, archive provenance, and remaining gates.
This commit is contained in:
@@ -78,7 +78,7 @@ UART RX -> serial-service RX stream -> broker task
|
||||
|
||||
The broker drains serial input even with no clients. A full client output stream drops only that client's copy and updates drop counters; it does not block UART reception or other clients.
|
||||
|
||||
Binary serial WebSocket output uses the IDF-5.5.0-pinned `web_httpd_ws_send_binary` adapter: one owner-only session-override send of a bounded header+payload copy (516 bytes of local scratch, 512-byte payload). Existing generation validation and one outstanding work item per slot remain required. Non-full sends immediately replace that session's send override with a reject-only guard, mark it closing and shut down the socket before deferred cleanup; this prevents automatic SDK control replies from reentering TLS after incomplete output. HTTPD retains TLS destruction ownership. Text/control and admin output retain the SDK sender. One send call does not imply one TLS record, packet or peer receipt; the user signed off drop-free230400-baud full-client-mix operation at160MHz. Runtime stack-margin validation remains a separate follow-up; evidence and acceptance scope are in `current-state.md`.
|
||||
Binary serial WebSocket output uses the IDF-5.5.3-pinned `web_httpd_ws_send_binary` adapter: one owner-only session-override send of a bounded header+payload copy (516 bytes of local scratch, 512-byte payload). Existing generation validation and one outstanding work item per slot remain required. Non-full sends immediately replace that session's send override with a reject-only guard, mark it closing and shut down the socket before deferred cleanup; this prevents automatic SDK control replies from reentering TLS after incomplete output. HTTPD retains TLS destruction ownership. Text/control and admin output retain the SDK sender. One send call does not imply one TLS record, packet or peer receipt; the user signed off drop-free230400-baud full-client-mix operation at160MHz. Runtime stack-margin validation remains a separate follow-up; evidence and acceptance scope are in `current-state.md`.
|
||||
|
||||
Active-client counter snapshots expose ID/type/pending/HWM/UART/queued/read/dropped. HWM is maintained independently of web tracing and counter clear seeds current occupancy. Read means transport handoff, not peer receipt. Disconnect removes the client row while global totals retain traffic and unread-output discards. Independent default-disabled web performance capture uses two fixed slot records and nonwrapping epoch/generation fences; toggles fence in-flight samples, disable freezes aggregates, clear preserves enable state. Binary-only timestamps bound reservation-entry to callback-entry before locking, synchronous send calls and completion to broker-read return; idle and intervening work prevent scheduler-only/backlog-at-completion claims. No new instrumentation allocations or scheduling/buffer changes. Exact fields, overhead and capture contracts: [throughput diagnostics](../web_throughput_diagnostics.md).
|
||||
|
||||
@@ -108,7 +108,7 @@ TinyUSB callbacks enqueue/copy data and state; the transport task owns broker li
|
||||
|
||||
Cookie login/logout replaces Basic/cache. Digest-only records carry copied principals, CSRF state, absolute expiry and nonreused originating-session IDs. Strict same-origin/CSRF mutations and session/principal checks gate admission; logout invalidates its session before transport cleanup, account mutations invalidate only the affected account, and ongoing currentness is authoritative. Authentication initialization failure gates HTTPS; failed start/accepted stop wipes records. A non-consuming quota/epoch check rejects exhausted login requests before body receive; verification reservation remains post-parse and only that reservation charges the existing fixed window. RNG/SHA/database calls run outside short spinlocks with post-call epoch/identity revalidation. [Authentication contract](../web_administration.md#authentication-and-admission).
|
||||
|
||||
`web_httpd_adapter` is the sole private IDF 5.5.0 boundary for duplicate headers, admission-before-101, consumed-scratch wiping, staged optional URI registration, combined binary sends and owner-only idle sweeps. Re-audit its version guard on SDK upgrades. HTTPD debug logging must not expose headers/tickets. `web_diagnostics` independently observes public post-TLS callbacks using six metadata records and a default-disabled 32-event ring; it cannot see preaccept/in-progress/failed TLS. [Admission diagnostics](../web_admission_diagnostics.md).
|
||||
`web_httpd_adapter` is the sole private IDF 5.5.3 boundary for duplicate headers, admission-before-101, consumed-scratch wiping, staged optional URI registration, combined binary sends and owner-only idle sweeps. Re-audit its version guard on SDK upgrades. The 5.5.3 rebase retains private-layout/ownership contracts and adds a pinned WS receive correction so negative fixed-header reads cannot pass unsigned length comparisons; see [rebase evidence](../idf_553_rebase_review.md). HTTPD debug logging must not expose headers/tickets. `web_diagnostics` independently observes public post-TLS callbacks using six metadata records and a default-disabled 32-event ring; it cannot see preaccept/in-progress/failed TLS. [Admission diagnostics](../web_admission_diagnostics.md).
|
||||
|
||||
`web_httpd_idle` uses one one-second timer, six rows and at most one queued owner probe. Current-owner shutdown follows 15 seconds of observed ordinary idle, exempting actual WebSockets/async/pending input. Owner delays prevent hard timeout guarantees; accepted-but-lost work stays reserved until successful destruction, failed stop retains ownership. [Idle lifecycle contract](../https_idle_cleanup.md).
|
||||
|
||||
|
||||
@@ -20,19 +20,19 @@ This is a semantic map, not a complete file inventory. Start here, then read the
|
||||
## Source-pinned dependency corrections (Phases 9C–9D)
|
||||
|
||||
- Files: root `CMakeLists.txt` (after `project()`), `cmake/security_overrides.cmake`, `tools/security_overrides.py`; tests: `tests/sdk_security_overrides/run.py --build-dir .pio/build/esp32-s3-devkitc-1-n16r8`.
|
||||
- Build input is the exact-hash original **plus checked-in edits**, not installed source alone. Generated copies replace eight target C sources plus one PUBLIC forced header without modifying SDK/managed components: HTTPS cleanup/private-key release, HTTPD scratch lifetime/null first read, ESP-TLS server-only protocol list, wolfSSH password bounds/payload wiping plus bounded IGNORE/service/string parsing, channel-window overflow rejection and ECC/Ed25519 label/exact-signature framing, DHCP option bounds (CVE-2026-45160), TLS 1.2 EMS error propagation (CVE-2026-50581), and X.509 OID allocation failure (CVE-2026-34874). The mbedTLS entries explicitly select validated nested `mbedtls`/`mbedx509` targets, not the component wrapper. Original notices and compile properties retained; all outputs carry the 2026-09-15 baseline modification notice and wolfSSH outputs add the 2026-09-16 ordering/provenance notice; source/hash/target ambiguity fails configuration. Never hand-edit generated copies or silently repin.
|
||||
- Build input is the exact-hash original **plus checked-in edits**, not installed source alone. Generated copies replace nine target C sources plus one PUBLIC forced header without modifying SDK/managed components: HTTPS cleanup/private-key release, HTTPD scratch lifetime/null first read, ESP-TLS server-only protocol list, wolfSSH password bounds/payload wiping plus bounded IGNORE/service/string parsing, channel-window overflow rejection and ECC/Ed25519 label/exact-signature framing, DHCP option bounds (CVE-2026-45160), TLS 1.2 EMS error propagation (CVE-2026-50581), X.509 OID allocation failure (CVE-2026-34874), and five signed WS fixed-header size comparisons on IDF 5.5.3 (982 cases / 10 mutation checks against generated code). The mbedTLS entries explicitly select validated nested `mbedtls`/`mbedx509` targets, not the component wrapper. Original notices and compile properties retained; all outputs carry the 2026-09-15 baseline modification notice and wolfSSH outputs add the 2026-09-16 ordering/provenance notice; source/hash/target ambiguity fails configuration. Never hand-edit generated copies or silently repin.
|
||||
- Xtensa crypto policy: root `CMakeLists.txt` sets `CURVE25519_SMALL`/`ED25519_SMALL` before component parsing; `cmake/wolf_crypto_policy.cmake` PUBLIC-propagates `cmake/wolf_crypto_policy.h` to wolfSSL consumers. PUBLIC `WOLFSSL_VALIDATE_ECC_IMPORT` and `WOLFSSL_ECDHX_SHARED_NOT_ZERO` enable existing P-256 import and X25519 all-zero-result checks. The resolved-settings guard requires these checks and both small implementations, rejects reviewed ECC validator-disabling/hardware-stub configurations, X25519 blinding and unreviewed Curve448/Ed448 enablement. Tests: `tests/wolf_crypto_policy/run.py` (strict actual production flags, real vendor crypto/ASN vectors and independently specified exact source deltas; candidate injection is not production evidence). [Key-validation review](../ssh_key_validation_review.md) records effective flags, caller/API limits and unmeasured validation cost. [Wolf review](../wolf_security_review.md) distinguishes implemented restricted ordering/crypto/parser mitigations from pending target gates and explicit dormant-profile exclusions; [IDF review](../idf_security_review.md) records three backports, finite six-finding applicability completion and first-page feature exclusions, not exhaustive advisory safety.
|
||||
- Parser tests: `tests/wolfssh_parser_contract/run.py`, [scope and exclusions](../../tests/wolfssh_parser_contract/README.md): 3,258 base cases per each of two stack modes plus five channel profiles in both modes (2,737/profile/mode for TERM, TERM+SHELL, TERM+SHELL+AGENT; 2,735 for neither/SHELL-only); 11 base + 18 name/length + 2 application-gate mutations rejected. Guard pages/UBSan and crypto doubles test parsing/gating, not arithmetic. [Finite review](../ssh_parser_remaining_review.md) closes exact CHANNEL_FAILURE recipient, ssh-userauth service and nine channel-name dispatch checks; PR899 client skips remain unchanged behind role/ordering gates, forwarding disabled, generic caller trace dispositioned but APIs unpatched. Unknown/trailing behavior is retained; not exhaustive safety. Ordering has its separate suite below.
|
||||
- Latest supplied parent `pio run` PASS: **94,340 B linked RAM / 1,768,901 B flash (+200 B versus 1,768,701 B)**. Final parent **all seven suites PASS**: ordering `--interop` (8,028 checks / seven rejected mutations / 12 exact-256-KiB sessions, clean channel close and transport EOF), SDK overrides `--build-dir .pio/build/esp32-s3-devkitc-1-n16r8`, auth (135), protocol, strict crypto, notices (30), and parser (3,258 × two modes plus channel profiles; 11 + 18 + 2 rejected mutations). Interop used unsandboxed approval only for local AF_UNIX sockets; no remote network/device operation. Review's misplaced EOF guard is corrected in `SendChannelEof` before lookup/serialization/state mutation; follow-up verification found no scoped blocker. Not target/runtime-reserve evidence.
|
||||
- Historical IDF 5.5.0 supplied parent `pio run` PASS: **94,340 B linked RAM / 1,768,901 B flash (+200 B versus 1,768,701 B)**. Final parent **all seven suites PASS**: ordering `--interop` (8,028 checks / seven rejected mutations / 12 exact-256-KiB sessions, clean channel close and transport EOF), SDK overrides `--build-dir .pio/build/esp32-s3-devkitc-1-n16r8`, auth (135), protocol, strict crypto, notices (30), and parser (3,258 × two modes plus channel profiles; 11 + 18 + 2 rejected mutations). Interop used unsandboxed approval only for local AF_UNIX sockets; no remote network/device operation. Review's misplaced EOF guard is corrected in `SendChannelEof` before lookup/serialization/state mutation; follow-up verification found no scoped blocker. Not target/runtime-reserve evidence.
|
||||
- Restricted **CVE-2025-14942** ordering correction implemented: `tools/wolfssh_order/delta.json` supplies exact executable edits to `internal.c`, `ssh.c` and `internal.h`; [README](../../tools/wolfssh_order/README.md) records PR793/819/840/855/921 prerequisite disposition, and `provenance.json` pins archived patch hashes/commit IDs. Existing X25519/P-256 KEX only; independent SELF/PEER bits, expected-message/auth gates and exactly-once queued NEWKEYS under WANT_WRITE. **EXT_INFO intentionally disabled, no `server-sig-algs`; `extInfoSent` stays zero.** No full upstream backport or dependency upgrade. CMake propagates the generated ABI header BEFORE PUBLIC plus PUBLIC forced include with stale-header rejection; joined `-include/path` flags preserve ordering/crypto guards through PlatformIO sorting/deduplication.
|
||||
- Ordering tests: [tests/wolfssh_order_contract](../../tests/wolfssh_order_contract/README.md), full generated C/real wolfCrypt, **8,028 checks / seven rejected mutations**; `pio_adapter.py` validates installed PlatformIO/SCons flags with a real Xtensa consumer. Initial interop failed a harness close race despite early PASS output; fixed harness awaits peer close and transport EOF and independently reaps the server via fd-passing proxy. Final agent `--interop --interop-repeat 3`: **36/36 sessions**, exact **256 KiB** each, **two or ten** key exchanges and clean client/server exits. [Remaining gates](../wolf_security_review.md#restricted-ordering-correction-and-remaining-gates): target cleanup during rekey, no-EXT_INFO client compatibility, resource/timing and whole-phase validation; no device operation or phase sign-off.
|
||||
- [Finite dependency advisory snapshot](../dependency_advisory_coverage.md): all 22 IDF index entries across three pages plus named managed/browser channels accounted for; declared search complete, not exhaustive clearance. **PMF-capable SoftAP security blocker remains unfixed**; [Wi-Fi update plan](../wifi_security_update_plan.md) requires coherent vendor framework/toolchain/source/header/binary ABI bundles and override revalidation, not PMF/WPA3 weakening. Residual applicability questions and target gates remain.
|
||||
- [Bounded Phase 9 runner](../../tools/validate_phase9.py), [usage/evidence](../phase9_validation.md), fixtures `tests/phase9_validation/run.py`: 23 default commands; build and OpenSSH interop are independent opt-ins. After the owning agent fixed the stale console-boundary anchor, final supplied parent `CCACHE_DISABLE=1 python3 -B tools/validate_phase9.py` **PASS 23/23** and orchestrator fixtures **PASS 10 tests**. No build/interop this turn; historical build/interop evidence above is separate, not phase acceptance.
|
||||
- [Finite dependency advisory snapshot](../dependency_advisory_coverage.md): all 22 IDF index entries across three pages plus named managed/browser channels accounted for; declared search complete, not exhaustive clearance. **Fix-bearing Wi-Fi vendor bundle integrated in the validated IDF 5.5.3 candidate; radio-hardware closure pending**; [Wi-Fi update plan](../wifi_security_update_plan.md) requires coherent vendor framework/toolchain/source/header/binary ABI bundles and override revalidation, not PMF/WPA3 weakening. Residual applicability questions and target gates remain.
|
||||
- [Bounded Phase 9 runner](../../tools/validate_phase9.py), [usage/evidence](../phase9_validation.md), fixtures `tests/phase9_validation/run.py`: 23 default commands; build and OpenSSH interop are independent opt-ins. After the owning agent fixed the stale console-boundary anchor, final supplied parent `CCACHE_DISABLE=1 python3 -B tools/validate_phase9.py` **PASS 23/23** and orchestrator fixtures **PASS 10 tests**. That default run is historical. Current supplied parent explicit-candidate validation with `--interop --web-performance` **PASS 24/24**; [exact command and pre/post source equality](../phase9_validation.md#current-candidate-execution--2026-09-18). Fresh isolated `app-validated` build PASS: **95,552 B RAM / 1,749,493 B flash (+1,212 / −19,408 B)**. Default root build timed out at 200 seconds during installation before compilation, not a root build PASS. Root now pins platform 6.13.0 / IDF 5.5.3 / both toolchains 14.2.0+20251107; [integration](../idf_candidate_integration.md), [semantic rebase and resolved historical WS blockers](../idf_553_rebase_review.md). Version pins are not complete immutable dependency closure or phase acceptance.
|
||||
- Policy/evidence/limits: [library review](../security_library_review.md), [Phase 9C](../security_hardening.md#9c-library-cleanup-and-protocol-policy). Source-contract tests must locate and verify actual generated compilation inputs, not assume original vendor paths.
|
||||
|
||||
## Offline release notice assembly
|
||||
|
||||
- [Procedure and release gates](../release_packaging.md), [tool](../../tools/release_notices.py), catalog `third_party/release-notices/inputs.json`, scope/TLSF provenance under `third_party/release-notices/`; tests: `tests/release_notices/run.py` (supplied agent 36 fixture tests PASS). 75 explicit hash/size-pinned inputs; no recursive collection, fetch, build, asset regeneration or device access; output must be fresh and outside input roots.
|
||||
- [Procedure and release gates](../release_packaging.md), [tool](../../tools/release_notices.py), catalog `third_party/release-notices/inputs.json`, scope/TLSF provenance under `third_party/release-notices/`; tests: `tests/release_notices/run.py` (supplied agent 36 fixture tests PASS). 75 explicit hash/size-pinned inputs (nine IDF/toolchain notice entries semantically rebased for the candidate; the other 66 unchanged); no recursive collection, fetch, build, asset regeneration or device access; output must be fresh and outside input roots.
|
||||
- Supplied agent evidence: two actual 75-input bundles verified deterministic, each **77 files / 4,433,930 bytes**; no reassembly here. [Exact retained SVG provenance](../icon_provenance.md) resolves official 7.4.47 sources at `9e04201d4557e729822fb57f62a316c3dea1d4a8` (Google USB, Simran Wi-Fi); manual bitmap derivation and distinct mockup Wi-Fi path remain unresolved. The catalog pins `docs/icon_provenance.md`, not this code map or the edited overview docs; coordinate changes to pinned evidence, never silently repin. Recipient delivery, corresponding source, legal/radio-blob basis and whole-phase target gates remain open. A notice manifest is not a source archive, complete SBOM or release clearance.
|
||||
|
||||
## Secure randomness
|
||||
|
||||
@@ -2,6 +2,15 @@
|
||||
|
||||
Working memory, not an implementation timeline. Source is authoritative; begin with [code map](code-map.md), then [architecture](architecture.md) and [decisions](design-decisions.md).
|
||||
|
||||
## Phase 9D continuation — IDF 5.5.3 integration — 2026-09-18
|
||||
|
||||
- Initial working tree already contained platform6.13/framework3.50503/toolchains14.2.0+20251107 migration, exact version guards/hash rebases and new httpd_ws override; preserved/audited rather than overwrote. Complete official packages archived/verified under .pio/idf-candidate-5.5.3; all WiFi/PHY/coex/supplicant trees match pinned package. PMF/WPA3 unchanged. Vendor radio correction integrated in candidate, not hardware exploit/closure proof.
|
||||
- Existing eightC+header corrections all retained after semantic audit; new ninth C override corrects five WS signed-receive/unsigned-size comparisons introduced upstream. Dedicated982guard/canary cases plus fivecompile+fivebehavior mutations PASS. HTTPD private layout/owner and heap extent contracts re-audited5.5.3; no weakening guards. Root dependencies.lock updated only IDF5.5.0→5.5.3 to match actual resolver result; six managed dependencies unchanged.
|
||||
- Default root pio run timed out200s during sharedpackage installation before compilation; no normalrootbuildPASS claim/retry. Separate fresh app-validated stage uses complete verified archives and isolatedcore. Agent buildPASS50.51s **95,552 B linked RAM / 1,749,493 B flash** (+1,212RAM/−19,408flash vs old). Staging fresh-only explicitdestination avoids prior app overwrite; manifest includes transformedconfig hashes and records sole resolver lock change. All ninegeneratedC+PUBLICheader actualregistration verified; WS vendororiginal absent.
|
||||
- Final parent tools/validate_phase9.py with --build-dir .pio/idf-candidate-5.5.3/app-validated/.pio/build/esp32-s3-devkitc-1-n16r8 --idf-path .pio/idf-candidate-5.5.3/core/packages/framework-espidf --platformio-core-dir .pio/idf-candidate-5.5.3/core --interop --web-performance **24/24PASS**. Pre/post3237sourceinputs equal hash3a1af78c15cfdd02da1055a8957b4086f9018862b7aa1c7c52fd2401a1a0a031. Earlier5webcookie failures were stale5.5.0 fixture updated5.5.3; allpassnow. Parent helper5/runner16fixturesPASS. Independent recheck noactionableblocker, reran WS/performance/runner, verified archive/linkpaths/sourceequality/rootlock. No device validation.
|
||||
- Notice catalog semanticallyrebased9entries, other66unchanged;75inputs36testsPASS twoactualbundles77files4433930bytes. Evidence/docs updated with candidatenotrootbuild distinction and historical baselines. No full environmentclosure or legalclearance claim.
|
||||
- **Next:** normal package installation/build completion remains unverified after timeout; ask user before retrying longer. Isolated source/currentconfig technically validated; wholePhase9 radio/resource/recovery/noEXTclient/abuse/operations targetgates stillpending, release source/notice/legal obligations open. Candidate5.5.3 is newest available official PlatformIO5.5 package verified, not latest upstream maintenance or alladvisoriesclosed. No upload/erase/assets/secureboot/encryptedNVS changes or commits.
|
||||
|
||||
## Phase 9D continuation — broader snapshot / reproducible validation — 2026-09-16
|
||||
|
||||
- Initial Git clean. `docs/dependency_advisory_coverage.md` completes finite dated search of all22official IDF advisories/3pages plus specified managed/browser channels. **New current-profile blocker:** PMF-capable mixed WPA2/WPA3 SoftAP matches vendor management-frame memory-corruption path; installed7S3WiFi archives match original5.5. No exploit demonstrated. `docs/wifi_security_update_plan.md` verifies fix/release bundle differences and both callback uint8_t*→size_t* ABI changes. Verified reference IDF5.5.3 commit2c211b236707889e8400c4dc5644dd5c4ee071e0 contains fix; not deployment approval or automatic recommendation over newer releases. Do not C-only patch/swap one archive/disable PMF as guessed mitigation. Next coherent framework/toolchain/source/header/radio update evaluation with audited override rebase; PHY/coexistence/privateABI compatibility unresolved. Direct SDK AES-DMA/PSRAM applicability remains explicit review gap.
|
||||
|
||||
Reference in New Issue
Block a user