Document the Phase 9 security baseline
Add source-reviewed operational guidance, update acceptance and roadmap references, consolidate agent state, and disable core dumps with silent panic reboot defaults.
This commit is contained in:
+52
-24
@@ -1,10 +1,10 @@
|
||||
# Implementation roadmap
|
||||
|
||||
This document tracks the implementation and hardware-validation plan for the ESP32 Serial Swiss Army Knife. It describes intended sequencing rather than a release schedule. A phase is complete only when its implementation, documentation, build validation, and relevant hardware tests have passed.
|
||||
This document tracks the implementation and hardware-validation plan for the ESP32 Serial Swiss Army Knife. It describes intended sequencing rather than a release schedule. Completion requires implementation, documentation and build validation, plus relevant hardware tests or an explicitly recorded user waiver. A waiver is not a hardware-test pass.
|
||||
|
||||
## Status legend
|
||||
|
||||
- **Complete** — implemented and validated on the target hardware.
|
||||
- **Complete** — implemented and accepted, with target-hardware validation or an explicit phase-specific user waiver recorded below.
|
||||
- **In progress** — implementation or validation is actively underway, but the overall phase is not complete.
|
||||
- **Implemented; validation pending** — code is present and builds, but the current implementation still needs the listed hardware checks.
|
||||
- **Planned** — accepted project direction, not yet implemented.
|
||||
@@ -38,8 +38,8 @@ These constraints apply across all phases:
|
||||
| 5B | Offline xterm.js WebSocket serial terminal | **Complete** |
|
||||
| 6 | Authenticated SSH serial transport | **Complete** |
|
||||
| 7 | Local display and button interface | **Complete** |
|
||||
| 8 | Role-based users and administrative access | **In progress (8A–8C complete; integrated web administration 8D planned)** |
|
||||
| 9 | Security and production hardening | **Planned** |
|
||||
| 8 | Role-based users and administrative access | **Complete** |
|
||||
| 9 | Small intermittent-use security baseline | **Complete (user signoff 2026-09-18; new hardware check waived)** |
|
||||
| 10 | Authenticated, rollback-capable OTA | **Planned** |
|
||||
| 11 | BLE serial transport and provisioning evaluation | **Planned** |
|
||||
| 12 | Advanced network integration | **Under evaluation** |
|
||||
@@ -200,32 +200,60 @@ Completed implementation (8A–8C target-hardware validated; 8D explicitly accep
|
||||
- Typed Serial, Accounts/password/authorized-key, Network/Wi-Fi/mDNS, Display, Broker and SSH settings; confirmed HTTPS stop/restart/reboot and HTTPS/SSH identity rotation. Canonical owners compare reserved service/identity/target generations, preserve commit/failure semantics and expose no stored secrets. Admitted work is not cancelled by later logout/timeout; uncertain mutations are never automatically replayed.
|
||||
- Accessible Serial/Wi-Fi/client/writer contextual controls reuse existing controllers. Writer transfer is explicit and generation-safe; one writer, isolated observers, binary transparency, UART0 administrative recovery and network-independent native USB UART1 remain invariants.
|
||||
- Scope removed by user: **8D.15** dedicated typed network diagnostics (shell diagnostics retained) and the unimplemented **8D.19** ordinary browser-session/native-USB controls (SSH controls retained). Browser identity reset/recovery/export is excluded; canonical recovery remains available.
|
||||
- Prior final build PASS: **100,556 B linked RAM / 1,828,573 B flash, CPU 160 MHz**. Earlier combined binary WebSocket-send fix was explicitly accepted at **160 MHz / 230400 baud with full client mix including browser admin**. Latest overall sign-off does not invent individual fault/soak/duration results or a latest exact zero-drop comparison.
|
||||
- Latest loaded internal/DMA minima **2,052 / 460 B** remain a conservative transient-headroom follow-up, not a blocker reopening acceptance or an approved reserve. See [acceptance and telemetry](web_administration_acceptance.md), [current contracts](web_administration.md), [regression procedures](user_administration_tests.md#integrated-web-administration-regression-procedure) and [legacy compatibility](legacy_credential_removal.md).
|
||||
- Acceptance evidence and legacy storage compatibility are condensed below. [Web administration](web_administration.md) retains current API/owner/recovery contracts; [regression procedures](user_administration_tests.md#integrated-web-administration-regression-procedure) retain practical checks, not claims of execution.
|
||||
|
||||
## Current and planned phases
|
||||
#### Phase8 acceptance evidence
|
||||
|
||||
Phase 8 is complete for its accepted scope. Phase 9 is next **only on a separate user request**; later work remains planned or under evaluation. Optional features must not weaken completed serial and recovery paths. General release gates below remain guidance for future work, not claims that every listed fault, soak or reserve measurement was individually performed for 8D.
|
||||
**8D.22 signoff, 2026-09-13:** the user explicitly accepted the tested firmware, superseding earlier per-slice target, review and integration gates for the retained scope above. Earlier scoped acceptance covered browser login/logout, shared administration, Serial/account/Network presentation and legacy cleanup. Prior host regression suites and independent reviews were reported passing; this consolidation asserts no new build or test execution.
|
||||
|
||||
Prior final Phase 8 build: **PASS, 100,556 B linked RAM / 1,828,573 B flash, CPU 160 MHz**. The combined binary WebSocket-send fix was separately accepted at **160 MHz / 230400 baud with the full client mix including browser admin**; retain combined send and bounded failed-send isolation, not the earlier frequency-only experiment. Ordinary HTTPS idle cleanup was accepted, not every possible admission failure or owner delay.
|
||||
|
||||
Latest reported Phase 8 telemetry (bytes):
|
||||
|
||||
| Sample / capability | Free | Minimum free | Largest block |
|
||||
|---|---:|---:|---:|
|
||||
| Boot internal 8-bit | 59,808 | 58,840 | 31,744 |
|
||||
| Boot internal DMA | 52,052 | 51,084 | 31,744 |
|
||||
| Boot PSRAM | 8,196,968 | 8,183,972 | 8,126,464 |
|
||||
| Loaded after burst, internal 8-bit | 31,508 | 2,052 | 18,432 |
|
||||
| Loaded after burst, internal DMA | 23,752 | 460 | 18,432 |
|
||||
| Loaded after burst, PSRAM | 8,136,624 | 8,065,972 | 7,995,392 |
|
||||
|
||||
Loaded SSH minimum-free stack: **15,028 B**. This sample had two SSH sessions (serial/admin), two serial WebSockets and USB, with SSH as writer. Browser admin had been used then closed; it was **not active in this capture**. Web send/queue/protocol and SSH IO errors were zero; one SSH handshake failure and one session revocation were reported without an established cause.
|
||||
|
||||
**Limits, not reopened acceptance gates:** missing latest broker/serial counters prevent an exact zero-drop or byte-integrity inference. No latest browser-admin-active full-mix, individual fault-injection, exact duration, soak, cleanup-cycle or reserve-floor result is inferred. Internal/DMA lifetime minima remain a nonblocking transient-headroom follow-up, not approved reserves or proof of allocation failure: capability pools overlap and per-region minima may be non-simultaneous, so do not sum them. HTTPD/dispatcher stack margins, correlated peaks and allocation-reserve policy remain unmeasured/unapproved here. SDK TLS `-0x004C` means NET_RECV_FAILED, not an OOM diagnosis; stale cookies as the cause of two boot authentication failures remain unconfirmed.
|
||||
|
||||
#### Phase8 legacy credential compatibility
|
||||
|
||||
Cleanup was accepted **2026-09-08**, with unchanged certificate fingerprint and usable preexisting test users. This is not an unreported provisioning, recovery, power-loss or all-key pass.
|
||||
|
||||
- **Users:** missing `user_db/database` is committed as an empty v1 database, with no legacy account/password import. First-admin provisioning is physical UART0 `user add <username> admin` (optional `--generate`); entered passwords use hidden confirmation and generated passwords appear once. UART0-only `user recover --force` rebuilds only an unavailable database empty, refuses healthy databases (including empty ones), and leaves other configuration and TLS/SSH identities untouched; follow with normal `user add`.
|
||||
- Valid existing v1 user bytes load without rewrite or changes to accounts, roles, IDs, authentication generations, verifiers or authorized keys; migrated `user` accounts are not promoted. Private `v1_admin_marker` preserves the old wire byte and is derived from admin count on mutation, not a new role/schema/public bootstrap state. Public bootstrap and legacy migration/synchronization APIs are removed; final-admin protection, conditional mutations, principal currentness and targeted revocation remain.
|
||||
- **TLS:** at unchanged `web_sec/material`, a private reader validates the **1,392-byte v1** layout and migrates to **1,340-byte TLS-only v2**, preserving exact private-key DER, certificate DER, SHA-256 fingerprint and material generation—not rotating identity. The candidate is validated and committed before publication; temporary credential-bearing input is wiped. Malformed/unknown material, read/crypto failures and migration write/commit failures fail closed without regeneration or overwrite; truly missing material may be generated normally.
|
||||
- `web credentials show`, `web credentials rotate --force` and `user bootstrap` are removed. `web certificate info` / `web certificate rotate --force` retain frontend policy. `web reset --force` replaces TLS identity only, not accounts/passwords/verifiers; replacement/restart may close browser sessions and require renewed certificate trust/login, without revoking unrelated SSH sessions. No public legacy credential reader, display, rotation or authentication path remains.
|
||||
- **Downgrade/security:** older v1-only firmware cannot read v2 HTTPS storage; user-database v1 compatibility does not make HTTPS downgrade safe or restore removed credentials. Logical NVS replacement is **not secure flash wiping**: historical plaintext legacy credentials may remain, Wi-Fi secrets and TLS/SSH private keys remain unencrypted, and PBKDF2 verifiers permit offline guessing. No erase is required or authorized by cleanup.
|
||||
- Prior host tests used NVS fault doubles and a generated identity wrapped in the historical layout, **not an independently captured old-device fixture**; they do not prove flash durability or power-loss behavior. Focused runners: `tests/web_security/run.py`, `tests/admin_console_boundary/accounts.py`, `tests/admin_ssh_policy/run.py`. See [legacy-removal regressions](user_administration_tests.md#current-legacy-removal-regression-procedure).
|
||||
|
||||
### Phase 9 — Security and production hardening
|
||||
|
||||
Harden stored credentials, boot integrity, update trust, failure diagnostics, and dependency maintenance. This phase must be staged carefully because some ESP32 eFuse operations are irreversible.
|
||||
**Complete — user signoff 2026-09-18; new hardware check explicitly waived.** Reduced scope: an intermittently powered wireless serial cable on a trusted network, not a permanently exposed Internet service. The earlier extensive Phase 9 implementation was abandoned in favor of the accepted pre-Phase-9 baseline. Short exposure reduces risk; it does not make upstream defects harmless or certify this firmware secure.
|
||||
|
||||
Planned work:
|
||||
Small implementation:
|
||||
|
||||
- Define development, test, and production security profiles.
|
||||
- Enable and validate NVS encryption for Wi-Fi, HTTPS, and SSH material.
|
||||
- Evaluate flash encryption and PSRAM encryption for the target hardware and threat model.
|
||||
- Enable secure boot with a documented signing-key and recovery process.
|
||||
- Define secret-safe core-dump handling; do not expose credentials or session material in diagnostics.
|
||||
- Audit key and session-material zeroization across application, mbedTLS, wolfSSL, and wolfSSH allocations.
|
||||
- Add authentication throttling, lockout/backoff, and security-event counters where appropriate.
|
||||
- Review TLS/SSH algorithms, certificate policy, password policy, and browser security headers.
|
||||
- Review current wolfSSL/wolfSSH and ESP-IDF security releases instead of treating integration versions as permanent pins.
|
||||
- Recheck all third-party licenses and preserve source/license notices.
|
||||
- Document provisioning, key rotation, factory reset, backup, recovery, and device decommissioning.
|
||||
- Keep the baseline PlatformIO/ESP-IDF and managed-library versions and upstream sources. No generated dependency patches, custom SSH state machine, crypto-policy replacement, allocator hooks, or SDK migration in this phase.
|
||||
- Explicitly disable core dumps and select silent panic reboot through standard `sdkconfig.defaults` options. Keep ordinary logging; reduced crash diagnostics are a deliberate trade-off. Existing generated configurations must be checked because defaults do not override saved selections.
|
||||
- Follow the [operational checklist](security_operations.md): unique credentials, trusted identity verification, no Internet forwarding, networking off/power off after use, and existing rotation/recovery procedures.
|
||||
- Review upstream maintenance releases occasionally and before planned upgrades. Evaluate reachable issues against actual use; any stock dependency upgrade is a separate build/device-tested task, not a reason to accumulate local backports. Preserve existing license notices.
|
||||
|
||||
Security features must be validated first on expendable hardware before any irreversible production configuration is recommended.
|
||||
**Excluded:** secure boot, NVS/flash/PSRAM encryption, eFuse changes, partition changes, exhaustive CVE/zeroization audits, new authentication throttling, and production certification. Existing authentication, session isolation and recovery behavior remain unchanged. There is no protection against physical extraction or replacement of firmware. OTA image trust remains a separate Phase 10 design.
|
||||
|
||||
**Host/build evidence (2026-09-18):** normal `pio run` passed on PlatformIO 6.12.0 / ESP-IDF 5.5.0 with the original 20241119 toolchains: **94,196 B linked RAM / 1,828,565 B flash**. Generated configuration confirms no dumps and silent panic reboot; compilation inputs contain none of the abandoned Phase 9 source overlays or wolf crypto guards. No device operation or target validation was performed.
|
||||
|
||||
**Acceptance (2026-09-18):** the user signed off this small Phase 9 and explicitly waived a new whole-phase hardware check of boot, UART0 recovery, native USB serial, HTTPS/SSH login and normal serial sharing. Application code is unchanged, **but crash-diagnostic defaults changed**; this is a waiver, not evidence of target or panic-path validation. The prior build/configuration evidence above remains the validation record; no new hardware test was performed. A separately controlled panic check with synthetic secrets remains optional, not a completion gate; do not add a production panic endpoint or erase flash. Disabling dumps does not erase old contents.
|
||||
|
||||
## Current and planned phases
|
||||
|
||||
**Phases 8 and 9 are complete** for their accepted scopes; Phase 9 completion includes the explicit new-hardware-check waiver above. Phase 10 is planned, not implemented; later work remains planned or under evaluation. Optional features must not weaken completed serial and recovery paths. General release gates below guide future work, not claims that every fault, soak or reserve measurement was performed for Phase 8 or that Phase 9 received new target validation.
|
||||
|
||||
### Phase 10 — Authenticated OTA and rollback
|
||||
|
||||
@@ -233,7 +261,7 @@ Use the existing dual 4 MiB application slots and `otadata` partition to add saf
|
||||
|
||||
Planned work:
|
||||
|
||||
- Accept only authenticated, integrity-checked firmware images compatible with the selected secure-boot policy.
|
||||
- Accept only authenticated, integrity-checked firmware images under an explicit image-signature/trust policy; secure boot is not enabled by Phase 9.
|
||||
- Support an update path that does not expose a plaintext management endpoint.
|
||||
- Stream downloads with bounded RAM use and explicit progress/error reporting.
|
||||
- Preserve serial, Wi-Fi, HTTPS, and SSH configuration across successful updates.
|
||||
@@ -286,7 +314,7 @@ Before enabling storage, define corruption recovery, wear limits, quotas, atomic
|
||||
|
||||
## Cross-phase release gates
|
||||
|
||||
Every phase should satisfy the following before being marked complete:
|
||||
Every phase should satisfy the following before being marked complete, with any user-waived check explicitly recorded in that phase rather than reported as passed:
|
||||
|
||||
1. A clean release build for the ESP32-S3 N16R8 target.
|
||||
2. No new compiler or project diagnostics attributable to the change.
|
||||
|
||||
Reference in New Issue
Block a user