Harden cleanup and reduce internal RAM use. Phase 8C nearly validated

and somewhat stable.
This commit is contained in:
2026-08-31 03:33:43 +02:00
parent 868e9ebc23
commit 6ad6c00d68
12 changed files with 169 additions and 68 deletions
+1 -1
View File
@@ -190,5 +190,5 @@ Hardware diagnostics are synchronous console commands. RS-232 tests own the phys
- Library/hardware ownership is centralized: serial task owns UART1 while running, display service owns I2C/framebuffer, the SSH owner task owns post-initialization wolfSSH runtime calls, and the console dispatcher owns `esp_console_run()`.
- Password authentication performs PBKDF2 outside the user-database mutex and revalidates afterward. Some password mutation paths currently derive verifiers while holding the mutation lock; do not generalize the authentication locking pattern without checking the exact path.
- Avoid holding service/database/broker locks across I2C, network sends, or other potentially long operations unless the existing contract explicitly requires it. Preserve the existing broker-before-serial lock order.
- Broker and selected cryptographic allocations prefer PSRAM but can fall back to internal RAM. FreeRTOS control structures and task stacks intentionally remain internal where flash/cache-disable safety matters.
- Serial RX/TX stream payloads, broker per-client payloads, the transactional user-database candidate, and selected cryptographic allocations prefer PSRAM with internal fallback. The live user database, FreeRTOS control structures, UART driver buffers, and task stacks remain internal where deterministic/cache-disable access matters.
- The build disables wolfSSL ESP32 AES/SHA acceleration, and the HTTPS path uses software AES for PSRAM-backed records. This preserves the validated workaround for uncoordinated mbedTLS/wolfSSL hardware-crypto locks and a prior mbedTLS external-RAM DMA watchdog stall.
+2 -2
View File
@@ -30,7 +30,7 @@ This is a semantic map, not a complete file inventory. Start here, then read the
- Normal data caller: `session_broker`; USB, WebSocket, role-`user` SSH, console, and local UI also call serial lifecycle/configuration APIs as appropriate
- Dependencies: ESP-IDF UART driver, `board_pins.h`, NVS
- Ownership: the cooperative owner arbitrates active diagnostics (`PHASE0`) against the service (`SERVICE`); boot-time static-safe GPIO initialization and service-owned static-mode restoration are explicit exceptions. Unsafe cleanup marks `FAULT` until reboot.
- Lifecycle: stop/reconfiguration discards serial-service RX/TX and task-local pending bytes, but leaves broker clients, writer ownership, events, and already-fanned output intact.
- Lifecycle: stop/reconfiguration discards serial-service RX/TX and task-local pending bytes, but leaves broker clients, writer ownership, events, and already-fanned output intact. The 16 KiB RX and 8 KiB TX payloads prefer PSRAM; their FreeRTOS controls and UART driver storage remain internal.
## Session broker
@@ -91,7 +91,7 @@ This is a semantic map, not a complete file inventory. Start here, then read the
- Interfaces: init/migration/recovery, authenticate, principal-currentness, account/password/role/key mutations, snapshots
- Called by: web and SSH authentication/currentness checks and console administration
- Dependencies: NVS, secure random, mbedTLS cryptography; after a committed command-layer mutation, best-effort web/SSH revocation calls supplement authoritative transport currentness checks
- Ownership: database mutex protects live records; password authentication runs PBKDF2 outside it and revalidates afterward, while mutation locking must be checked per operation.
- Ownership: database mutex protects the internal live record and PSRAM-preferred transactional candidate; password authentication runs PBKDF2 outside the mutex and revalidates afterward, while mutation locking must be checked per operation.
- Authorization: UART0 exclusively owns initial administrator bootstrap and unavailable-database recovery; current admins may use admin SSH for other commands unless handler policy denies them. HTTPS currently treats both roles alike.
- Constraint: final administrator cannot be deleted or demoted; transport principals must be rechecked after mutations.
+9 -2
View File
@@ -13,6 +13,15 @@ Based on checked-in source plus `README.md` and `docs/roadmap.md`:
- Security/production hardening, OTA, BLE evaluation, advanced networking, and optional filesystem features remain future roadmap work.
- Reserved OTA, coredump, NVS-key, and storage partitions do not imply those runtime features are implemented.
## Recent memory audit
- Fixed failed-initialization ownership leaks for wolfSSH, partial HTTPS startup, and TinyUSB teardown. Failed teardown now retains ownership and blocks unsafe duplicate initialization.
- Serial-service RX/TX stream payloads (16 KiB and 8 KiB effective capacity) now prefer PSRAM with internal fallback; FreeRTOS controls and UART driver buffers remain internal.
- The 5,360-byte transactional user-database candidate now prefers PSRAM with internal fallback while the live database remains internal. Candidate contents are wiped after each transaction and wiped/freed on initialization or recovery failure.
- UART and admin-SSH completion formatter buffers were reduced from 2 KiB to 1 KiB each; current worst-case output is 890 bytes and overflow remains fail-closed.
- Linked RAM fell from 99,508 to 92,188 bytes (7,320 bytes). PSRAM placement of serial payloads additionally removes about 24 KiB of normal internal-heap pressure on the target.
- `pio run` passes. A preliminary target run reports significantly more free memory and stable, improved operation after these changes. This is useful evidence but not completion of Phase 8C validation.
- Remaining targeted checks include stored/migrated/recovered user-database mutations, USB enumeration, HTTPS start/stop failure recovery where injectable, SSH initialization/login, completion display, and sustained multi-transport serial traffic while checking `memory` telemetry.
## Clearly incomplete or transitional areas
@@ -21,12 +30,10 @@ Based on checked-in source plus `README.md` and `docs/roadmap.md`:
- Browser authentication still uses HTTP Basic; integrated login/logout sessions are planned.
- NVS encryption, secure boot/flash encryption review, authentication rate limiting, production certificate/provisioning policy, and OTA are not implemented.
## Known inconsistencies
These observations should be checked when touching the relevant area; they are not automatically bugs requiring unrelated cleanup.
- Some source comments still call shared commands UART0-only or call the current local status/control task read-only.
- `USER_DATABASE_LOAD_EMPTY` is only an initialization/failure sentinel at the checked-in revision: every successful `user_database_init()` path returns `STORED` or `MIGRATED_LEGACY`, so `main.c`'s successful "new empty" log branch is unreachable.
- SSH startup is currently gated on successful `web_security` initialization even though SSH uses separate host-key material. **Needs verification:** whether this coupling is intentional recovery policy or an accidental startup dependency.
+2 -2
View File
@@ -96,9 +96,9 @@ Only constraints supported by implementation or current project documentation be
**Decision:** Application settings, users, and identities use separate fixed/versioned NVS blobs. Serial, Wi-Fi, and local-UI working edits are RAM-only until explicitly saved. User mutations and HTTPS/SSH identity changes commit directly as part of the operation. Invalid ordinary configuration generally selects RAM defaults without erasing storage; malformed security material fails closed and needs explicit reset.
**Rationale/evidence:** Serial, Wi-Fi, local UI, web security, users, and SSH security each validate schema/size and own their namespace. User/security mutations build and validate candidate state before committing it; security modules avoid silently replacing an established identity.
**Rationale/evidence:** Serial, Wi-Fi, local UI, web security, users, and SSH security each validate schema/size and own their namespace. User/security mutations build and validate candidate state before committing it; security modules avoid silently replacing an established identity. The live user database remains internal while its 5,360-byte candidate is a persistent PSRAM-preferred allocation with internal fallback and is wiped after every transaction.
**Consequence for future changes:** Add schema versions and transactional candidate validation. Do not overwrite unknown records automatically; provide explicit migration/reset behavior. Preserve the distinct persistence contracts: explicit save/load/default/reset for working configuration and per-blob commit-before-live-install for user and identity mutation. Pre-bootstrap legacy credential rotation spans `web_sec/material` and `user_db/database`, is not cross-namespace atomic, and relies on boot reconciliation after interruption.
**Consequence for future changes:** Add schema versions and transactional candidate validation. Do not overwrite unknown records automatically; provide explicit migration/reset behavior. Preserve the distinct persistence contracts: explicit save/load/default/reset for working configuration and per-blob commit-before-live-install for user and identity mutation. Keep candidate ownership mutex-local and wipe/free it on initialization or recovery failure. Recheck external-buffer staging in the flash/NVS implementation when upgrading from the pinned ESP-IDF 5.5 baseline. Pre-bootstrap legacy credential rotation spans `web_sec/material` and `user_db/database`, is not cross-namespace atomic, and relies on boot reconciliation after interruption.
**Relevant files:** `src/serial_config.c`, `src/wifi_config.c`, `src/local_ui_config.c`, `src/web_security.c`, `src/user_database.c`, `src/ssh_security.c`