ESP32_Serial_Swiss_Army_Knife/docs/agent/code-map.md

# Code map

This is a semantic map, not a complete file inventory. Start here, then read the listed headers and only the implementation paths relevant to the task.

## Bootstrap and system composition

**Responsibility:** establish startup order, recovery behavior, configuration loading, service dependencies, and command registration.

- Files: `src/main.c`, `src/CMakeLists.txt`, root `CMakeLists.txt`, `platformio.ini`, `partitions.csv`, `src/idf_component.yml`; inspect targeted settings in `sdkconfig.defaults` when crypto, PSRAM, HTTPS/HTTPD, USB, or socket capacity matters
- Entry point: `app_main()`
- Called by: ESP-IDF runtime
- Dependencies: every subsystem initializer
- Lifecycle constraint: optional display/network failures should not remove UART0 administrative recovery or USB UART1 access; the custom administration frontend starts only after command registration.

## Secure randomness

**Responsibility:** provide the sole project-owned, mutex-serialized application DRBG, seeded before Wi-Fi/radio use.

- Files: `src/secure_random.{h,c}`
- Interfaces: `secure_random_init()`, random-byte helpers, `secure_wipe()`
- Called by: HTTPS material, SSH keys, users, Wi-Fi defaults, tickets, the HTTPS authentication cache, and the wolfCrypt seed callback
- Constraint: initialization order is security-significant; the DRBG deliberately avoids post-radio reseeding and fails closed at its generation limit. Do not add independent weak RNGs or radio-dependent early entropy paths.

## Physical RS-232 and serial service

**Responsibility:** protect the MAX3243/UART resource, own UART1 while running, buffer binary RX/TX, apply serial configuration, and expose status/counters.

- Files: `src/rs232_port_owner.{h,c}`, `src/serial_service.{h,c}`, `src/serial_config.{h,c}`, `src/serial_console.{h,c}`
- Interfaces: owner claim/release/fault; serial init/start/stop/read/write/configuration/snapshots; versioned NVS load/save
- Normal data caller: `session_broker`; USB, WebSocket, role-`user` SSH, console, and local UI also call serial lifecycle/configuration APIs as appropriate
- Dependencies: ESP-IDF UART driver, `board_pins.h`, NVS
- Ownership: the cooperative owner arbitrates active diagnostics (`PHASE0`) against the service (`SERVICE`); boot-time static-safe GPIO initialization and service-owned static-mode restoration are explicit exceptions. Unsafe cleanup marks `FAULT` until reboot.
- Lifecycle: stop/reconfiguration discards serial-service RX/TX and task-local pending bytes, but leaves broker clients, writer ownership, events, and already-fanned output intact. The 16 KiB RX and 8 KiB TX payloads prefer PSRAM; their FreeRTOS controls and UART driver storage remain internal.

## Session broker

**Responsibility:** mediate all transport access to the serial service; provide one writer lease and multiple isolated observers.

- Files: `src/session_broker.{h,c}`, `src/session_console.{h,c}`
- Interfaces: connect/disconnect, request/release/force writer, nonblocking read/write/event APIs, snapshots and counters; `session_broker_get_management_snapshot()` and `session_broker_assign_writer_current()` atomically project/compare target and lease generation. Exhausted 29-bit client generations retire; saturated lease generation preserves ordinary recovery. Tests: `tests/session_broker_diagnostics/run.py`.
- Called by: USB, web serial, role-`user` SSH, console tests, local UI snapshots/actions
- Dependencies: `serial_service`
- Data path: `transport -> broker -> serial service -> UART1`; reverse data is fanned out per client.
- Ownership: client IDs are slot/generation-safe; events are advisory and can drop, so use snapshots as authority.
- Lifecycle: one permanent task and eight preallocated client slots; slow output drops only for the affected client.
- Diagnostics: `broker counters` adds active-client ID/type/pending/HWM/UART/queued/read/dropped rows; clear seeds HWM from pending, disconnect removes rows but retains global discard accounting. `tests/session_broker_diagnostics/run.py`; capture before disconnect, never use consuming `broker read` as a probe. Semantics/recipe: `docs/web_throughput_diagnostics.md`.

## Native USB CDC

**Responsibility:** adapt TinyUSB CDC host state/data to one broker client.

- Files: `src/usb_cdc_transport.{h,c}`, `src/usb_console.{h,c}`
- Interfaces: `usb_cdc_transport_init()`, snapshots/counters, queued writer request/release
- Called by: startup, TinyUSB callbacks, console/local UI
- Dependencies: TinyUSB, broker, serial service
- Flow: `USB host <-> USB task <-> broker`
- Lifecycle: permanent owner task; broker client exists only while attached with host DTR asserted.
- Constraint: callbacks retain the latest host line coding only for diagnostics; it never reconfigures UART1. UART1 framing and speed remain controlled by the serial configuration and its explicit persistence commands.

## Web and WebSocket serial

**Responsibility:** serve authenticated HTTPS UI/API, issue WebSocket tickets, and adapt browser serial sessions to broker clients.

- Files: `src/web_server.{h,c}`, `src/web_serial_transport.{h,c}`, `src/web_ui.{h,c}`, `src/web_console.{h,c}`
- Ordinary HTTPS idle cleanup: `src/web_httpd_idle.{c,h}`, owner sweep in `web_httpd_adapter.{c,h}`, lifecycle/TLS composition in `web_server.c`; `tests/web_httpd_idle/run.py`. Independent of diagnostics/optional transports: 15-second observed idle, one-second timer/one queued probe, six rows, actual WS/async/pending exemptions, safe current-owner shutdown and stop/restart fencing. No LRU/socket/timeout/stack increase. SDK queue/owner-delay and regression contract: `docs/https_idle_cleanup.md`.
- Independent throughput diagnostics: `web_serial_transport.{c,h}` owns two fixed per-slot binary-TX aggregates and epoch fences; `web_console.c` exposes default-disabled `web performance enable|disable|show|clear`. Queue-entry/callback-entry, synchronous-send and completion/drain-return estimates, not peer receipt or scheduler-only latency. `tests/web_serial_performance/run.py`; resource/evidence limits and UART0 paired capture: `docs/web_throughput_diagnostics.md`.
- Storage compatibility: `user_database` persists missing storage empty and preserves valid v1 user bytes; private derived `v1_admin_marker`, no public bootstrap/migration/sync APIs. `web_security` privately migrates v1 1392-byte material to TLS-only v2 1340-byte material, exact identity/generation retained, commit before publish, fail closed without fallback overwrite. Credential commands removed; user generated passwords and TLS rotation remain. Contracts, downgrade and evidence limits: `docs/legacy_credential_removal.md`.
- Security files: `src/web_security.{h,c}`, `src/web_cookie_auth.{h,c}`, `src/web_session_store.{h,c}`, `src/web_auth_parse.{h,c}`. Private IDF boundary: `src/web_httpd_adapter.{h,c}`.
- HTTP policy/UI: `web_cookie_auth` + `web_auth_parse` enforce bounded cookie/Origin/CSRF/admin admission; `web_login_ui.{c,h}` serves login, `web_ui.c` owns session-fenced Serial/Admin/Settings and shared quick controllers. Tests: `tests/web_cookie_auth/run.py` (domain variants), `tests/web_auth_parse/run.py`, `tests/web_login_ui/run.py`, `tests/web_ui_session/run.py`.
- Admission diagnostics: `web_diagnostics.{c,h}`, `tests/web_diagnostics/run.py`; six post-TLS records/32-event opt-in ring, no HTTPD off-owner inspection. [Contract](../web_admission_diagnostics.md).
- Identity/lifecycle: `web_server_replace_identity()` + `web_security` reserve service before identity; commit before reserved stop/start, no rollback after commit. `web_lifecycle_settings.{c,h}` owns original-login ID/ACK handoff. Tests: `tests/web_security/run.py`, `tests/web_admin_transport/server_lifecycle.py`.
- Asset files: authored/generated boundary in `src/web_assets_data.{h,c}`, `web_assets/SOURCES.md`, `web_assets/generate_embedded_assets.py`
- Interfaces: web init/start/stop/snapshots; HTTP handlers; ticket mint/consume; attach/detach; targeted session revocation
- Called by: startup, ESP-IDF HTTPS server, user administration revocation, console/local UI
- Dependencies: user database, secure random, broker, successful Wi-Fi manager initialization at boot, mbedTLS/HTTPS server; actual network reachability is an operational prerequisite, not an initializer invariant
- Flow: `browser -> HTTPS login/cookie session -> CSRF-protected ticket -> cookie/Origin/ticket admission -> WebSocket -> web transport -> broker`
- Ownership: HTTPD owns socket send/close work; transport task owns broker mediation; two fixed WebSocket slots and four outstanding tickets.
- Security constraints: Basic/cache removed; four absolute one-hour cookie sessions revalidate principal currentness. Four pre-login challenges (120 s), five credential attempts/60 s globally, no live session/challenge/ticket eviction. Origin/CSRF required for mutations; Origin/cookie/ticket before upgrade. Disconnect pauses reconnect but retains login; Sign out invalidates its session. Authored loader changes must update their hard-coded CSP hashes atomically.
- Session-store boundary: admitted HTTPS start initializes records; auth-init failure gates HTTPS. Failed start/accepted stop disables and wipes state. Tickets/slots require nonzero non-reused session IDs; session/account/global revocation invalidates store records before socket cleanup. RNG/SHA/database calls run outside short portMUX sections; ID/expiry/epoch checks reject stale work. Run `python3 tests/web_session_store/run.py` and its `--serial` integration mode.
- Asset constraint: `web_assets_data.c` is checked-in generated input to the build; do not hand-edit or regenerate casually.

### Browser admin backend

- Files: `src/web_admin_transport.{c,h}`, `src/web_admin_tickets.{c,h}`, protected registration/lifecycle in `web_server.c`, revocation through `web_serial_transport_revoke_*`, diagnostics in `web_console.c`.
- Routes: CSRF-protected admin-only `POST /api/admin/ws-ticket`; ordinary `GET /ws/admin` with cookie/Origin/ticket/shared-console admission before explicit 101. Admin UI entry is explicit; no admin broker client. One socket, two tickets, existing two shared console slots; six total HTTPD sockets, LRU disabled; current overall capacity is 39 URI handlers.
- Currentness/policy: `admin_ssh_console_open_available()` shares two slots with runtime SSH; transport-qualified tokens and owner adapters revalidate outside console locks before commands/prompts. Parsed browser policy remains narrower than typed Settings; [shell contract](../web_administration.md#browser-shell-policy). Tests: `tests/admin_console_boundary/{run,accounts,lifecycle}.py`, `tests/admin_ssh_policy/run.py`, `tests/web_admin_transport/run.py --tickets`, `tests/web_cookie_auth/run.py --admin`.
- Ownership: 20 ms ESP timer queues at most one HTTPD poll, no new task; HTTPD owns 1,552 B PSRAM-only payload and IO. Closure uses HTTPD-owned `shutdown`, not IDF's reusable-pointer queued close. Detach fences submitters; only successful HTTPD stop retires queued state before restart. Session/principal currentness and generation checks protect all sensitive boundaries.

## Typed settings source and regression map

HTTPD reads zero-wait projections and queues only IDs to the existing dispatcher. One original-login slot per domain; canonical owners compare/reserve at execution. [API/lifetime and failure contracts](../web_administration.md#typed-settings-api-and-operation-lifetime).

| Domain | API / canonical owner | Focused source tests |
|---|---|---|
| Serial | `web_serial_settings.{c,h}` / `serial_service` | `tests/web_cookie_auth/run.py --serial-settings`, `tests/admin_console_boundary/run.py` |
| Accounts / keys | `web_account_settings.{c,h}` / `user_database` | `tests/web_cookie_auth/run.py --accounts`, `tests/admin_console_boundary/accounts.py` |
| Network | `web_network_settings.{c,h}` / `wifi_manager`, `mdns_service` | `tests/web_cookie_auth/run.py --network`, `tests/web_network_settings/run.py` |
| Display | `web_display_settings.{c,h}` / `local_status_ui` | `tests/web_cookie_auth/run.py --display` |
| Broker | `web_broker_settings.{c,h}` / `session_broker` | `tests/web_cookie_auth/run.py --broker`, `tests/session_broker_diagnostics/run.py` |
| SSH | `web_ssh_settings.{c,h}` / `ssh_transport`, `ssh_security` | `tests/web_cookie_auth/run.py --ssh`, `tests/ssh_management/run.py`, `tests/ssh_management/security.py` |
| HTTPS / reboot | `web_lifecycle_settings.{c,h}` / `web_server`, `web_security` | `tests/web_cookie_auth/run.py --lifecycle`, `tests/web_admin_transport/server_lifecycle.py` |

Shared UI regression: `tests/web_ui_session/run.py` and its domain `.cjs` fixtures / `layout.py`. These are navigation pointers, not claims of test execution.

## SSH

**Responsibility:** authenticate SSH, route users to serial and administrators to the command dispatcher, and own wolfSSH lifecycle.

- Files: `src/ssh_transport.{h,c}`, `src/ssh_security.{h,c}`, `src/ssh_console.{h,c}`
- Interfaces: init/start/stop, session snapshots/disconnect/revocation, host-key replacement, counters; `ssh_transport_get_management_snapshot()` / `ssh_transport_manage_current()` fence lifecycle and exact session admission. `ssh_transport_replace_identity()` reserves service before task-bound security identity across stop/commit/restart, retains context until all slots retire and rejects orphan starts. Tests: `tests/ssh_management/run.py`, `tests/ssh_management/security.py`, `tests/ssh_management/runtime.py`.
- Called by: startup, network clients, user revocation, console/local UI
- Dependencies: user database, broker, admin SSH console, secure random, wolfSSH/wolfSSL; boot start gate requires Wi-Fi and SSH security/runtime readiness, independently of HTTPS identity readiness (verified in `main.c` after accepted legacy cleanup).
- Flow: role `user` -> broker; role `admin` -> `admin_ssh_console`
- Ownership: after caller-side library initialization, one task pinned to core 1 owns runtime wolfSSH contexts/sessions; two fixed generation-tagged slots.
- Security constraint: an interactive shell request is required; exec and subsystems are rejected, and no project file-transfer or forwarding route exists. PTY is not explicitly required.

## Users, authentication, and authorization

**Responsibility:** persist bounded accounts, verify passwords/SSH keys, issue secret-free principals, and enforce account invariants.

- Files: `src/user_database.{h,c}`, `src/user_console.{h,c}`; `src/admin_command_gate.{h,c}` is currently a narrow recursive wrapper used only by the `user` command handler, not the global command serializer
- Interfaces: credential-independent init/empty recovery, authenticate, principal-currentness, account/password/role/key mutations, snapshots
- Called by: web and SSH authentication/currentness checks and console administration
- Dependencies: NVS, secure random, mbedTLS cryptography; after a committed command-layer mutation, best-effort web/SSH revocation calls supplement authoritative transport currentness checks
- Ownership: database mutex protects the internal live record and PSRAM-preferred transactional candidate; password authentication runs PBKDF2 outside the mutex and revalidates afterward, while mutation locking must be checked per operation.
- Authorization: UART0 establishes the first administrator through normal `user add` and exclusively owns unavailable-database recovery to empty (healthy database refused); current admins may use admin SSH for other commands unless handler policy denies them. HTTPS serial/status permits both roles; administration requires `admin`.
- Constraint: final administrator cannot be deleted or demoted; transport principals must be rechecked after mutations.

## Administration console infrastructure

**Responsibility:** provide one canonical command registry and serialized execution for UART0, admin SSH and browser admin.

- Files: `src/admin_ssh_console.{h,c}`, `src/console_input.{h,c}`, `src/console_completion.{h,c}`, `src/system_console.{h,c}`, `src/network_console.{h,c}` and all `*_console.{h,c}` modules
- Entry points: `admin_ssh_console_init()`, `admin_ssh_console_start_uart_frontend()`, command registration functions
- Called by: startup, UART0 frontend, role-`admin` SSH transport, browser admin transport
- Dependencies: ESP-IDF console/linenoise, all command handlers, user-principal currentness
- Flow: `UART0/admin SSH/browser admin -> bounded request queue -> one dispatcher -> esp_console_run()`
- Ownership: dispatcher is sole `esp_console_run()` caller; the SSH owner exclusively performs post-initialization wolfSSH runtime calls.
- Lifecycle: remote session tokens include slot generation; fixed output/history/prompt state is wiped immediately on idle close or after an executing handler returns. Admin SSH `exit` and Ctrl+D on an empty command line request bounded deferred self-disconnect after best-effort output draining.
- Constraint: one slow command or prompt serializes all administration. Admin SSH is unavailable until command registration and UART frontend creation complete; supported deferred actions wait only for a bounded application-buffer drain heuristic.

## Wi-Fi

**Responsibility:** persist station/AP policy and own asynchronous ESP-NETIF/Wi-Fi state transitions.

- Files: `src/wifi_config.{h,c}`, `src/wifi_manager.{h,c}`, `src/wifi_console.{h,c}`, `src/mdns_config.{h,c}`, `src/mdns_service.{h,c}`, `src/mdns_console.{h,c}`, `src/network_console.{h,c}`
- Interfaces: config defaults/validate/load/save; manager init/start/stop/apply/reconnect/next-profile/snapshot
- Dependencies: secure random for default AP password, NVS, ESP-NETIF/Wi-Fi/events, Espressif mDNS, lwIP diagnostics
- Lifecycle: permanent manager task and bounded queue; callbacks enqueue compact events only.
- Constraint: application NVS is authoritative (`WIFI_STORAGE_RAM`); working edits are not persisted until save. Start/stop, including local controls, intentionally update the RAM `enabled_at_boot` field. Working-config copies contain PSKs and must be tightly scoped and wiped; routine status/local UI must use secret-free snapshots.

## Local display and controls

**Responsibility:** own OLED I2C/framebuffer operations and present status plus constrained button actions.

- Files: `src/local_display.{h,c}`, `src/local_status_ui.{h,c}`, `src/local_boot_animation.{h,c}`, `src/local_ui_config.{h,c}`, `src/local_ui_console.{h,c}`
- Interfaces: display init/frame/draw/commit/snapshot; UI start/activity/config; generation-checked settings projection/update and explicit persistence reservation; versioned NVS settings
- Called by: startup, local UI task, diagnostics, display console
- Dependencies: copied snapshots/public APIs from serial, broker, USB, Wi-Fi, web, SSH
- Ownership: `local_display` solely owns I2C0 and framebuffer mutex; a frame belongs to its initiating task.
- Lifecycle: the low-priority task is firmware-lifetime only if button GPIO initialization succeeds; it still runs with an absent panel so a press can reprobe after successful I2C bus setup. Failed bus creation is not recoverable by that reprobe, and `display` configuration commands depend on the UI task.
- Constraint: collect service snapshots before I2C; local UI never joins broker or handles secrets. All configuration writers honor the UI owner's zero-wait reservation; NVS runs outside timing critical sections. Reset commits defaults before RAM publication, including CLI; buttons/diagnostic holds update activity, not configuration generation.

## Hardware and diagnostics

**Responsibility:** centralize board wiring and provide bounded electrical tests with safe cleanup.

- Files: `src/board_pins.h`, `src/rs232_hw_test.{h,c}`, `src/local_ui_hw_test.{h,c}`, `src/status_led.{h,c}`
- Documentation: `docs/wiring.md`, `docs/electrical_tests.md`
- Called by: startup and `debug` commands
- Dependencies: physical RS-232 owner, serial/display services, ESP-IDF GPIO/UART/I2C/LED drivers
- Ownership: RS-232 diagnostics refuse to run while the service owns the port; display diagnostics reuse `local_display`.
- Constraint: wiring and voltage assumptions are safety-relevant; verify target hardware before running diagnostics. RGB LED initialization is currently boot-fatal, and its colors report diagnostic state rather than aggregate firmware health.

## Where should I look?

| Task | Start here |
|---|---|
| Change boot order or failure behavior | `src/main.c`, then affected subsystem `init/start` contracts |
| Change serial framing, flow control, or persistence | `serial_config.*`, `serial_service.*`, `serial_console.*` |
| Change writer/observer policy | `session_broker.*`, then all three transports |
| Debug missing or duplicated serial bytes | `serial_service.c` -> `session_broker.c` -> relevant transport task |
| Change USB open/DTR or line coding | `usb_cdc_transport.*` |
| Change browser terminal protocol | `web_serial_transport.*`, `web_ui.c`, `web_server.c` |
| Change HTTPS endpoints/authentication | `web_server.*`, `web_security.*`, `user_database.*` |
| Change SSH login or role routing | `ssh_transport.*`, `ssh_security.*`, `user_database.*` |
| Add or change a command | relevant `*_console.c`, `console_completion.c`, `admin_ssh_console.c` policy/deferred handling |
| Change account roles/passwords/keys | `user_database.*`, `user_console.c`, transport revocation APIs |
| Change Wi-Fi policy or profile persistence | `wifi_manager.*`, `wifi_config.*`, `wifi_console.c` |
| Change station mDNS hostname or persistence | `mdns_service.*`, `mdns_config.*`, `mdns_console.c`, then `wifi_manager.c` |
| Change OLED rendering or buttons | `local_status_ui.c`, `local_display.*`, `local_ui_config.*` |
| Change board GPIO or electrical tests | `board_pins.h`, hardware test module, `docs/wiring.md` |
| Change embedded browser assets | `web_assets/SOURCES.md`, generator, then generated data only as an explicit regeneration task |
| Investigate memory/watchdog regressions | broker/web/SSH bounded loops, allocation placement, root `CMakeLists.txt`, relevant roadmap Phase 6 history |