ESP32_Serial_Swiss_Army_Knife/docs/agent/code-map.md

# Code map

This is a semantic map, not a complete file inventory. Start here, then read the listed headers and only the implementation paths relevant to the task.

## Bootstrap and system composition

**Responsibility:** establish startup order, recovery behavior, configuration loading, service dependencies, and command registration.

- Files: `src/main.c`, `src/CMakeLists.txt`, root `CMakeLists.txt`, `platformio.ini`, `partitions.csv`, `src/idf_component.yml`; inspect targeted settings in `sdkconfig.defaults` when crypto, PSRAM, HTTPS/HTTPD, USB, or socket capacity matters
- Entry point: `app_main()`
- Called by: ESP-IDF runtime
- Dependencies: every subsystem initializer
- Lifecycle constraint: optional display/network failures should not remove UART0 administrative recovery or USB UART1 access; the custom administration frontend starts only after command registration.

## Secure randomness

**Responsibility:** provide the sole project-owned, mutex-serialized application DRBG, seeded before Wi-Fi/radio use.

- Files: `src/secure_random.{h,c}`
- Interfaces: `secure_random_init()`, random-byte helpers, `secure_wipe()`
- Called by: HTTPS material, SSH keys, users, Wi-Fi defaults, tickets, the HTTPS authentication cache, and the wolfCrypt seed callback
- Constraint: initialization order is security-significant; the DRBG deliberately avoids post-radio reseeding and fails closed at its generation limit. Do not add independent weak RNGs or radio-dependent early entropy paths.

## Physical RS-232 and serial service

**Responsibility:** protect the MAX3243/UART resource, own UART1 while running, buffer binary RX/TX, apply serial configuration, and expose status/counters.

- Files: `src/rs232_port_owner.{h,c}`, `src/serial_service.{h,c}`, `src/serial_config.{h,c}`, `src/serial_console.{h,c}`
- Interfaces: owner claim/release/fault; serial init/start/stop/read/write/configuration/snapshots; versioned NVS load/save
- Normal data caller: `session_broker`; USB, WebSocket, role-`user` SSH, console, and local UI also call serial lifecycle/configuration APIs as appropriate
- Dependencies: ESP-IDF UART driver, `board_pins.h`, NVS
- Ownership: the cooperative owner arbitrates active diagnostics (`PHASE0`) against the service (`SERVICE`); boot-time static-safe GPIO initialization and service-owned static-mode restoration are explicit exceptions. Unsafe cleanup marks `FAULT` until reboot.
- Lifecycle: stop/reconfiguration discards serial-service RX/TX and task-local pending bytes, but leaves broker clients, writer ownership, events, and already-fanned output intact. The 16 KiB RX and 8 KiB TX payloads prefer PSRAM; their FreeRTOS controls and UART driver storage remain internal.

## Session broker

**Responsibility:** mediate all transport access to the serial service; provide one writer lease and multiple isolated observers.

- Files: `src/session_broker.{h,c}`, `src/session_console.{h,c}`
- Interfaces: connect/disconnect, request/release/force writer, nonblocking read/write/event APIs, snapshots and counters
- Called by: USB, web serial, role-`user` SSH, console tests, local UI snapshots/actions
- Dependencies: `serial_service`
- Data path: `transport -> broker -> serial service -> UART1`; reverse data is fanned out per client.
- Ownership: client IDs are slot/generation-safe; events are advisory and can drop, so use snapshots as authority.
- Lifecycle: one permanent task and eight preallocated client slots; slow output drops only for the affected client.

## Native USB CDC

**Responsibility:** adapt TinyUSB CDC host state/data to one broker client.

- Files: `src/usb_cdc_transport.{h,c}`, `src/usb_console.{h,c}`
- Interfaces: `usb_cdc_transport_init()`, snapshots/counters, queued writer request/release
- Called by: startup, TinyUSB callbacks, console/local UI
- Dependencies: TinyUSB, broker, serial service
- Flow: `USB host <-> USB task <-> broker`
- Lifecycle: permanent owner task; broker client exists only while attached with host DTR asserted.
- Constraint: callbacks retain the latest host line coding only for diagnostics; it never reconfigures UART1. UART1 framing and speed remain controlled by the serial configuration and its explicit persistence commands.

## Web and WebSocket serial

**Responsibility:** serve authenticated HTTPS UI/API, issue WebSocket tickets, and adapt browser serial sessions to broker clients.

- Files: `src/web_server.{h,c}`, `src/web_serial_transport.{h,c}`, `src/web_ui.{h,c}`, `src/web_console.{h,c}`
- Security files: `src/web_security.{h,c}`, `src/web_cookie_auth.{h,c}`, `src/web_session_store.{h,c}`, `src/web_auth_parse.{h,c}`. Private IDF boundary: `src/web_httpd_adapter.{h,c}`.
- Asset files: authored/generated boundary in `src/web_assets_data.{h,c}`, `web_assets/SOURCES.md`, `web_assets/generate_embedded_assets.py`
- Interfaces: web init/start/stop/snapshots; HTTP handlers; ticket mint/consume; attach/detach; targeted session revocation
- Called by: startup, ESP-IDF HTTPS server, user administration revocation, console/local UI
- Dependencies: user database, secure random, broker, successful Wi-Fi manager initialization at boot, mbedTLS/HTTPS server; actual network reachability is an operational prerequisite, not an initializer invariant
- Flow: `browser -> HTTPS login/cookie session -> CSRF-protected ticket -> cookie/Origin/ticket admission -> WebSocket -> web transport -> broker`
- Ownership: HTTPD owns socket send/close work; transport task owns broker mediation; two fixed WebSocket slots and four outstanding tickets.
- Security constraints: Basic/cache removed; four absolute one-hour cookie sessions revalidate principal currentness. Four pre-login challenges (120 s), five credential attempts/60 s globally, no live session/challenge/ticket eviction. Origin/CSRF required for mutations; Origin/cookie/ticket before upgrade. Disconnect pauses reconnect but retains login; Sign out invalidates its session. Authored loader changes must update their hard-coded CSP hashes atomically.
- Session-store boundary: admitted HTTPS start initializes records; auth-init failure gates HTTPS. Failed start/accepted stop disables and wipes state. Tickets/slots require nonzero non-reused session IDs; session/account/global revocation invalidates store records before socket cleanup. RNG/SHA/database calls run outside short portMUX sections; ID/expiry/epoch checks reject stale work. Run `python3 tests/web_session_store/run.py` and its `--serial` integration mode.
- 8D.3 HTTP policy: `web_cookie_auth` owns public login/challenge/login POST/session/logout routes and protected-route checks; `web_auth_parse` handles bounded values/JSON. `web_httpd_adapter` alone reads private IDF 5.5.0 header scratch, rejects duplicate fields, defers 101 until transport admission and wipes consumed scratch while preserving right-aligned pending bytes. No SDK patch. `src/CMakeLists.txt` supplies private includes and compiles HTTPD warning/debug logs out. Test with `python3 tests/web_cookie_auth/run.py` and `python3 tests/web_auth_parse/run.py`.
- 8D.3 UI: `src/web_login_ui.{c,h}` serves standalone `/login`; `web_ui.c` validates session before serial connect/restore and handles logout/401 safely. Both scripts hash-bound, auth documents/app no-store. Tests: `python3 tests/web_login_ui/run.py` and `python3 tests/web_ui_session/run.py`. Live cutover host-tested/build-verified, M1 validated by user sign-off (numeric reserves open): `docs/phase8d3_implementation.md`.
- Asset constraint: `web_assets_data.c` is checked-in generated input to the build; do not hand-edit or regenerate casually.
- 8D.6 UI: `web_ui.c` adds admin-only Serial/Admin selection and explicit admin open/close through existing endpoints. Serial socket/client/lease survives mode switches; hidden output drains into independent 5,000-line/64 KiB-pending terminals with visible browser-drop counts. Selected keyboard only; logout/expiry/pagehide closes both with handler cleanup. Session identity changes require a clean document before adopting the view; same-session restore retains hidden-until-validated buffers. Fit readiness retries are bounded to three and cache only success. Focused `tests/web_ui_session/run.py` now has 16 groups; target checks remain pending in `docs/phase8d6_implementation.md`. 8D.5 is user-validated; no 8D.7 restriction change.

### Browser admin backend (8D.5)

- Files: `src/web_admin_transport.{c,h}`, `src/web_admin_tickets.{c,h}`, protected registration/lifecycle in `web_server.c`, revocation through `web_serial_transport_revoke_*`, diagnostics in `web_console.c`.
- Routes: CSRF-protected admin-only `POST /api/admin/ws-ticket`; ordinary `GET /ws/admin` with cookie/Origin/ticket/shared-console admission before explicit 101. No UI entry or broker client. One socket, two tickets, existing two shared console slots; six total HTTPD sockets, LRU disabled, 16 URI handlers.
- Ownership: 20 ms ESP timer queues at most one HTTPD poll, no new task; HTTPD owns 1,552 B PSRAM-only payload and IO. Closure uses HTTPD-owned `shutdown`, not IDF's reusable-pointer queued close. Detach fences submitters; only successful HTTPD stop retires queued state before restart. Session/principal currentness and generation checks protect all sensitive boundaries.
- Tests: `python3 tests/web_admin_transport/run.py --tickets`, `python3 tests/web_admin_transport/server_lifecycle.py`, `python3 tests/web_cookie_auth/run.py --admin`; manual smoke client/procedure in `tests/web_admin_transport/README.md` and `docs/phase8d5_implementation.md`. Final shutdown fix is host-tested and build-verified by the parent's sequential final `pio run`; target validation remains pending.

## SSH

**Responsibility:** authenticate SSH, route users to serial and administrators to the command dispatcher, and own wolfSSH lifecycle.

- Files: `src/ssh_transport.{h,c}`, `src/ssh_security.{h,c}`, `src/ssh_console.{h,c}`
- Interfaces: init/start/stop, session snapshots/disconnect/revocation, host-key replacement, counters
- Called by: startup, network clients, user revocation, console/local UI
- Dependencies: user database, broker, admin SSH console, secure random, wolfSSH/wolfSSL; current boot start gate also depends on `web_security` readiness
- Flow: role `user` -> broker; role `admin` -> `admin_ssh_console`
- Ownership: after caller-side library initialization, one task pinned to core 1 owns runtime wolfSSH contexts/sessions; two fixed generation-tagged slots.
- Security constraint: an interactive shell request is required; exec and subsystems are rejected, and no project file-transfer or forwarding route exists. PTY is not explicitly required.

## Users, authentication, and authorization

**Responsibility:** persist bounded accounts, verify passwords/SSH keys, issue secret-free principals, and enforce account invariants.

- Files: `src/user_database.{h,c}`, `src/user_console.{h,c}`; `src/admin_command_gate.{h,c}` is currently a narrow recursive wrapper used only by the `user` command handler, not the global command serializer
- Interfaces: init/migration/recovery, authenticate, principal-currentness, account/password/role/key mutations, snapshots
- Called by: web and SSH authentication/currentness checks and console administration
- Dependencies: NVS, secure random, mbedTLS cryptography; after a committed command-layer mutation, best-effort web/SSH revocation calls supplement authoritative transport currentness checks
- Ownership: database mutex protects the internal live record and PSRAM-preferred transactional candidate; password authentication runs PBKDF2 outside the mutex and revalidates afterward, while mutation locking must be checked per operation.
- Authorization: UART0 exclusively owns initial administrator bootstrap and unavailable-database recovery; current admins may use admin SSH for other commands unless handler policy denies them. HTTPS currently treats both roles alike.
- Constraint: final administrator cannot be deleted or demoted; transport principals must be rechecked after mutations.

## Administration console infrastructure

**Responsibility:** provide one canonical command registry and serialized execution for UART0 and admin SSH.

- Files: `src/admin_ssh_console.{h,c}`, `src/console_input.{h,c}`, `src/console_completion.{h,c}`, `src/system_console.{h,c}`, `src/network_console.{h,c}` and all `*_console.{h,c}` modules
- Entry points: `admin_ssh_console_init()`, `admin_ssh_console_start_uart_frontend()`, command registration functions
- Called by: startup, UART0 frontend, role-`admin` SSH transport
- Dependencies: ESP-IDF console/linenoise, all command handlers, user-principal currentness
- Flow: `UART0/admin SSH -> bounded request queue -> one dispatcher -> esp_console_run()`
- Ownership: dispatcher is sole `esp_console_run()` caller; the SSH owner exclusively performs post-initialization wolfSSH runtime calls.
- Lifecycle: remote session tokens include slot generation; fixed output/history/prompt state is wiped immediately on idle close or after an executing handler returns. Admin SSH `exit` and Ctrl+D on an empty command line request bounded deferred self-disconnect after best-effort output draining.
- Constraint: one slow command or prompt serializes all administration. Admin SSH is unavailable until command registration and UART frontend creation complete; supported deferred actions wait only for a bounded application-buffer drain heuristic.
- 8D.4/8D.5 boundary: `admin_ssh_console_open_owned()` retains explicit-index admission; runtime SSH and browser owners use `admin_ssh_console_open_available()` for the same two slots. Copied transport-qualified identity and immutable firmware-lifetime currentness/drain/lifecycle adapters; SSH publishes its allocated console index separately from its physical SSH slot. Owners handle liveness/output; dispatcher and prompt waits additionally require owner currentness (250 ms polling plus check/scheduling latency). SSH publishes locked principal copies; consumed console output is wiped. Completion scratch is nonblockingly serialized. Browser unsupported lifecycle/account mutations are rejected before execution. Focused host command: `python3 tests/admin_console_boundary/run.py`.

## Wi-Fi

**Responsibility:** persist station/AP policy and own asynchronous ESP-NETIF/Wi-Fi state transitions.

- Files: `src/wifi_config.{h,c}`, `src/wifi_manager.{h,c}`, `src/wifi_console.{h,c}`, `src/mdns_config.{h,c}`, `src/mdns_service.{h,c}`, `src/mdns_console.{h,c}`, `src/network_console.{h,c}`
- Interfaces: config defaults/validate/load/save; manager init/start/stop/apply/reconnect/next-profile/snapshot
- Called by: startup, console, local UI, ESP event callbacks
- Dependencies: secure random for default AP password, NVS, ESP-NETIF/Wi-Fi/events, Espressif mDNS, lwIP diagnostics
- Lifecycle: permanent manager task and bounded queue; callbacks enqueue compact events only.
- Constraint: application NVS is authoritative (`WIFI_STORAGE_RAM`); working edits are not persisted until save. Start/stop, including local controls, intentionally update the RAM `enabled_at_boot` field. Working-config copies contain PSKs and must be tightly scoped and wiped; routine status/local UI must use secret-free snapshots.

## Local display and controls

**Responsibility:** own OLED I2C/framebuffer operations and present status plus constrained button actions.

- Files: `src/local_display.{h,c}`, `src/local_status_ui.{h,c}`, `src/local_boot_animation.{h,c}`, `src/local_ui_config.{h,c}`, `src/local_ui_console.{h,c}`
- Interfaces: display init/frame/draw/commit/snapshot; UI start/activity/config; versioned NVS settings
- Called by: startup, local UI task, diagnostics, display console
- Dependencies: copied snapshots/public APIs from serial, broker, USB, Wi-Fi, web, SSH
- Ownership: `local_display` solely owns I2C0 and framebuffer mutex; a frame belongs to its initiating task.
- Lifecycle: the low-priority task is firmware-lifetime only if button GPIO initialization succeeds; it still runs with an absent panel so a press can reprobe after successful I2C bus setup. Failed bus creation is not recoverable by that reprobe, and `display` configuration commands depend on the UI task.
- Constraint: collect service snapshots before I2C; local UI never joins broker or handles secrets.

## Hardware and diagnostics

**Responsibility:** centralize board wiring and provide bounded electrical tests with safe cleanup.

- Files: `src/board_pins.h`, `src/rs232_hw_test.{h,c}`, `src/local_ui_hw_test.{h,c}`, `src/status_led.{h,c}`
- Documentation: `docs/wiring.md`, `docs/electrical_tests.md`
- Called by: startup and `debug` commands
- Dependencies: physical RS-232 owner, serial/display services, ESP-IDF GPIO/UART/I2C/LED drivers
- Ownership: RS-232 diagnostics refuse to run while the service owns the port; display diagnostics reuse `local_display`.
- Constraint: wiring and voltage assumptions are safety-relevant; verify target hardware before running diagnostics. RGB LED initialization is currently boot-fatal, and its colors report diagnostic state rather than aggregate firmware health.

## Where should I look?

| Task | Start here |
|---|---|
| Change boot order or failure behavior | `src/main.c`, then affected subsystem `init/start` contracts |
| Change serial framing, flow control, or persistence | `serial_config.*`, `serial_service.*`, `serial_console.*` |
| Change writer/observer policy | `session_broker.*`, then all three transports |
| Debug missing or duplicated serial bytes | `serial_service.c` -> `session_broker.c` -> relevant transport task |
| Change USB open/DTR or line coding | `usb_cdc_transport.*` |
| Change browser terminal protocol | `web_serial_transport.*`, `web_ui.c`, `web_server.c` |
| Change HTTPS endpoints/authentication | `web_server.*`, `web_security.*`, `user_database.*` |
| Change SSH login or role routing | `ssh_transport.*`, `ssh_security.*`, `user_database.*` |
| Add or change a command | relevant `*_console.c`, `console_completion.c`, `admin_ssh_console.c` policy/deferred handling |
| Change account roles/passwords/keys | `user_database.*`, `user_console.c`, transport revocation APIs |
| Change Wi-Fi policy or profile persistence | `wifi_manager.*`, `wifi_config.*`, `wifi_console.c` |
| Change station mDNS hostname or persistence | `mdns_service.*`, `mdns_config.*`, `mdns_console.c`, then `wifi_manager.c` |
| Change OLED rendering or buttons | `local_status_ui.c`, `local_display.*`, `local_ui_config.*` |
| Change board GPIO or electrical tests | `board_pins.h`, hardware test module, `docs/wiring.md` |
| Change embedded browser assets | `web_assets/SOURCES.md`, generator, then generated data only as an explicit regeneration task |
| Investigate memory/watchdog regressions | broker/web/SSH bounded loops, allocation placement, root `CMakeLists.txt`, relevant roadmap Phase 6 history |