ESP32_Serial_Swiss_Army_Knife/docs/agent/code-map.md

Code map

This is a semantic map, not a complete file inventory. Start here, then read the listed headers and only the implementation paths relevant to the task.

Bootstrap and system composition

Responsibility: establish startup order, recovery behavior, configuration loading, service dependencies, and command registration.

• Files: src/main.c, src/CMakeLists.txt, root CMakeLists.txt, platformio.ini, partitions.csv, src/idf_component.yml; inspect targeted settings in sdkconfig.defaults when crypto, PSRAM, HTTPS/HTTPD, USB, or socket capacity matters
• Entry point: app_main()
• Called by: ESP-IDF runtime
• Dependencies: every subsystem initializer
• Lifecycle constraint: optional display/network failures should not remove UART0 administrative recovery or USB UART1 access; the custom administration frontend starts only after command registration.

Secure randomness

Responsibility: provide the sole project-owned, mutex-serialized application DRBG, seeded before Wi-Fi/radio use.

• Files: src/secure_random.{h,c}
• Interfaces: secure_random_init(), random-byte helpers, secure_wipe()
• Called by: HTTPS material, SSH keys, users, Wi-Fi defaults, tickets, the HTTPS authentication cache, and the wolfCrypt seed callback
• Constraint: initialization order is security-significant; the DRBG deliberately avoids post-radio reseeding and fails closed at its generation limit. Do not add independent weak RNGs or radio-dependent early entropy paths.

Physical RS-232 and serial service

Responsibility: protect the MAX3243/UART resource, own UART1 while running, buffer binary RX/TX, apply serial configuration, and expose status/counters.

• Files: src/rs232_port_owner.{h,c}, src/serial_service.{h,c}, src/serial_config.{h,c}, src/serial_console.{h,c}
• Interfaces: owner claim/release/fault; serial init/start/stop/read/write/configuration/snapshots; versioned NVS load/save
• Normal data caller: session_broker; USB, WebSocket, role-user SSH, console, and local UI also call serial lifecycle/configuration APIs as appropriate
• Dependencies: ESP-IDF UART driver, board_pins.h, NVS
• Ownership: the cooperative owner arbitrates active diagnostics (PHASE0) against the service (SERVICE); boot-time static-safe GPIO initialization and service-owned static-mode restoration are explicit exceptions. Unsafe cleanup marks FAULT until reboot.
• Lifecycle: stop/reconfiguration discards serial-service RX/TX and task-local pending bytes, but leaves broker clients, writer ownership, events, and already-fanned output intact. The 16 KiB RX and 8 KiB TX payloads prefer PSRAM; their FreeRTOS controls and UART driver storage remain internal.

Session broker

Responsibility: mediate all transport access to the serial service; provide one writer lease and multiple isolated observers.

• Files: src/session_broker.{h,c}, src/session_console.{h,c}
• Interfaces: connect/disconnect, request/release/force writer, nonblocking read/write/event APIs, snapshots and counters
• Called by: USB, web serial, role-user SSH, console tests, local UI snapshots/actions
• Dependencies: serial_service
• Data path: transport -> broker -> serial service -> UART1; reverse data is fanned out per client.
• Ownership: client IDs are slot/generation-safe; events are advisory and can drop, so use snapshots as authority.
• Lifecycle: one permanent task and eight preallocated client slots; slow output drops only for the affected client.

Native USB CDC

Responsibility: adapt TinyUSB CDC host state/data to one broker client.

• Files: src/usb_cdc_transport.{h,c}, src/usb_console.{h,c}
• Interfaces: usb_cdc_transport_init(), snapshots/counters, queued writer request/release
• Called by: startup, TinyUSB callbacks, console/local UI
• Dependencies: TinyUSB, broker, serial service
• Flow: USB host <-> USB task <-> broker
• Lifecycle: permanent owner task; broker client exists only while attached with host DTR asserted.
• Constraint: callbacks retain the latest host line coding only for diagnostics; it never reconfigures UART1. UART1 framing and speed remain controlled by the serial configuration and its explicit persistence commands.

Web and WebSocket serial

Responsibility: serve authenticated HTTPS UI/API, issue WebSocket tickets, and adapt browser serial sessions to broker clients.

• Files: src/web_server.{h,c}, src/web_serial_transport.{h,c}, src/web_ui.{h,c}, src/web_console.{h,c}

• Security files: src/web_security.{h,c}, src/web_cookie_auth.{h,c}, src/web_session_store.{h,c}, src/web_auth_parse.{h,c}. Private IDF boundary: src/web_httpd_adapter.{h,c}.

• Asset files: authored/generated boundary in src/web_assets_data.{h,c}, web_assets/SOURCES.md, web_assets/generate_embedded_assets.py

• Interfaces: web init/start/stop/snapshots; HTTP handlers; ticket mint/consume; attach/detach; targeted session revocation

• Called by: startup, ESP-IDF HTTPS server, user administration revocation, console/local UI

• Dependencies: user database, secure random, broker, successful Wi-Fi manager initialization at boot, mbedTLS/HTTPS server; actual network reachability is an operational prerequisite, not an initializer invariant

• Flow: browser -> HTTPS login/cookie session -> CSRF-protected ticket -> cookie/Origin/ticket admission -> WebSocket -> web transport -> broker

• Ownership: HTTPD owns socket send/close work; transport task owns broker mediation; two fixed WebSocket slots and four outstanding tickets.

• Security constraints: Basic/cache removed; four absolute one-hour cookie sessions revalidate principal currentness. Four pre-login challenges (120 s), five credential attempts/60 s globally, no live session/challenge/ticket eviction. Origin/CSRF required for mutations; Origin/cookie/ticket before upgrade. Disconnect pauses reconnect but retains login; Sign out invalidates its session. Authored loader changes must update their hard-coded CSP hashes atomically.

• Session-store boundary: admitted HTTPS start initializes records; auth-init failure gates HTTPS. Failed start/accepted stop disables and wipes state. Tickets/slots require nonzero non-reused session IDs; session/account/global revocation invalidates store records before socket cleanup. RNG/SHA/database calls run outside short portMUX sections; ID/expiry/epoch checks reject stale work. Run python3 tests/web_session_store/run.py and its --serial integration mode.

• 8D.3 HTTP policy: web_cookie_auth owns public login/challenge/login POST/session/logout routes and protected-route checks; web_auth_parse handles bounded values/JSON. web_httpd_adapter alone reads private IDF 5.5.0 header scratch, rejects duplicate fields, defers 101 until transport admission and wipes consumed scratch while preserving right-aligned pending bytes. No SDK patch. src/CMakeLists.txt supplies private includes and compiles HTTPD warning/debug logs out. Test with python3 tests/web_cookie_auth/run.py and python3 tests/web_auth_parse/run.py.

• 8D.3 UI: src/web_login_ui.{c,h} serves standalone /login; web_ui.c validates session before serial connect/restore and handles logout/401 safely. Both scripts hash-bound, auth documents/app no-store. Tests: python3 tests/web_login_ui/run.py and python3 tests/web_ui_session/run.py. Live cutover host-tested/build-verified, M1 validated by user sign-off (numeric reserves open): docs/phase8d3_implementation.md.

• Asset constraint: web_assets_data.c is checked-in generated input to the build; do not hand-edit or regenerate casually.

• 8D.6 UI: web_ui.c adds admin-only Serial/Admin selection and explicit admin open/close through existing endpoints. Serial socket/client/lease survives mode switches; hidden output drains into independent 5,000-line/64 KiB-pending terminals with visible browser-drop counts. Selected keyboard only; logout/expiry/pagehide closes both with handler cleanup. Session identity changes require a clean document before adopting the view; same-session restore retains hidden-until-validated buffers. Fit readiness retries are bounded to three and cache only success. Focused tests/web_ui_session/run.py has 17 groups plus toolbar-order/CSP checks. 8D.6 is user-validated; telemetry, evidence limits and 8D.7 handoff are in docs/phase8d6_implementation.md. Numeric reserves remain open; no 8D.7 restriction change.

• 8D.8: web_ui.c adds admin-only Settings/Serial without socket/lease changes. web_server.c exposes optional admin-only bodyless GET /api/settings/serial, eight working serial values, 256-byte response, no writes/NVS. serial_service_get_snapshot() is a zero-wait consistent config/running copy. web_httpd_register_optional_get() stages both new-route allocations before table publication (installed IDF public registration leaves a dangling descriptor on name-allocation failure); only Settings uses this startup/exact-GET adapter. 17 URI slots, six sockets/no LRU, no new task. Tests: cookie auth --settings (5 groups), UI (21 groups), lifecycle (12 groups). Implemented/build-verified, target/signoff pending; exact accounting and inherited registration-audit followup: docs/phase8d8_implementation.md. M2 remains signed off; no 8D.9.

• 8D.9: web_serial_settings.{c,h} owns strict 256-byte typed mutation admission and one session-bound pending/result slot. Existing admin_ssh_console dispatcher consumes only an ID, revalidates currentness/dequeue deadline and calls canonical serial APIs. web_server.c adds optional GET/POST /api/settings/serial-operation (19 handlers total); web_cookie_auth_require_json() retains Origin/CSRF/admin policy, private optional registration supports exact GET/POST. UI adds explicit framing/lifecycle/persistence with automatic completion checks (1 s, at most 10 GETs/15 s overall), refresh on known terminal results and manual uncertainty recovery without socket/lease changes. Settings stay visible/stale while pending; only Reset confirms NVS overwrite; selecting the current view is a no-op. /api/status uses a consistent zero-wait serial snapshot (running:null when unavailable). Tests: cookie --serial-settings (10 groups), --settings (6), UI (35 after UX refinement), console boundary and lifecycle (13). Build verified, target/signoff pending; bounds and failure contracts: docs/phase8d9_implementation.md. Supersedes 8D.8's no-8D.9 status above.

Browser admin backend (8D.5)

• 8D.7 current status (2026-09-07): implemented scope validated; M2 explicitly signed off by the user ("Jupp, sign M2 off"). Supersedes M2-open/target-pending/continuation statements in the historical slices below; accepted M2 does not require revalidation. User verified certificate rotation and web start/stop via UART0/SSH admin/web admin, restarting after browser stop via another route; full mix without broker drops up to 230400 baud after external adapter correction is user-reported. Intermittent supported two serial + one admin admission failures, recently not recurring, are accepted nonblocking, not fixed. Browser self/generated/key/legacy-credential and other owner command restrictions remain deferred; bootstrap/recovery remain permanently UART0-only. Numeric memory reserves/stack margins remain unapproved; no full parity or individual unreported checklist passes. Next: separately requested 8D.8 read-only settings entry and Serial page; sign-off alone authorizes no implementation. Evidence: docs/phase8d7_implementation.md.

• 8D.7 third account slice: admin_ssh_console shares parsed browser other-account policy with user_console; interactive add/password and forced delete/role now allowed, self/generated/key/bootstrap/recovery still blocked. Post-prompt/pre-DB-API currentness is operation admission, not cancellation of admitted derivation/commit. Existing target-only notifications follow success. Review has no actionable findings; python3 tests/admin_console_boundary/accounts.py adds deterministic handler/database failure and stale-next-operation regressions. Target/M2 pending; see docs/phase8d7_implementation.md.

• 8D.7 second slice: exact parsed browser web certificate rotate --force; admin_ssh_console.{c,h} supplies the typed request union/owner dispatcher_actions mask, bounded drain/200 ms handoff to the existing 12 KiB dispatcher, persistent pending gate and revalidated executing-slot reservation. web_console.c schedules; web_admin_transport.c revalidates then calls transactional web_security_rotate_certificate() → web_server_stop() → web_server_start(), short-circuiting errors and retaining ownership on failed stop. SSH/UART0 unchanged. No tasks/depth/routes/assets/stacks added; target stack margins unknown. Boundary run.py includes certificate.c; lifecycle/policy and transport 25/tickets 12 host groups pass as reported. Credential/account then other owner slices remain; user authorized stacking, not target/M2 sign-off. See docs/phase8d7_implementation.md.

• 8D.7 first-slice history: browser reboot/web stop defer via admin_ssh_console control task; WEB owner revalidates cookie/principal/token before lifecycle calls. Pending console input is discarded (incoming-frame disposition latched before receive). web_console.c defers stop only for browser origin; other restrictions remain. Tests additionally include python3 tests/admin_console_boundary/lifecycle.py; handoff: docs/phase8d7_implementation.md. No 8D.7/M2 acceptance yet.

• Files: src/web_admin_transport.{c,h}, src/web_admin_tickets.{c,h}, protected registration/lifecycle in web_server.c, revocation through web_serial_transport_revoke_*, diagnostics in web_console.c.

• Routes: CSRF-protected admin-only POST /api/admin/ws-ticket; ordinary GET /ws/admin with cookie/Origin/ticket/shared-console admission before explicit 101. No UI entry or broker client. One socket, two tickets, existing two shared console slots; six total HTTPD sockets, LRU disabled, 16 URI handlers.

• Ownership: 20 ms ESP timer queues at most one HTTPD poll, no new task; HTTPD owns 1,552 B PSRAM-only payload and IO. Closure uses HTTPD-owned shutdown, not IDF's reusable-pointer queued close. Detach fences submitters; only successful HTTPD stop retires queued state before restart. Session/principal currentness and generation checks protect all sensitive boundaries.

• Tests: python3 tests/web_admin_transport/run.py --tickets, python3 tests/web_admin_transport/server_lifecycle.py, python3 tests/web_cookie_auth/run.py --admin; manual smoke client/procedure in tests/web_admin_transport/README.md and docs/phase8d5_implementation.md. Final shutdown fix is host-tested and build-verified by the parent's sequential final pio run; target validation remains pending.

SSH

Responsibility: authenticate SSH, route users to serial and administrators to the command dispatcher, and own wolfSSH lifecycle.

• Files: src/ssh_transport.{h,c}, src/ssh_security.{h,c}, src/ssh_console.{h,c}
• Interfaces: init/start/stop, session snapshots/disconnect/revocation, host-key replacement, counters
• Called by: startup, network clients, user revocation, console/local UI
• Dependencies: user database, broker, admin SSH console, secure random, wolfSSH/wolfSSL; current boot start gate also depends on web_security readiness
• Flow: role user -> broker; role admin -> admin_ssh_console
• Ownership: after caller-side library initialization, one task pinned to core 1 owns runtime wolfSSH contexts/sessions; two fixed generation-tagged slots.
• Security constraint: an interactive shell request is required; exec and subsystems are rejected, and no project file-transfer or forwarding route exists. PTY is not explicitly required.

Users, authentication, and authorization

Responsibility: persist bounded accounts, verify passwords/SSH keys, issue secret-free principals, and enforce account invariants.

• Files: src/user_database.{h,c}, src/user_console.{h,c}; src/admin_command_gate.{h,c} is currently a narrow recursive wrapper used only by the user command handler, not the global command serializer
• Interfaces: init/migration/recovery, authenticate, principal-currentness, account/password/role/key mutations, snapshots
• Called by: web and SSH authentication/currentness checks and console administration
• Dependencies: NVS, secure random, mbedTLS cryptography; after a committed command-layer mutation, best-effort web/SSH revocation calls supplement authoritative transport currentness checks
• Ownership: database mutex protects the internal live record and PSRAM-preferred transactional candidate; password authentication runs PBKDF2 outside the mutex and revalidates afterward, while mutation locking must be checked per operation.
• Authorization: UART0 exclusively owns initial administrator bootstrap and unavailable-database recovery; current admins may use admin SSH for other commands unless handler policy denies them. HTTPS currently treats both roles alike.
• Constraint: final administrator cannot be deleted or demoted; transport principals must be rechecked after mutations.

Administration console infrastructure

Responsibility: provide one canonical command registry and serialized execution for UART0 and admin SSH.

• Files: src/admin_ssh_console.{h,c}, src/console_input.{h,c}, src/console_completion.{h,c}, src/system_console.{h,c}, src/network_console.{h,c} and all *_console.{h,c} modules
• Entry points: admin_ssh_console_init(), admin_ssh_console_start_uart_frontend(), command registration functions
• Called by: startup, UART0 frontend, role-admin SSH transport
• Dependencies: ESP-IDF console/linenoise, all command handlers, user-principal currentness
• Flow: UART0/admin SSH -> bounded request queue -> one dispatcher -> esp_console_run()
• Ownership: dispatcher is sole esp_console_run() caller; the SSH owner exclusively performs post-initialization wolfSSH runtime calls.
• Lifecycle: remote session tokens include slot generation; fixed output/history/prompt state is wiped immediately on idle close or after an executing handler returns. Admin SSH exit and Ctrl+D on an empty command line request bounded deferred self-disconnect after best-effort output draining.
• Constraint: one slow command or prompt serializes all administration. Admin SSH is unavailable until command registration and UART frontend creation complete; supported deferred actions wait only for a bounded application-buffer drain heuristic.
• 8D.4/8D.5 boundary: admin_ssh_console_open_owned() retains explicit-index admission; runtime SSH and browser owners use admin_ssh_console_open_available() for the same two slots. Copied transport-qualified identity and immutable firmware-lifetime currentness/drain/lifecycle adapters; SSH publishes its allocated console index separately from its physical SSH slot. Owners handle liveness/output; dispatcher and prompt waits additionally require owner currentness (250 ms polling plus check/scheduling latency). SSH publishes locked principal copies; consumed console output is wiped. Completion scratch is nonblockingly serialized. Browser unsupported lifecycle/account mutations are rejected before execution. Focused host command: python3 tests/admin_console_boundary/run.py.

Wi-Fi

Responsibility: persist station/AP policy and own asynchronous ESP-NETIF/Wi-Fi state transitions.

• Files: src/wifi_config.{h,c}, src/wifi_manager.{h,c}, src/wifi_console.{h,c}, src/mdns_config.{h,c}, src/mdns_service.{h,c}, src/mdns_console.{h,c}, src/network_console.{h,c}
• Interfaces: config defaults/validate/load/save; manager init/start/stop/apply/reconnect/next-profile/snapshot
• Called by: startup, console, local UI, ESP event callbacks
• Dependencies: secure random for default AP password, NVS, ESP-NETIF/Wi-Fi/events, Espressif mDNS, lwIP diagnostics
• Lifecycle: permanent manager task and bounded queue; callbacks enqueue compact events only.
• Constraint: application NVS is authoritative (WIFI_STORAGE_RAM); working edits are not persisted until save. Start/stop, including local controls, intentionally update the RAM enabled_at_boot field. Working-config copies contain PSKs and must be tightly scoped and wiped; routine status/local UI must use secret-free snapshots.

Local display and controls

Responsibility: own OLED I2C/framebuffer operations and present status plus constrained button actions.

• Files: src/local_display.{h,c}, src/local_status_ui.{h,c}, src/local_boot_animation.{h,c}, src/local_ui_config.{h,c}, src/local_ui_console.{h,c}
• Interfaces: display init/frame/draw/commit/snapshot; UI start/activity/config; versioned NVS settings
• Called by: startup, local UI task, diagnostics, display console
• Dependencies: copied snapshots/public APIs from serial, broker, USB, Wi-Fi, web, SSH
• Ownership: local_display solely owns I2C0 and framebuffer mutex; a frame belongs to its initiating task.
• Lifecycle: the low-priority task is firmware-lifetime only if button GPIO initialization succeeds; it still runs with an absent panel so a press can reprobe after successful I2C bus setup. Failed bus creation is not recoverable by that reprobe, and display configuration commands depend on the UI task.
• Constraint: collect service snapshots before I2C; local UI never joins broker or handles secrets.

Hardware and diagnostics

Responsibility: centralize board wiring and provide bounded electrical tests with safe cleanup.

• Files: src/board_pins.h, src/rs232_hw_test.{h,c}, src/local_ui_hw_test.{h,c}, src/status_led.{h,c}
• Documentation: docs/wiring.md, docs/electrical_tests.md
• Called by: startup and debug commands
• Dependencies: physical RS-232 owner, serial/display services, ESP-IDF GPIO/UART/I2C/LED drivers
• Ownership: RS-232 diagnostics refuse to run while the service owns the port; display diagnostics reuse local_display.
• Constraint: wiring and voltage assumptions are safety-relevant; verify target hardware before running diagnostics. RGB LED initialization is currently boot-fatal, and its colors report diagnostic state rather than aggregate firmware health.

Where should I look?

Task	Start here
Change boot order or failure behavior	src/main.c, then affected subsystem init/start contracts
Change serial framing, flow control, or persistence	serial_config.*, serial_service.*, serial_console.*
Change writer/observer policy	session_broker.*, then all three transports
Debug missing or duplicated serial bytes	serial_service.c -> session_broker.c -> relevant transport task
Change USB open/DTR or line coding	usb_cdc_transport.*
Change browser terminal protocol	web_serial_transport.*, web_ui.c, web_server.c
Change HTTPS endpoints/authentication	web_server.*, web_security.*, user_database.*
Change SSH login or role routing	ssh_transport.*, ssh_security.*, user_database.*
Add or change a command	relevant *_console.c, console_completion.c, admin_ssh_console.c policy/deferred handling
Change account roles/passwords/keys	user_database.*, user_console.c, transport revocation APIs
Change Wi-Fi policy or profile persistence	wifi_manager.*, wifi_config.*, wifi_console.c
Change station mDNS hostname or persistence	mdns_service.*, mdns_config.*, mdns_console.c, then wifi_manager.c
Change OLED rendering or buttons	local_status_ui.c, local_display.*, local_ui_config.*
Change board GPIO or electrical tests	board_pins.h, hardware test module, docs/wiring.md
Change embedded browser assets	web_assets/SOURCES.md, generator, then generated data only as an explicit regeneration task
Investigate memory/watchdog regressions	broker/web/SSH bounded loops, allocation placement, root CMakeLists.txt, relevant roadmap Phase 6 history