ESP32_Serial_Swiss_Army_Knife/docs/agent/code-map.md

Code map

This is a semantic map, not a complete file inventory. Start here, then read the listed headers and only the implementation paths relevant to the task.

Bootstrap and system composition

Responsibility: establish startup order, recovery behavior, configuration loading, service dependencies, and command registration.

• Files: src/main.c, src/CMakeLists.txt, root CMakeLists.txt, platformio.ini, partitions.csv, src/idf_component.yml; inspect targeted settings in sdkconfig.defaults when crypto, PSRAM, HTTPS/HTTPD, USB, or socket capacity matters
• Entry point: app_main()
• Called by: ESP-IDF runtime
• Dependencies: every subsystem initializer
• Lifecycle constraint: optional display/network failures should not remove UART0 administrative recovery or USB UART1 access; the custom administration frontend starts only after command registration.

Secure randomness

Responsibility: provide the sole project-owned, mutex-serialized application DRBG, seeded before Wi-Fi/radio use.

• Files: src/secure_random.{h,c}
• Interfaces: secure_random_init(), random-byte helpers, secure_wipe()
• Called by: HTTPS material, SSH keys, users, Wi-Fi defaults, tickets, the HTTPS authentication cache, and the wolfCrypt seed callback
• Constraint: initialization order is security-significant; the DRBG deliberately avoids post-radio reseeding and fails closed at its generation limit. Do not add independent weak RNGs or radio-dependent early entropy paths.

Physical RS-232 and serial service

Responsibility: protect the MAX3243/UART resource, own UART1 while running, buffer binary RX/TX, apply serial configuration, and expose status/counters.

• Files: src/rs232_port_owner.{h,c}, src/serial_service.{h,c}, src/serial_config.{h,c}, src/serial_console.{h,c}
• Interfaces: owner claim/release/fault; serial init/start/stop/read/write/configuration/snapshots; versioned NVS load/save
• Normal data caller: session_broker; USB, WebSocket, role-user SSH, console, and local UI also call serial lifecycle/configuration APIs as appropriate
• Dependencies: ESP-IDF UART driver, board_pins.h, NVS
• Ownership: the cooperative owner arbitrates active diagnostics (PHASE0) against the service (SERVICE); boot-time static-safe GPIO initialization and service-owned static-mode restoration are explicit exceptions. Unsafe cleanup marks FAULT until reboot.
• Lifecycle: stop/reconfiguration discards serial-service RX/TX and task-local pending bytes, but leaves broker clients, writer ownership, events, and already-fanned output intact. The 16 KiB RX and 8 KiB TX payloads prefer PSRAM; their FreeRTOS controls and UART driver storage remain internal.

Session broker

Responsibility: mediate all transport access to the serial service; provide one writer lease and multiple isolated observers.

• Files: src/session_broker.{h,c}, src/session_console.{h,c}
• Interfaces: connect/disconnect, request/release/force writer, nonblocking read/write/event APIs, snapshots and counters; session_broker_get_management_snapshot() and session_broker_assign_writer_current() atomically project/compare target and lease generation. Exhausted 29-bit client generations retire; saturated lease generation preserves ordinary recovery. Tests: tests/session_broker_diagnostics/run.py.
• Called by: USB, web serial, role-user SSH, console tests, local UI snapshots/actions
• Dependencies: serial_service
• Data path: transport -> broker -> serial service -> UART1; reverse data is fanned out per client.
• Ownership: client IDs are slot/generation-safe; events are advisory and can drop, so use snapshots as authority.
• Lifecycle: one permanent task and eight preallocated client slots; slow output drops only for the affected client.
• Diagnostics: broker counters adds active-client ID/type/pending/HWM/UART/queued/read/dropped rows; clear seeds HWM from pending, disconnect removes rows but retains global discard accounting. tests/session_broker_diagnostics/run.py; capture before disconnect, never use consuming broker read as a probe. Semantics/recipe: docs/web_throughput_diagnostics.md.

Native USB CDC

Responsibility: adapt TinyUSB CDC host state/data to one broker client.

• Files: src/usb_cdc_transport.{h,c}, src/usb_console.{h,c}
• Interfaces: usb_cdc_transport_init(), snapshots/counters, queued writer request/release
• Called by: startup, TinyUSB callbacks, console/local UI
• Dependencies: TinyUSB, broker, serial service
• Flow: USB host <-> USB task <-> broker
• Lifecycle: permanent owner task; broker client exists only while attached with host DTR asserted.
• Constraint: callbacks retain the latest host line coding only for diagnostics; it never reconfigures UART1. UART1 framing and speed remain controlled by the serial configuration and its explicit persistence commands.

Web and WebSocket serial

Responsibility: serve authenticated HTTPS UI/API, issue WebSocket tickets, and adapt browser serial sessions to broker clients.

• Files: src/web_server.{h,c}, src/web_serial_transport.{h,c}, src/web_ui.{h,c}, src/web_console.{h,c}
• Ordinary HTTPS idle cleanup: src/web_httpd_idle.{c,h}, owner sweep in web_httpd_adapter.{c,h}, lifecycle/TLS composition in web_server.c; tests/web_httpd_idle/run.py. Independent of diagnostics/optional transports: 15-second observed idle, one-second timer/one queued probe, six rows, actual WS/async/pending exemptions, safe current-owner shutdown and stop/restart fencing. No LRU/socket/timeout/stack increase. SDK queue/owner-delay and regression contract: docs/https_idle_cleanup.md.
• Independent throughput diagnostics: web_serial_transport.{c,h} owns two fixed per-slot binary-TX aggregates and epoch fences; web_console.c exposes default-disabled web performance enable|disable|show|clear. Queue-entry/callback-entry, synchronous-send and completion/drain-return estimates, not peer receipt or scheduler-only latency. tests/web_serial_performance/run.py; resource/evidence limits and UART0 paired capture: docs/web_throughput_diagnostics.md.
• Storage compatibility: user_database persists missing storage empty and preserves valid v1 user bytes; private derived v1_admin_marker, no public bootstrap/migration/sync APIs. web_security privately migrates v1 1392-byte material to TLS-only v2 1340-byte material, exact identity/generation retained, commit before publish, fail closed without fallback overwrite. Credential commands removed; user generated passwords and TLS rotation remain. Contracts, downgrade and evidence limits: docs/legacy_credential_removal.md.
• Security files: src/web_security.{h,c}, src/web_cookie_auth.{h,c}, src/web_session_store.{h,c}, src/web_auth_parse.{h,c}. Private IDF boundary: src/web_httpd_adapter.{h,c}.
• HTTP policy/UI: web_cookie_auth + web_auth_parse enforce bounded cookie/Origin/CSRF/admin admission; web_login_ui.{c,h} serves login, web_ui.c owns session-fenced Serial/Admin/Settings and shared quick controllers. Tests: tests/web_cookie_auth/run.py (domain variants), tests/web_auth_parse/run.py, tests/web_login_ui/run.py, tests/web_ui_session/run.py.
• Admission diagnostics: web_diagnostics.{c,h}, tests/web_diagnostics/run.py; six post-TLS records/32-event opt-in ring, no HTTPD off-owner inspection. Contract.
• Identity/lifecycle: web_server_replace_identity() + web_security reserve service before identity; commit before reserved stop/start, no rollback after commit. web_lifecycle_settings.{c,h} owns original-login ID/ACK handoff. Tests: tests/web_security/run.py, tests/web_admin_transport/server_lifecycle.py.
• Asset files: authored/generated boundary in src/web_assets_data.{h,c}, web_assets/SOURCES.md, web_assets/generate_embedded_assets.py
• Interfaces: web init/start/stop/snapshots; HTTP handlers; ticket mint/consume; attach/detach; targeted session revocation
• Called by: startup, ESP-IDF HTTPS server, user administration revocation, console/local UI
• Dependencies: user database, secure random, broker, successful Wi-Fi manager initialization at boot, mbedTLS/HTTPS server; actual network reachability is an operational prerequisite, not an initializer invariant
• Flow: browser -> HTTPS login/cookie session -> CSRF-protected ticket -> cookie/Origin/ticket admission -> WebSocket -> web transport -> broker
• Ownership: HTTPD owns socket send/close work; transport task owns broker mediation; two fixed WebSocket slots and four outstanding tickets.
• Security constraints: Basic/cache removed; four absolute one-hour cookie sessions revalidate principal currentness. Four pre-login challenges (120 s), five credential attempts/60 s globally, no live session/challenge/ticket eviction. Origin/CSRF required for mutations; Origin/cookie/ticket before upgrade. Disconnect pauses reconnect but retains login; Sign out invalidates its session. Authored loader changes must update their hard-coded CSP hashes atomically.
• Session-store boundary: admitted HTTPS start initializes records; auth-init failure gates HTTPS. Failed start/accepted stop disables and wipes state. Tickets/slots require nonzero non-reused session IDs; session/account/global revocation invalidates store records before socket cleanup. RNG/SHA/database calls run outside short portMUX sections; ID/expiry/epoch checks reject stale work. Run python3 tests/web_session_store/run.py and its --serial integration mode.
• Asset constraint: web_assets_data.c is checked-in generated input to the build; do not hand-edit or regenerate casually.

Browser admin backend

• Files: src/web_admin_transport.{c,h}, src/web_admin_tickets.{c,h}, protected registration/lifecycle in web_server.c, revocation through web_serial_transport_revoke_*, diagnostics in web_console.c.
• Routes: CSRF-protected admin-only POST /api/admin/ws-ticket; ordinary GET /ws/admin with cookie/Origin/ticket/shared-console admission before explicit 101. Admin UI entry is explicit; no admin broker client. One socket, two tickets, existing two shared console slots; six total HTTPD sockets, LRU disabled; current overall capacity is 39 URI handlers.
• Currentness/policy: admin_ssh_console_open_available() shares two slots with runtime SSH; transport-qualified tokens and owner adapters revalidate outside console locks before commands/prompts. Parsed browser policy remains narrower than typed Settings; shell contract. Tests: tests/admin_console_boundary/{run,accounts,lifecycle}.py, tests/admin_ssh_policy/run.py, tests/web_admin_transport/run.py --tickets, tests/web_cookie_auth/run.py --admin.
• Ownership: 20 ms ESP timer queues at most one HTTPD poll, no new task; HTTPD owns 1,552 B PSRAM-only payload and IO. Closure uses HTTPD-owned shutdown, not IDF's reusable-pointer queued close. Detach fences submitters; only successful HTTPD stop retires queued state before restart. Session/principal currentness and generation checks protect all sensitive boundaries.

Typed settings source and regression map

HTTPD reads zero-wait projections and queues only IDs to the existing dispatcher. One original-login slot per domain; canonical owners compare/reserve at execution. API/lifetime and failure contracts.

Domain	API / canonical owner	Focused source tests
Serial	web_serial_settings.{c,h} / serial_service	tests/web_cookie_auth/run.py --serial-settings, tests/admin_console_boundary/run.py
Accounts / keys	web_account_settings.{c,h} / user_database	tests/web_cookie_auth/run.py --accounts, tests/admin_console_boundary/accounts.py
Network	web_network_settings.{c,h} / wifi_manager, mdns_service	tests/web_cookie_auth/run.py --network, tests/web_network_settings/run.py
Display	web_display_settings.{c,h} / local_status_ui	tests/web_cookie_auth/run.py --display
Broker	web_broker_settings.{c,h} / session_broker	tests/web_cookie_auth/run.py --broker, tests/session_broker_diagnostics/run.py
SSH	web_ssh_settings.{c,h} / ssh_transport, ssh_security	tests/web_cookie_auth/run.py --ssh, tests/ssh_management/run.py, tests/ssh_management/security.py
HTTPS / reboot	web_lifecycle_settings.{c,h} / web_server, web_security	tests/web_cookie_auth/run.py --lifecycle, tests/web_admin_transport/server_lifecycle.py

Shared UI regression: tests/web_ui_session/run.py and its domain .cjs fixtures / layout.py. These are navigation pointers, not claims of test execution.

SSH

Responsibility: authenticate SSH, route users to serial and administrators to the command dispatcher, and own wolfSSH lifecycle.

• Files: src/ssh_transport.{h,c}, src/ssh_security.{h,c}, src/ssh_console.{h,c}
• Interfaces: init/start/stop, session snapshots/disconnect/revocation, host-key replacement, counters; ssh_transport_get_management_snapshot() / ssh_transport_manage_current() fence lifecycle and exact session admission. ssh_transport_replace_identity() reserves service before task-bound security identity across stop/commit/restart, retains context until all slots retire and rejects orphan starts. Tests: tests/ssh_management/run.py, tests/ssh_management/security.py, tests/ssh_management/runtime.py.
• Called by: startup, network clients, user revocation, console/local UI
• Dependencies: user database, broker, admin SSH console, secure random, wolfSSH/wolfSSL; boot start gate requires Wi-Fi and SSH security/runtime readiness, independently of HTTPS identity readiness (verified in main.c after accepted legacy cleanup).
• Flow: role user -> broker; role admin -> admin_ssh_console
• Ownership: after caller-side library initialization, one task pinned to core 1 owns runtime wolfSSH contexts/sessions; two fixed generation-tagged slots.
• Security constraint: an interactive shell request is required; exec and subsystems are rejected, and no project file-transfer or forwarding route exists. PTY is not explicitly required.

Users, authentication, and authorization

Responsibility: persist bounded accounts, verify passwords/SSH keys, issue secret-free principals, and enforce account invariants.

• Files: src/user_database.{h,c}, src/user_console.{h,c}; src/admin_command_gate.{h,c} is currently a narrow recursive wrapper used only by the user command handler, not the global command serializer
• Interfaces: credential-independent init/empty recovery, authenticate, principal-currentness, account/password/role/key mutations, snapshots
• Called by: web and SSH authentication/currentness checks and console administration
• Dependencies: NVS, secure random, mbedTLS cryptography; after a committed command-layer mutation, best-effort web/SSH revocation calls supplement authoritative transport currentness checks
• Ownership: database mutex protects the internal live record and PSRAM-preferred transactional candidate; password authentication runs PBKDF2 outside the mutex and revalidates afterward, while mutation locking must be checked per operation.
• Authorization: UART0 establishes the first administrator through normal user add and exclusively owns unavailable-database recovery to empty (healthy database refused); current admins may use admin SSH for other commands unless handler policy denies them. HTTPS serial/status permits both roles; administration requires admin.
• Constraint: final administrator cannot be deleted or demoted; transport principals must be rechecked after mutations.

Administration console infrastructure

Responsibility: provide one canonical command registry and serialized execution for UART0, admin SSH and browser admin.

• Files: src/admin_ssh_console.{h,c}, src/console_input.{h,c}, src/console_completion.{h,c}, src/system_console.{h,c}, src/network_console.{h,c} and all *_console.{h,c} modules
• Entry points: admin_ssh_console_init(), admin_ssh_console_start_uart_frontend(), command registration functions
• Called by: startup, UART0 frontend, role-admin SSH transport, browser admin transport
• Dependencies: ESP-IDF console/linenoise, all command handlers, user-principal currentness
• Flow: UART0/admin SSH/browser admin -> bounded request queue -> one dispatcher -> esp_console_run()
• Ownership: dispatcher is sole esp_console_run() caller; the SSH owner exclusively performs post-initialization wolfSSH runtime calls.
• Lifecycle: remote session tokens include slot generation; fixed output/history/prompt state is wiped immediately on idle close or after an executing handler returns. Admin SSH exit and Ctrl+D on an empty command line request bounded deferred self-disconnect after best-effort output draining.
• Constraint: one slow command or prompt serializes all administration. Admin SSH is unavailable until command registration and UART frontend creation complete; supported deferred actions wait only for a bounded application-buffer drain heuristic.

Wi-Fi

Responsibility: persist station/AP policy and own asynchronous ESP-NETIF/Wi-Fi state transitions.

• Files: src/wifi_config.{h,c}, src/wifi_manager.{h,c}, src/wifi_console.{h,c}, src/mdns_config.{h,c}, src/mdns_service.{h,c}, src/mdns_console.{h,c}, src/network_console.{h,c}
• Interfaces: config defaults/validate/load/save; manager init/start/stop/apply/reconnect/next-profile/snapshot
• Dependencies: secure random for default AP password, NVS, ESP-NETIF/Wi-Fi/events, Espressif mDNS, lwIP diagnostics
• Lifecycle: permanent manager task and bounded queue; callbacks enqueue compact events only.
• Constraint: application NVS is authoritative (WIFI_STORAGE_RAM); working edits are not persisted until save. Start/stop, including local controls, intentionally update the RAM enabled_at_boot field. Working-config copies contain PSKs and must be tightly scoped and wiped; routine status/local UI must use secret-free snapshots.

Local display and controls

Responsibility: own OLED I2C/framebuffer operations and present status plus constrained button actions.

• Files: src/local_display.{h,c}, src/local_status_ui.{h,c}, src/local_boot_animation.{h,c}, src/local_ui_config.{h,c}, src/local_ui_console.{h,c}
• Interfaces: display init/frame/draw/commit/snapshot; UI start/activity/config; generation-checked settings projection/update and explicit persistence reservation; versioned NVS settings
• Called by: startup, local UI task, diagnostics, display console
• Dependencies: copied snapshots/public APIs from serial, broker, USB, Wi-Fi, web, SSH
• Ownership: local_display solely owns I2C0 and framebuffer mutex; a frame belongs to its initiating task.
• Lifecycle: the low-priority task is firmware-lifetime only if button GPIO initialization succeeds; it still runs with an absent panel so a press can reprobe after successful I2C bus setup. Failed bus creation is not recoverable by that reprobe, and display configuration commands depend on the UI task.
• Constraint: collect service snapshots before I2C; local UI never joins broker or handles secrets. All configuration writers honor the UI owner's zero-wait reservation; NVS runs outside timing critical sections. Reset commits defaults before RAM publication, including CLI; buttons/diagnostic holds update activity, not configuration generation.

Hardware and diagnostics

Responsibility: centralize board wiring and provide bounded electrical tests with safe cleanup.

• Files: src/board_pins.h, src/rs232_hw_test.{h,c}, src/local_ui_hw_test.{h,c}, src/status_led.{h,c}
• Documentation: docs/wiring.md, docs/electrical_tests.md
• Called by: startup and debug commands
• Dependencies: physical RS-232 owner, serial/display services, ESP-IDF GPIO/UART/I2C/LED drivers
• Ownership: RS-232 diagnostics refuse to run while the service owns the port; display diagnostics reuse local_display.
• Constraint: wiring and voltage assumptions are safety-relevant; verify target hardware before running diagnostics. RGB LED initialization is currently boot-fatal, and its colors report diagnostic state rather than aggregate firmware health.

Where should I look?

Task	Start here
Change boot order or failure behavior	src/main.c, then affected subsystem init/start contracts
Change serial framing, flow control, or persistence	serial_config.*, serial_service.*, serial_console.*
Change writer/observer policy	session_broker.*, then all three transports
Debug missing or duplicated serial bytes	serial_service.c -> session_broker.c -> relevant transport task
Change USB open/DTR or line coding	usb_cdc_transport.*
Change browser terminal protocol	web_serial_transport.*, web_ui.c, web_server.c
Change HTTPS endpoints/authentication	web_server.*, web_security.*, user_database.*
Change SSH login or role routing	ssh_transport.*, ssh_security.*, user_database.*
Add or change a command	relevant *_console.c, console_completion.c, admin_ssh_console.c policy/deferred handling
Change account roles/passwords/keys	user_database.*, user_console.c, transport revocation APIs
Change Wi-Fi policy or profile persistence	wifi_manager.*, wifi_config.*, wifi_console.c
Change station mDNS hostname or persistence	mdns_service.*, mdns_config.*, mdns_console.c, then wifi_manager.c
Change OLED rendering or buttons	local_status_ui.c, local_display.*, local_ui_config.*
Change board GPIO or electrical tests	board_pins.h, hardware test module, docs/wiring.md
Change embedded browser assets	web_assets/SOURCES.md, generator, then generated data only as an explicit regeneration task
Investigate memory/watchdog regressions	broker/web/SSH bounded loops, allocation placement, root CMakeLists.txt, relevant roadmap Phase 6 history