Files
ESP32_Serial_Swiss_Army_Knife/docs/agent/current-state.md
T

9.8 KiB
Raw Blame History

Current project state

This file is working memory. Update it during active work and before handoff; do not treat it as a permanent design record.

Development state

Based on checked-in source plus README.md and docs/roadmap.md:

  • Hardware characterization, serial service, session broker, USB CDC, Wi-Fi, HTTPS/WebSocket, SSH serial transport, and local display/control are implemented and documented as target-hardware validated.
  • Phase 8A role-based user storage/UART0 administration and Phase 8B role-aware HTTPS/SSH authentication and targeted revocation are documented as target-hardware validated.
  • Phase 8C admin SSH is implemented in source, uses the shared esp_console registry, and has passed target-hardware validation.
  • Phase 8D integrated web administration is planned, not implemented. Follow docs/phase8d_plan.md: one numbered chunk per request, with baseline/resource budgeting first, target-validated login/logout (M1) before the browser admin shell (M2), then one typed-settings/control domain at a time (M3). Changing terminal modes must preserve the browser serial broker client and any writer lease. The roadmap retains the full end-state requirements.
  • Security/production hardening, OTA, BLE evaluation, advanced networking, and optional filesystem features remain future roadmap work.
  • Reserved OTA, coredump, NVS-key, and storage partitions do not imply those runtime features are implemented.

Recent memory audit

  • Fixed failed-initialization ownership leaks for wolfSSH, partial HTTPS startup, and TinyUSB teardown. Failed teardown now retains ownership and blocks unsafe duplicate initialization.
  • Serial-service RX/TX stream payloads (16 KiB and 8 KiB effective capacity) now prefer PSRAM with internal fallback; FreeRTOS controls and UART driver buffers remain internal.
  • The 5,360-byte transactional user-database candidate now prefers PSRAM with internal fallback while the live database remains internal. Candidate contents are wiped after each transaction and wiped/freed on initialization or recovery failure.
  • UART and admin-SSH completion formatter buffers were reduced from 2 KiB to 1 KiB each; current worst-case output is 890 bytes and overflow remains fail-closed.
  • Linked RAM fell from 99,508 to 92,188 bytes (7,320 bytes). PSRAM placement of serial payloads additionally removes about 24 KiB of normal internal-heap pressure on the target.
  • pio run passes. A preliminary target run reports significantly more free memory and stable, improved operation after these changes. This is useful evidence but not completion of Phase 8C validation.
  • The reviewed mDNS-enabled build uses 94,532 bytes of linked static RAM, 2,344 bytes above the earlier 92,188-byte baseline, and 1,599,765 bytes of flash. Minimizing the managed component saved 112 bytes of linked RAM and about 5.9 KiB flash versus the first mDNS build. Its 4 KiB task stack remains internal, while checked-in settings move general mDNS allocations to PSRAM and disable unused browse, component CLI, AP/ETH, and multiple-instance features. Runtime heap impact still requires target measurement.
  • Remaining targeted checks include stored/migrated/recovered user-database mutations, USB enumeration, HTTPS start/stop failure recovery where injectable, SSH initialization/login, completion display, and sustained multi-transport serial traffic while checking memory telemetry.

Clearly incomplete or transitional areas

  • Phase 8C hardware validation passed, including route separation, shared command serialization, history/completion, prompts, output backpressure, revocation during queued work, deferred SSH lifecycle/reboot actions, and full concurrent transport operation. At 460800 baud with SSH and WebSocket clients in parallel, substantial packet drops and slow display controls were observed under load, without memory exhaustion; no baud-rate reduction is planned.
  • Current HTTPS has no web-based user administration and gives both roles the same status/terminal routes.
  • Browser authentication still uses HTTP Basic; Phase 8D plans integrated login/logout sessions before exposing administrative browser routes.
  • NVS encryption, secure boot/flash encryption review, authentication rate limiting, production certificate/provisioning policy, and OTA are not implemented.

Known inconsistencies

These observations should be checked when touching the relevant area; they are not automatically bugs requiring unrelated cleanup.

  • Some source comments still call shared commands UART0-only or call the current local status/control task read-only.
  • USER_DATABASE_LOAD_EMPTY is only an initialization/failure sentinel at the checked-in revision: every successful user_database_init() path returns STORED or MIGRATED_LEGACY, so main.c's successful "new empty" log branch is unreachable.
  • SSH startup is currently gated on successful web_security initialization even though SSH uses separate host-key material. Needs verification: whether this coupling is intentional recovery policy or an accidental startup dependency.

Items to verify in future work

  • Confirm task-local Newlib standard-stream behavior if ESP-IDF/Newlib configuration changes; admin SSH command output relies on dispatcher-task stream redirection.
  • If HTTPD concurrency configuration changes, add locking around the boot-local Basic-authentication cache.

Active task — Phase 8D replanning

  • Objective: Split Phase 8D into independently reviewable, session-sized implementation and validation chunks; documentation only.
  • Context: The user reports that the earlier experimental implementation was moved to a separate branch and devel rolled back after repeated agent interruptions, a roughly +10k/-1k-line diff, broken browser login, and severe memory pressure. These symptoms have not been independently diagnosed. Do not import that branch wholesale.
  • Changes completed: Added docs/phase8d_plan.md with chunks 8D.08D.22, M1/M2 target-validation pauses, per-chunk scope/stop rules, incremental and cumulative resource gates, and next-request templates. Linked the execution plan from the roadmap and staged the existing administration acceptance matrix. Firmware, generated assets, and durable implementation architecture are unchanged.
  • Validation: Documentation links and diff consistency are checked for this planning change; no firmware build, browser test, or target memory measurement is claimed. All new implementation chunks remain planned.
  • Next: 8D.0 only: verify the rolled-back baseline, record actual build/runtime memory and supported concurrency, and define the minimal browser-authentication contract and measured budgets. Missing target evidence stays pending; do not proceed to an all-in-one implementation.
  • Risks / things to remember: Historical linked RAM/flash values below are not a fresh runtime baseline. Cookie sessions, tickets, serial sockets, admin sockets, HTTPS requests, and SSH sessions have separate capacity costs. Reserve time for validation and handoff in every implementation request; stop/split instead of accumulating an unvalidated diff.

Previous task — mDNS (target checks still pending)

  • Objective: Announce a configurable sak-<suffix>.local hostname through mDNS when Wi-Fi STA has an IPv4 address, without changing the Wi-Fi NVS blob schema.
  • Relevant files: src/mdns_config.{c,h}, src/mdns_service.{c,h}, src/mdns_console.{c,h}, src/wifi_manager.{c,h}, src/main.c, src/CMakeLists.txt, src/idf_component.yml, dependencies.lock, completion and command documentation.
  • Findings: wifi_manager already serializes all meaningful STA transitions through its permanent task; callbacks only enqueue events. This is the appropriate lifecycle owner for mDNS, while a separate configuration module preserves the existing wifi_app/config wire format.
  • Decision: Persist a fixed v1 record under mdns_cfg/config, separate from Wi-Fi configuration. Defaults derive a safe lower-case hexadecimal suffix from the STA MAC. The manager initializes mDNS at most once after validating IP_EVENT_STA_GOT_IP; the managed component's own handlers withdraw/restore the STA responder across connectivity changes, and online hostname changes use mdns_hostname_set() without teardown. Initialization failure is latched instead of retried because the resolved upstream 1.12.0 component has an unsafe partial low-memory initialization path. mDNS errors cannot fail Wi-Fi, UART0, UART1, or native USB.
  • Changes completed: Added the espressif/mdns managed dependency (resolved to 1.12.0 on IDF 5.5), mDNS config/service/console modules, mdns status|suffix|save|load|defaults|reset, completion, CMake integration, and command/architecture documentation. Minimized the component to STA-only responder use, moved general allocations to PSRAM, retained the internal task stack, and removed reconnect-time free/reinit churn. Final pio run passes at 94,532 bytes linked RAM and 1,599,765 bytes flash.
  • Remaining work: Target-hardware verification: associate a station and resolve the default sak-<mac>.local; change/save/load a suffix and confirm live reannouncement plus reboot persistence; stop Wi-Fi or remove the STA lease and confirm the record withdraws. Verify serial, native USB, and UART0 remain available if mDNS initialization fails.
  • Risks / things to remember: Hostnames are STA-only and are intentionally not announced by fallback AP mode. NVS changes to mdns_cfg/config are independent of the unchanged wifi_app/config blob. mDNS remains allocated after first successful initialization (including its internal 4 KiB task stack) to avoid fragmentation and unsafe repeated initialization; measure free/minimum/largest internal heap and mDNS stack margin during reconnect stress.

Handoff template

  • Objective:
  • Relevant files:
  • Findings:
  • Decisions made:
  • Changes completed:
  • Remaining work:
  • Risks / things to remember: