Stabilize concurrent SSH and WebSocket transports

CMakeLists.txt

@@ -1,7 +1,13 @@
 cmake_minimum_required(VERSION 3.16)
 
 # Route wolfCrypt seeding through the project's sole pre-radio DRBG callback.
-add_compile_definitions(WC_RNG_SEED_CB)
+# wolfSSL's ESP32 AES/SHA locks do not coordinate with ESP-IDF mbedTLS/Wi-Fi;
+# define the documented software-fallback controls before components are parsed.
+add_compile_definitions(
+    WC_RNG_SEED_CB
+    NO_WOLFSSL_ESP32_CRYPT_AES
+    NO_WOLFSSL_ESP32_CRYPT_HASH
+)
 
 include($ENV{IDF_PATH}/tools/cmake/project.cmake)
 set(PROJECT_VER "0.1.0")

docs/roadmap.md

@@ -35,7 +35,7 @@ These constraints apply across all phases:
 | 4 | Wi-Fi station/AP foundation and network diagnostics | **Complete** |
 | 5A | Authenticated HTTPS administration foundation | **Complete** |
 | 5B | Offline xterm.js WebSocket serial terminal | **Complete** |
-| 6 | Authenticated SSH serial transport | **Implemented; validation pending** |
+| 6 | Authenticated SSH serial transport | **Complete** |
 | 7 | Local display and button interface | **Planned** |
 | 8 | Security and production hardening | **Planned** |
 | 9 | Authenticated, rollback-capable OTA | **Planned** |
@@ -134,11 +134,11 @@ Implemented and hardware-validated:
 - Viewport-constrained terminal fitting without recursive page growth.
 - Validation with multiple clients, ANSI colors, advanced escape sequences, and full-screen terminal software.
 
-## Current phase
+## Latest completed phase
 
 ### Phase 6 — Authenticated SSH serial transport
 
-The initial SSH implementation is present and has passed single-service hardware tests. It provides:
+The SSH implementation is complete and hardware-validated. It provides:
 
 - wolfSSH on TCP port 22 while HTTPS remains on mbedTLS.
 - Password authentication using the shared `admin` credential.
@@ -154,57 +154,26 @@ The initial SSH implementation is present and has passed single-service hardware
 - A single bounded wolfSSH owner task pinned to CPU 1.
 - wolfSSL/wolfSSH allocations that prefer PSRAM with internal-memory fallback.
 - A per-call wolfSSH receive budget to prevent pipelined input from monopolizing the owner task.
+- Bounded broker and WebSocket work bursts that guarantee scheduler/idle progress during sustained serial traffic.
+- Software AES for PSRAM-backed mbedTLS records plus software wolfCrypt AES/SHA, preventing mbedTLS and wolfSSL from independently reconfiguring shared ESP32 crypto hardware.
 - Internal task stacks retained for safety during flash/NVS cache-disable windows.
 - Heap and SSH stack telemetry through `memory` and `ssh status`.
 
-SSH login, host-key verification, serial traffic, Web Serial coexistence, and USB CDC coexistence have been validated. Before the latest resource-placement changes, however, the following three-client arrangement could stall or fail the web service:
+SSH login, host-key verification, serial traffic, Web Serial coexistence, and USB CDC coexistence have been validated. During concurrency testing, the previously failing three-client arrangement used USB CDC and SSH as writer in turn while the remaining transports observed.
 
-1. USB CDC connected as writer.
+Early runs showed healthy heap values but CPU 0 idle-task watchdog timeouts, disproving the initial memory-pressure hypothesis. One trace exposed an unbounded broker scheduling path. A later, decisive trace repeatedly stopped inside mbedTLS's external-RAM hardware-AES DMA wait while HTTPD encrypted a WebSocket record. The broker and WebSocket producer now use bounded active bursts, mbedTLS software AES avoids the hanging PSRAM DMA path, and wolfSSL AES/SHA hardware acceleration is disabled because its private locks do not coordinate with ESP-IDF's crypto lock.
-2. Web serial connected as observer.
-3. SSH connected as observer.
 
-The mitigation moves broker and Wi-Fi/lwIP payload pressure toward PSRAM and moves SSH processing away from the CPU 0 Wi-Fi/web path. Phase 6 remains open until that exact arrangement is revalidated.
+#### Phase 6 hardware validation
 
-#### Phase 6 completion gate
+The final target-hardware retest covered:
 
-After flashing the current build:
+1. USB CDC, one WebSocket terminal, and one SSH terminal connected concurrently.
+2. SSH as writer with USB CDC and WebSocket observing.
+3. USB CDC as writer with SSH and WebSocket observing.
+4. Serial traffic with active readers and writer-token release/reassignment.
+5. Continued HTTPS/WebSocket, SSH, USB CDC, serial-service, and UART0 responsiveness.
 
-1. At idle, record:
+The software-crypto build no longer reproduces the HTTPD watchdog stall. This validates that the failure was a shared hardware-crypto/PSRAM DMA problem rather than heap exhaustion. Phase 6 is complete; these concurrent arrangements remain regression tests for future transport, TLS, memory-placement, and ESP-IDF changes.
-
-   ```text
-   memory
-   ssh status
-   ```
-
-2. Reproduce this order exactly:
-
-   1. Connect USB CDC and confirm it owns the writer lease.
-   2. Connect one WebSocket terminal and confirm observer status.
-   3. Connect one SSH terminal and confirm observer status.
-
-3. While all three are connected, collect:
-
-   ```text
-   memory
-   ssh status
-   web status
-   web counters
-   ssh counters
-   broker clients
-   ```
-
-4. Exercise sustained RS-232 output, writer release/reassignment, and input through the newly selected writer. Verify that all observers continue receiving data and that UART0 remains responsive.
-5. Disconnect the transports in different orders and verify that sockets, broker clients, and writer ownership are cleaned up.
-6. Capture the complete UART0 log if any failure occurs, especially task-watchdog output, backtraces, `ESP_ERR_NO_MEM`, TLS allocation failures, lwIP/pbuf failures, or HTTP server accept/session errors.
-
-Interpret the telemetry as follows:
-
-- Low internal or DMA-capable free space/largest block indicates more internal-memory reduction is needed.
-- Healthy heap metrics combined with a watchdog or stall indicates a scheduling or blocking-path problem rather than simple exhaustion.
-- A consistently large SSH stack minimum-free value permits a measured reduction from the current 20 KiB stack; it must not be reduced speculatively.
-- Low or fragmented PSRAM indicates that queue, socket, or allocation capacities need review.
-
-Phase 6 is complete when this concurrent configuration remains stable, cleanup succeeds, the relevant counters show no unexplained loss, and no watchdog or memory-allocation error occurs.
 
 ## Planned phases
 

sdkconfig.defaults

@@ -17,6 +17,11 @@ CONFIG_LWIP_TCP_OOSEQ_MAX_PBUFS=4
 # Active TLS material remains unencrypted in PSRAM until the hardening phase.
 CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC=y
 # CONFIG_MBEDTLS_INTERNAL_MEM_ALLOC is not set
+# Hardware AES can hang in the PSRAM DMA path and cannot be shared safely with
+# wolfSSL's independently locked ESP32 acceleration. Software AES is fast enough
+# at 240 MHz for bounded serial traffic and leaves ESP-IDF's SHA/MPI paths intact.
+# CONFIG_MBEDTLS_HARDWARE_AES is not set
+# CONFIG_MBEDTLS_AES_USE_INTERRUPT is not set
 # HTTPS remains on ESP-IDF's mbedTLS backend; wolfSSL is linked only for wolfSSH.
 CONFIG_ESP_TLS_USING_MBEDTLS=y
 
@@ -40,6 +45,7 @@ CONFIG_ESP_MAIN_TASK_STACK_SIZE=8192
 
 # Build wolfSSH and wolfCrypt without replacing the HTTPS TLS implementation.
 CONFIG_ESP_ENABLE_WOLFSSH=y
+
 # The managed component emits a generic RSA stack warning although this target disables RSA.
 CONFIG_ESP_WOLFSSL_NO_STACK_SIZE_BUILD_WARNING=y
 

src/session_broker.c

@@ -11,6 +11,7 @@
 #include "serial_service.h"
 
 #define SESSION_BROKER_RX_CHUNK_SIZE 256U
+#define SESSION_BROKER_ACTIVE_BURST_BYTES (8U * SESSION_BROKER_RX_CHUNK_SIZE)
 #define SESSION_BROKER_TASK_STACK_SIZE 4096U
 #define SESSION_BROKER_TASK_PRIORITY 9U
 #define SESSION_BROKER_IDLE_POLL_MS 5U
@@ -160,6 +161,7 @@ static void broker_task(void *context)
 {
     (void)context;
     uint8_t data[SESSION_BROKER_RX_CHUNK_SIZE];
+    size_t active_burst_bytes = 0U;
 
     for (;;) {
         /*
@@ -175,7 +177,20 @@ static void broker_task(void *context)
         xSemaphoreGive(s_mutex);
 
         if (received == 0U) {
+            active_burst_bytes = 0U;
             vTaskDelay(milliseconds_to_ticks(SESSION_BROKER_IDLE_POLL_MS));
+        } else {
+            active_burst_bytes += received;
+            if (active_burst_bytes >= SESSION_BROKER_ACTIVE_BURST_BYTES) {
+                active_burst_bytes = 0U;
+                /*
+                 * A continuously readable UART must not make this priority-9
+                 * task permanently runnable. One tick after each bounded burst
+                 * preserves 1 Mbaud headroom while allowing idle and transports
+                 * to run on a loaded dual-core system.
+                 */
+                vTaskDelay(1U);
+            }
         }
     }
 }

src/ssh_transport.c

@@ -18,6 +18,7 @@
 #include "lwip/inet.h"
 #include "lwip/sockets.h"
 #include "lwip/tcp.h"
+#include "sdkconfig.h"
 #include "secure_random.h"
 #include "serial_service.h"
 #include "ssh_security.h"
@@ -26,6 +27,14 @@
 #include <wolfssl/wolfcrypt/random.h>
 #include <wolfssh/ssh.h>
 
+#if defined(CONFIG_MBEDTLS_HARDWARE_AES) && CONFIG_MBEDTLS_HARDWARE_AES
+#error "Concurrent mbedTLS/wolfSSH operation requires mbedTLS software AES"
+#endif
+#if !defined(NO_WOLFSSL_ESP32_CRYPT_AES) || \
+    !defined(NO_WOLFSSL_ESP32_CRYPT_HASH)
+#error "wolfSSH AES/SHA must not use independently locked ESP32 crypto hardware"
+#endif
+
 #define SSH_TRANSPORT_TASK_STACK_SIZE 20480U
 #define SSH_TRANSPORT_TASK_PRIORITY 5U
 #define SSH_TRANSPORT_TASK_CORE 1

src/usb_cdc_transport.c

@@ -58,7 +58,7 @@ static bool s_line_coding_pending;
 static usb_cdc_transport_counters_t s_counters;
 
 static const char s_language_descriptor[] = {0x09, 0x04};
-static const char s_manufacturer[] = "ESP32 Serial Tools";
+static const char s_manufacturer[] = "Commander1024";
 /* esp_tinyusb's default UTF-16 conversion accepts at most 31 characters. */
 static const char s_product[] = "ESP32 Serial Swiss Army Knife";
 static char s_serial_number[13];

src/web_serial_transport.c

@@ -27,7 +27,8 @@
 #define WEB_SERIAL_ORIGIN_CAPACITY (sizeof("https://") + WEB_SERIAL_HOST_CAPACITY)
 #define WEB_SERIAL_TICKET_RESPONSE_CAPACITY 96U
 #define WEB_SERIAL_TASK_STACK_SIZE 6144U
-#define WEB_SERIAL_TASK_PRIORITY 7U
+#define WEB_SERIAL_TASK_PRIORITY 4U
+#define WEB_SERIAL_ACTIVE_BURST_LOOPS 8U
 #define WEB_SERIAL_POLL_MS 5U
 #define WEB_SERIAL_DETACH_TIMEOUT_US 1000000LL
 
@@ -1214,6 +1215,8 @@ static void process_active_output(web_serial_slot_t *slot)
 static void transport_task(void *context)
 {
     (void)context;
+    size_t active_burst_loops = 0U;
+
     for (;;) {
         for (size_t index = 0U; index < WEB_SERIAL_TRANSPORT_MAX_SESSIONS;
              ++index) {
@@ -1222,8 +1225,20 @@ static void transport_task(void *context)
             process_broker_disconnect(slot);
             process_active_output(slot);
         }
-        (void)ulTaskNotifyTake(pdTRUE,
+
-                              milliseconds_to_ticks(WEB_SERIAL_POLL_MS));
+        uint32_t notifications = ulTaskNotifyTake(
+            pdTRUE, milliseconds_to_ticks(WEB_SERIAL_POLL_MS));
+        if (notifications == 0U) {
+            active_burst_loops = 0U;
+        } else if (++active_burst_loops >= WEB_SERIAL_ACTIVE_BURST_LOOPS) {
+            active_burst_loops = 0U;
+            /*
+             * Send completion normally wakes this task immediately. Bound that
+             * producer/HTTPD hand-off so sustained serial output cannot keep an
+             * application task runnable forever and starve an idle watchdog.
+             */
+            vTaskDelay(1U);
+        }
     }
 }