Commander1024/ESP32_Serial_Swiss_Army_Knife
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Stabilize concurrent SSH and WebSocket transports

Browse Source
This commit is contained in:
Commander1024
2026-08-26 23:53:13 +02:00
parent a350ba2f5e
commit fa339a3dfa
7 changed files with 71 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/roadmap.md
+15 -46
View File
@@ -35,7 +35,7 @@ These constraints apply across all phases:
| 4 | Wi-Fi station/AP foundation and network diagnostics | **Complete** |
| 5A | Authenticated HTTPS administration foundation | **Complete** |
| 5B | Offline xterm.js WebSocket serial terminal | **Complete** |
| 6 | Authenticated SSH serial transport | **Implemented; validation pending** |
| 6 | Authenticated SSH serial transport | **Complete** |
| 7 | Local display and button interface | **Planned** |
| 8 | Security and production hardening | **Planned** |
| 9 | Authenticated, rollback-capable OTA | **Planned** |
@@ -134,11 +134,11 @@ Implemented and hardware-validated:
- Viewport-constrained terminal fitting without recursive page growth.
- Validation with multiple clients, ANSI colors, advanced escape sequences, and full-screen terminal software.
## Current phase
## Latest completed phase
### Phase 6 — Authenticated SSH serial transport
The initial SSH implementation is present and has passed single-service hardware tests. It provides:
The SSH implementation is complete and hardware-validated. It provides:
- wolfSSH on TCP port 22 while HTTPS remains on mbedTLS.
- Password authentication using the shared `admin` credential.
@@ -154,57 +154,26 @@ The initial SSH implementation is present and has passed single-service hardware
- A single bounded wolfSSH owner task pinned to CPU 1.
- wolfSSL/wolfSSH allocations that prefer PSRAM with internal-memory fallback.
- A per-call wolfSSH receive budget to prevent pipelined input from monopolizing the owner task.
- Bounded broker and WebSocket work bursts that guarantee scheduler/idle progress during sustained serial traffic.
- Software AES for PSRAM-backed mbedTLS records plus software wolfCrypt AES/SHA, preventing mbedTLS and wolfSSL from independently reconfiguring shared ESP32 crypto hardware.
- Internal task stacks retained for safety during flash/NVS cache-disable windows.
- Heap and SSH stack telemetry through `memory` and `ssh status`.
SSH login, host-key verification, serial traffic, Web Serial coexistence, and USB CDC coexistence have been validated. Before the latest resource-placement changes, however, the following three-client arrangement could stall or fail the web service:
SSH login, host-key verification, serial traffic, Web Serial coexistence, and USB CDC coexistence have been validated. During concurrency testing, the previously failing three-client arrangement used USB CDC and SSH as writer in turn while the remaining transports observed.
1. USB CDC connected as writer.
2. Web serial connected as observer.
3. SSH connected as observer.
Early runs showed healthy heap values but CPU 0 idle-task watchdog timeouts, disproving the initial memory-pressure hypothesis. One trace exposed an unbounded broker scheduling path. A later, decisive trace repeatedly stopped inside mbedTLS's external-RAM hardware-AES DMA wait while HTTPD encrypted a WebSocket record. The broker and WebSocket producer now use bounded active bursts, mbedTLS software AES avoids the hanging PSRAM DMA path, and wolfSSL AES/SHA hardware acceleration is disabled because its private locks do not coordinate with ESP-IDF's crypto lock.
The mitigation moves broker and Wi-Fi/lwIP payload pressure toward PSRAM and moves SSH processing away from the CPU 0 Wi-Fi/web path. Phase 6 remains open until that exact arrangement is revalidated.
#### Phase 6 hardware validation
#### Phase 6 completion gate
The final target-hardware retest covered:
After flashing the current build:
1. USB CDC, one WebSocket terminal, and one SSH terminal connected concurrently.
2. SSH as writer with USB CDC and WebSocket observing.
3. USB CDC as writer with SSH and WebSocket observing.
4. Serial traffic with active readers and writer-token release/reassignment.
5. Continued HTTPS/WebSocket, SSH, USB CDC, serial-service, and UART0 responsiveness.
1. At idle, record:
```text
memory
ssh status
```
2. Reproduce this order exactly:
1. Connect USB CDC and confirm it owns the writer lease.
2. Connect one WebSocket terminal and confirm observer status.
3. Connect one SSH terminal and confirm observer status.
3. While all three are connected, collect:
```text
memory
ssh status
web status
web counters
ssh counters
broker clients
```
4. Exercise sustained RS-232 output, writer release/reassignment, and input through the newly selected writer. Verify that all observers continue receiving data and that UART0 remains responsive.
5. Disconnect the transports in different orders and verify that sockets, broker clients, and writer ownership are cleaned up.
6. Capture the complete UART0 log if any failure occurs, especially task-watchdog output, backtraces, `ESP_ERR_NO_MEM`, TLS allocation failures, lwIP/pbuf failures, or HTTP server accept/session errors.
Interpret the telemetry as follows:
- Low internal or DMA-capable free space/largest block indicates more internal-memory reduction is needed.
- Healthy heap metrics combined with a watchdog or stall indicates a scheduling or blocking-path problem rather than simple exhaustion.
- A consistently large SSH stack minimum-free value permits a measured reduction from the current 20 KiB stack; it must not be reduced speculatively.
- Low or fragmented PSRAM indicates that queue, socket, or allocation capacities need review.
Phase 6 is complete when this concurrent configuration remains stable, cleanup succeeds, the relevant counters show no unexplained loss, and no watchdog or memory-allocation error occurs.
The software-crypto build no longer reproduces the HTTPD watchdog stall. This validates that the failure was a shared hardware-crypto/PSRAM DMA problem rather than heap exhaustion. Phase 6 is complete; these concurrent arrangements remain regression tests for future transport, TLS, memory-placement, and ESP-IDF changes.
## Planned phases