Commander1024/ESP32_Serial_Swiss_Army_Knife
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Implemented initial SSH support. Memory pressure too high for HTTPS and

Browse Source 
SSH. Dirty commit.
This commit is contained in:
Commander1024
2026-08-24 22:30:34 +02:00
parent c018bfe361
commit 72d030bc7a
17 changed files with 2314 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+81 -13
View File
@@ -12,7 +12,7 @@ Universal wireless serial adaptor firmware for the ESP32-S3.
- 8 MB octal PSRAM
- Adafruit MAX3243 full-pinout RS-232 breakout, product 5988
The firmware has completed **Phase 0 hardware characterization**, the **Phase 1 serial-core foundation**, the **Phase 2 transport-neutral session broker**, native USB CDC-ACM, the **Phase 4 Wi-Fi foundation**, and the **Phase 5 authenticated HTTPS web terminal**. Phase 5B adds an offline xterm.js interface and bounded WebSocket transport to the persistent HTTPS identity and recovery foundation from Phase 5A. The MAX3243 diagnostics and recovery consoles remain available. No electrical test starts automatically; UART1 starts when requested explicitly or when a native USB or authenticated web-terminal session opens.
The firmware has completed **Phase 0 hardware characterization**, the **Phase 1 serial-core foundation**, the **Phase 2 transport-neutral session broker**, native USB CDC-ACM, the **Phase 4 Wi-Fi foundation**, the **Phase 5 authenticated HTTPS web terminal**, and the **Phase 6 authenticated SSH serial transport**. Phase 5B provides an offline xterm.js interface and bounded WebSocket transport; Phase 6 adds a bounded, password-authenticated, shell-only SSH path with an independent persistent host identity. The MAX3243 diagnostics and recovery consoles remain available. No electrical test starts automatically; UART1 starts when requested explicitly or when native USB, an authenticated web terminal, or an authenticated SSH session opens.
## Hardware wiring
@@ -24,7 +24,7 @@ The N16R8 target has 16 MiB flash and 8 MiB octal PSRAM. PlatformIO uses the cus
| Partition | Offset | Size | Purpose |
|---|---:|---:|---|
| `nvs` | `0x009000` | 512 KiB | Serial, Wi-Fi, HTTPS credential, and certificate/key data |
| `nvs` | `0x009000` | 512 KiB | Serial, Wi-Fi, shared admin credential, HTTPS identity, and SSH host-key data |
| `otadata` | `0x089000` | 8 KiB | Active OTA-slot selection metadata |
| `phy_init` | `0x08B000` | 4 KiB | Optional PHY initialization data |
| `nvs_key` | `0x08C000` | 4 KiB | Reserved for future encrypted-NVS keys |
@@ -68,7 +68,7 @@ pio device monitor -b 115200
The firmware starts an interactive console on UART0 with the prompt `serial-tool>`. Type `help` to display concise root-command descriptions. This USB-to-UART device normally appears as `/dev/ttyUSB*`; it is separate from the native USB CDC serial transport described below.
The console provides line editing, history for the current session, terminal-aware cursor movement, and Tab completion. ESP-IDF supplies completion for root commands; the project extends it to nested `debug`, `serial`, `broker`, `usb`, `wifi`, and `web` subcommands plus safe fixed values such as AP policy and serial framing. Password values are intentionally never completion candidates.
The console provides line editing, history for the current session, terminal-aware cursor movement, and Tab completion. ESP-IDF supplies completion for root commands; the project extends it to nested `debug`, `serial`, `broker`, `usb`, `wifi`, `web`, and `ssh` subcommands plus safe fixed values such as AP policy and serial framing. Password values are intentionally never completion candidates.
ESP-IDF normally probes terminal cursor support once while constructing the UART REPL. If the board boots without a terminal attached, that probe times out and selects dumb mode. The DevKit's USB-to-UART bridge exposes no host-open signal that firmware can monitor, and entering enhanced mode before a terminal exists would block linenoise while it waits for a cursor-position response.
@@ -119,7 +119,7 @@ UART1 has exclusive ownership while the service runs. Phase 0 commands will refu
### Phase 2 session broker
The broker is initialized at boot and continuously drains the serial service whenever UART1 is running. It is transport-neutral: console test clients and native USB CDC use the same API that WebSocket and SSH transports will use later.
The broker is initialized at boot and continuously drains the serial service whenever UART1 is running. It is transport-neutral: console test clients, native USB CDC, authenticated WebSocket terminals, and authenticated SSH sessions all use the same API.
```text
broker status
@@ -208,7 +208,7 @@ Power down and remove the DE-9 pin 3-to-2 jumper before connecting an external s
### Wi-Fi foundation
Wi-Fi is managed independently of the serial-session broker. It provides network connectivity and recovery access-point policy for the HTTPS service. The firmware does not run plaintext HTTP, DNS interception, a captive portal, NAPT, or any TCP serial listener.
Wi-Fi is managed independently of the serial-session broker. It provides network connectivity and recovery access-point policy for the HTTPS and SSH services. The firmware does not run plaintext HTTP, DNS interception, a captive portal, NAPT, or any unauthenticated/plaintext TCP serial listener.
Configuration uses four fixed station-profile slots. Lower numeric priority values are tried first, with slot number breaking ties. Profiles support WPA2/WPA3 mixed operation or require WPA3-SAE. ESP-IDF's station threshold can express “WPA2 or stronger” but not a strict WPA2-only maximum, so the configuration does not pretend to offer a distinct WPA2-only mode. Each profile attempt has a 12-second association/DHCP deadline. After all enabled profiles fail, the manager uses exponential retry delays from 2 to 60 seconds.
@@ -294,12 +294,13 @@ The document CSP permits scripts only from the device itself. xterm.js requires
On first boot, the device generates and persists:
- username `admin` and a random 24-character Base64URL-safe password;
- an ECDSA P-256 private key;
- username `admin` and a random 24-character Base64URL-safe password shared by HTTPS and SSH;
- an ECDSA P-256 HTTPS private key;
- a device-specific self-signed SHA-256 certificate valid from 2025-01-01 through 2049-12-31;
- certificate SANs for `192.168.4.1` and a MAC-suffixed name such as `esp32-sak-a1b2c3.local`.
- certificate SANs for `192.168.4.1` and a MAC-suffixed name such as `esp32-sak-a1b2c3.local`;
- a separate ECDSA P-256 SSH host key and OpenSSH-compatible SHA-256 fingerprint.
The certificate and credentials remain stable across ordinary reboot and OTA-slot changes until explicitly rotated. The `.local` name is included for future hostname discovery, but this phase does not yet advertise mDNS; use the AP address or the station address reported by `wifi status`.
The HTTPS certificate and shared administrative credentials remain stable across ordinary reboot and OTA-slot changes until explicitly rotated. The `.local` name is included for future hostname discovery, but this phase does not yet advertise mDNS; use the AP address or the station address reported by `wifi status`.
The physical UART0 administration console provides:
@@ -316,7 +317,7 @@ web certificate rotate --force
web reset --force
```
`web` and `web help` print the same usage summary. `web credentials show` is the intended first-boot credential-retrieval path. Rotation and reset operations write NVS immediately rather than creating RAM-only secrets. Destructive operations require a literal `--force`; `web reset --force` is also the recovery path for an incompatible or damaged `web_sec/material` blob and starts HTTPS with the recovered material. Invalid stored material is never overwritten automatically, and HTTPS failure never disables UART0, native USB, the serial core, or Wi-Fi recovery.
`web` and `web help` print the same usage summary. `web credentials show` is the intended first-boot credential-retrieval path for both HTTPS and SSH. Rotation and reset operations write NVS immediately rather than creating RAM-only secrets. Credential rotation revokes active WebSocket and SSH sessions. Destructive operations require a literal `--force`; `web reset --force` is also the recovery path for an incompatible or damaged `web_sec/material` blob and starts HTTPS with the recovered material. Invalid stored material is never overwritten automatically, and HTTPS failure never disables UART0, native USB, the serial core, or Wi-Fi recovery.
The self-signed certificate is not trusted by browsers or host tools by default. After retrieving the password, validate from a host connected to the fallback AP with:
@@ -342,7 +343,7 @@ Replace the example SNI name with the DNS SAN printed by `web certificate info`.
#### WebSocket authentication and broker behavior
Browser JavaScript cannot reliably attach a Basic `Authorization` header to a WebSocket constructor. The authenticated page therefore obtains a 192-bit random, RAM-only ticket with `POST /api/ws-ticket`, then presents that ticket once in the WSS URL. The server stores only its SHA-256 digest, accepts it once within 30 seconds, binds it to the current credential generation, and creates no broker client until validation succeeds. Credential rotation invalidates outstanding tickets and closes active web-terminal sessions.
Browser JavaScript cannot reliably attach a Basic `Authorization` header to a WebSocket constructor. The authenticated page therefore obtains a 192-bit random, RAM-only ticket with `POST /api/ws-ticket`, then presents that ticket once in the WSS URL. The server stores only its SHA-256 digest, accepts it once within 30 seconds, binds it to the current credential generation, and creates no broker client until validation succeeds. Credential rotation invalidates outstanding tickets and closes active web-terminal and SSH sessions.
ESP-IDF 5.5 sends the RFC 6455 `101 Switching Protocols` response before invoking the application WebSocket handler. Consequently, an invalid ticket receives the protocol upgrade and is then closed immediately rather than receiving an HTTP `401`; it never gains a broker session, serial output, or writer access. Strict rejection before `101` would require a framework-level pre-handshake authorization hook that ESP-IDF 5.5 does not provide.
@@ -372,7 +373,74 @@ Vendored browser sources, versions, hashes/provenance, deterministic gzip artifa
HTTPS permits up to six simultaneous client sockets: two bounded persistent WebSocket terminals plus parallel browser asset, ticket, and status requests. ESP-IDF documents approximately 40 KiB per active TLS socket, so this is a concurrency ceiling rather than preallocated per-socket memory. WebSocket serial sessions themselves remain fixed at two. To preserve internal DRAM during concurrent handshakes, the firmware configures ESP-IDF's mbedTLS allocator for the board's external PSRAM. Basic authentication is acceptable here only because plaintext HTTP is disabled. It is an initial administration mechanism, not the final authorization design.
**Current security limitation:** the web password and ECDSA private key are stored as plaintext in the application-owned `web_sec/material` NVS blob, just as Wi-Fi credentials are currently plaintext in `wifi_app/config`. The reserved `nvs_key` partition does not activate NVS encryption. ESP-IDF 5.5 also keeps active mbedTLS allocations, including TLS key/session material, in external PSRAM under this memory configuration and does not guarantee zeroization when those allocations are freed. PSRAM encryption is not enabled in the current firmware. Do not treat the current firmware as resistant to physical flash or RAM extraction; NVS encryption, flash/PSRAM encryption, secure boot, protected OTA, secret-aware core-dump handling, and framework-level key zeroization belong to the later hardening phase.
**Current security limitation:** the shared admin password and HTTPS ECDSA private key are stored as plaintext in the application-owned `web_sec/material` NVS blob, the SSH ECDSA private key is plaintext in `ssh_sec/material`, and Wi-Fi credentials are plaintext in `wifi_app/config`. The reserved `nvs_key` partition does not activate NVS encryption. ESP-IDF 5.5 keeps active mbedTLS allocations in external PSRAM under this memory configuration; wolfSSL/wolfSSH allocations also prefer PSRAM with internal-memory fallback. Neither framework guarantees complete zeroization of released session allocations, and PSRAM encryption is not enabled. Do not treat the current firmware as resistant to physical flash or RAM extraction; NVS encryption, flash/PSRAM encryption, secure boot, protected OTA, secret-aware core-dump handling, and framework-level key zeroization belong to the later hardening phase.
### Phase 6 authenticated SSH serial transport
A wolfSSH server listens on TCP port 22 across whichever Wi-Fi interfaces are active. It uses the same `admin` username and 24-character password as HTTPS, but has a separate persistent ECDSA P-256 host key in `ssh_sec/material`. The SSH identity is deliberately independent of the HTTPS certificate key so certificate rotation does not unexpectedly invalidate SSH `known_hosts` entries.
The UART0 administration console provides:
```text
ssh
ssh help
ssh status
ssh start|stop
ssh sessions
ssh disconnect <session-id>
ssh counters|clear-counters
ssh host-key info
ssh host-key rotate --force
ssh reset --force
```
`ssh` and `ssh help` print the same usage summary. `ssh host-key info` prints the OpenSSH-style `SHA256:...` fingerprint over the standard `ecdsa-sha2-nistp256` public-key wire blob. `ssh host-key rotate --force` replaces a valid key; `ssh reset --force` explicitly replaces missing, incompatible, or corrupt SSH material. Both operations persist immediately, stop active SSH sessions before changing identity, and restart the service when appropriate. After deliberate rotation, remove the old host entry on the client with `ssh-keygen -R <address>` and verify the new fingerprint before reconnecting.
Only password authentication and an interactive shell/PTY channel are accepted. Public-key user authentication, arbitrary `exec`, subsystem requests, SFTP, SCP, agent forwarding, and TCP forwarding are not provided. Authentication is bounded to three password attempts per connection and a 15-second handshake deadline. At most two total SSH sessions or in-progress handshakes are admitted at once. Two deliberately stalled clients can occupy both fixed slots until their deadlines, so these bounds limit resource consumption but do not prevent availability attacks. This is useful embedded admission control, not a substitute for network filtering, strong unique credentials, monitoring, or mature brute-force protection.
After authentication and shell negotiation, the channel becomes a binary-transparent serial byte stream. The firmware does not inject prompts, status messages, writer-control escapes, line handling, or terminal emulation into channel data. SSH framing and encryption terminate at the ESP32; bytes between the SSH channel and session broker are retained across partial nonblocking transfers.
Each admitted shell becomes a `SESSION_BROKER_CLIENT_SSH`. The first session opportunistically requests the writer lease; a later session remains an observer if USB, WebSocket, a test client, or another SSH session already owns it. Observers receive UART output but their input is discarded. SSH deliberately has no in-band writer-control sequence because any reserved byte sequence would break binary transparency. Inspect ownership with `ssh sessions` or `broker clients`, and use the physical UART0 `broker force-writer <client-id|none>` recovery command when administrative reassignment is required.
Before connecting for the first time:
1. Run `web credentials show` on UART0 and keep the password private.
2. Run `ssh host-key info` and record the exact `SHA256:...` fingerprint.
3. Obtain the AP or station address from `wifi status`.
Then connect with OpenSSH, forcing the implemented authentication method if the host has unrelated keys configured:
```sh
ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no admin@192.168.4.1
```
Compare OpenSSH's first-connection ECDSA fingerprint with `ssh host-key info` before accepting it. Once connected, all terminal bytes go directly to the physical RS-232 session. OpenSSH may print a one-time pre-authentication device banner; this is SSH protocol metadata and is not inserted into the serial stream.
The wolfSSH runtime has one owner task and two fixed session slots. A single owner serializes the managed component's `SINGLE_THREADED` build, avoids one large crypto stack per client, and services both nonblocking sessions fairly. The task has a 20 KiB internal stack; wolfSSL/wolfSSH dynamic allocations prefer external PSRAM and fall back to internal 8-bit memory. WolfCrypt's seed callback is installed after `wolfSSH_Init()` and routes all SSH RNG seeding through the project's serialized pre-radio CTR_DRBG instead of the registry component's default post-radio `esp_random()` path. HTTPS remains on ESP-IDF mbedTLS and is not switched to wolfSSL.
The official Espressif registry baseline is pinned in `src/idf_component.yml` and `dependencies.lock`:
```text
wolfssl/wolfssl 5.8.2~1
wolfssl/wolfssh 1.4.20
```
The build enables `CONFIG_ESP_ENABLE_WOLFSSH`, keeps `CONFIG_ESP_TLS_USING_MBEDTLS`, and raises `CONFIG_LWIP_MAX_SOCKETS` to 16 for HTTPS/WebSocket and SSH concurrency. Adding the linked SSH implementation increased the measured release image from approximately 29.3% to 33.9% of the 4 MiB OTA slot and static RAM from 17.5% to 18.5% of 320 KiB. Runtime task stacks, Wi-Fi/TLS state, and PSRAM allocations are not fully represented by the static RAM percentage.
#### SSH validation
1. Boot and confirm `ssh status` reports `running=yes`, TCP port 22, password authentication, and no sessions.
2. Compare the first OpenSSH host-key prompt exactly with `ssh host-key info`; disconnect if it differs.
3. Verify a wrong password fails and the shared password from `web credentials show` succeeds.
4. Exercise text, NUL-containing/binary payloads, UTF-8, ANSI color, cursor-control sequences, and a full-screen terminal application through an RS-232 loopback or peer.
5. Confirm `ssh sessions`, `broker clients`, `ssh counters`, and serial counters reflect traffic and cleanup.
6. Connect a second SSH client. It should authenticate as an observer, receive identical UART output, and be unable to inject input while the first owns the writer lease.
7. Test ownership against native USB and WebSocket clients, using UART0 `broker force-writer` for deterministic reassignment.
8. Run `ssh disconnect <session-id>` and verify the socket and broker client disappear without affecting another observer.
9. Leave an unauthenticated TCP/SSH client stalled and verify its slot closes after 15 seconds while an established session continues.
10. Run `web credentials rotate --force`; active SSH clients should disconnect and the old password should fail.
11. Run `ssh host-key rotate --force`, remove the old client entry, and verify the newly printed fingerprint persists across reboot.
12. Exercise `ssh stop`, `ssh start`, Wi-Fi reconnect, and reboot while confirming UART0/native USB recovery remains available and no stale SSH broker clients survive.
### Phase 0 diagnostics
@@ -408,4 +476,4 @@ This hardware profile uses the onboard RGB LED on GPIO48. Official ESP32-S3-DevK
## License
This project is licensed under the [GNU General Public License version 3 only](LICENSE) (`GPL-3.0-only`). This is compatible with using the GPLv3 releases of wolfSSL and wolfSSH later. Third-party components remain subject to their respective licenses.
This project is licensed under the [GNU General Public License version 3 only](LICENSE) (`GPL-3.0-only`). Phase 6 pins the official Espressif registry components `wolfssl/wolfssl` 5.8.2~1 and `wolfssl/wolfssh` 1.4.20. Their GPL releases are compatible with this project; third-party components remain subject to their respective licenses. The wolfSSH 1.4.20 managed archive contains stale GPLv2-or-later wording in `LICENSE.txt` while upstream wolfSSH source headers identify GPLv3-or-later. Both are GPLv3-compatible, but distributors should preserve the upstream notices and re-check the registry package's license metadata. These registry versions are an integration baseline rather than a permanent security pin and must be reviewed against current upstream wolfSSL/wolfSSH releases during production hardening.