ESP32_Serial_Swiss_Army_Knife/docs/user_administration_tests.md

User administration and authentication tests

This document retains phase-specific regression procedures. The Phase 8A and 8B sections describe the behavior of those historical implementation baselines; they are not the current end-to-end acceptance behavior. In current Phase 8C firmware, HTTPS and SSH authenticate through the user database, role-user SSH sessions receive the broker-backed serial stream, and role-admin SSH sessions receive the administration shell. Use the Phase 8C section for current routing and shared-console validation. Never include generated or entered passwords in test logs.

Historical Phase 8A baseline — role-based database and UART0 administration

Keep UART0 physically attached throughout these tests. At the Phase 8A baseline, HTTPS and SSH still used the legacy shared credential while the new database and public keys were prepared for the Phase 8B cutover.

1. First-boot migration and persistence

Upgrade a device that already has working HTTPS/SSH credentials and no user_db blob. After boot:

1. Run user status and user show admin.
2. Confirm exactly one admin-named account exists with role user, bootstrapped=no, no displayed password/salt/verifier, and no public keys.
3. Confirm the pre-upgrade credential still authenticates to HTTPS and SSH and that serial terminal behavior is unchanged.
4. Reboot and confirm user status reports the same account ID and authentication generation.
5. Confirm routine boot logs, status, web status, and ssh status contain no user password, verifier, salt, or raw public-key blob.

A transient failure reading the legacy web credential must not create or commit an empty user database. If a valid user_db blob already exists, it must still load without the legacy credential being readable.

2. Administrator bootstrap

Run user bootstrap, verify that typed password characters are not echoed, enter a 12–64 character printable password twice, and confirm user show admin now reports role admin and a higher authentication generation. Reboot and confirm bootstrapped=yes and one administrator persist. On a separate resettable test image, exercise user bootstrap --generate; record the generated password securely for the test, verify it is displayed once, then clear the terminal scrollback after use.

During Phase 8A, verify the newly bootstrapped password does not replace the legacy HTTPS/SSH login yet and the existing network credential continues to work. This separation is intentional until Phase 8B. A second user bootstrap must be rejected without changing the account.

3. Account CRUD, capacity, and invariants

Use generated test passwords where practical:

user add operator user --generate
user add maint admin --generate
user list
user show operator
user role operator admin --force
user role operator user --force
user password operator --generate

Confirm unique valid usernames are accepted; uppercase, empty, oversized, or punctuation-invalid usernames are rejected. Confirm duplicate users are rejected. Fill all eight slots and verify a ninth account is rejected without altering existing records; delete a non-final account and confirm its slot can be reused with a new account ID.

With two administrators, demote or delete one and confirm the operation persists. Then attempt both deletion and demotion of the final administrator and confirm both are rejected. Before bootstrap on a separate migration run, verify user delete admin --force is rejected so legacy synchronization remains recoverable. Ctrl-C during either hidden password prompt must cancel without changing the account or leaving stale input in the next console command.

4. SSH public-key import and fingerprints

Generate disposable Ed25519 and ECDSA P-256 OpenSSH keys on a trusted host. For each supported type, run user key add <username>, paste the single .pub line, and confirm user show <username> reports the type, stable slot index, and expected SHA256: fingerprint without displaying the Base64 blob or comment.

Verify all of the following fail without changing the account generation or key list:

• Unsupported RSA, certificate, or unknown key types.
• Truncated/invalid Base64, mismatched outer and embedded types, trailing blob data, malformed Ed25519 lengths, and an invalid/off-curve ECDSA point.
• Adding the same key again to the same account.
• Adding a fourth key to an account that already has three.
• Deleting an empty/out-of-range slot.

Assign the same public key to a second account and confirm it is accepted, receives an independent slot, and authenticates as the username selected by the SSH client.

Then exercise user key delete <username> <index> --force and user key clear <username> --force, reboot, and confirm the exact remaining fingerprints persist. Phase 8A stores these keys but does not yet accept SSH key login.

5. Legacy credential rotation boundary and reboot reconciliation

Before administrator bootstrap on a resettable test image:

1. Run web credentials rotate --force.
2. Confirm the console reports that the pre-bootstrap migrated verifier was synchronized.
3. Verify the new legacy credential works for HTTPS and SSH, reboot, and confirm user show admin retains the synchronized authentication generation.
4. Where NVS fault injection or controlled power interruption is available, interrupt after the web credential commits but before the user blob commits. On the next boot, confirm reconciliation updates the migrated verifier rather than committing an empty database or losing network recovery.

After bootstrap, rotate the legacy credential again. Confirm the console explicitly says it remains separate from user passwords, existing network sessions are revoked, the new legacy credential works, and the bootstrapped administrator generation/password are not silently replaced. Apply the same checks to web reset --force, additionally confirming HTTPS certificate recovery/restart behavior remains intact.

6. Explicit database recovery and storage failures

Using a disposable device or test NVS image, inject a wrong-size/version or malformed user_db blob and reboot. Confirm normal database initialization fails without erasing shared NVS, but the user command is still registered. user status must report the database unavailable. Run:

user recover --force
user status
user bootstrap --generate

Confirm recovery explicitly replaces only user_db, imports the current legacy credential as role user, and leaves serial, Wi-Fi, HTTPS identity, SSH host key, and legacy network login intact. Confirm recovery is rejected while a valid database is initialized. Simulate NVS write/commit exhaustion or failure during create, password, role, key, and recovery operations; after reboot, each operation must expose either the complete old blob or complete new blob, never a partially live mutation.

Because NVS is not yet encrypted, this test validates logical verifier storage rather than resistance to physical extraction. PBKDF2 verifiers remain susceptible to offline guessing, and append-oriented NVS may retain historical plaintext legacy credentials until a full partition/flash erase.

7. Phase 8A regression

With the maximum supported mix of USB CDC, WebSocket, and SSH serial clients active, repeat user list/show and several UART0 mutations. Confirm password prompts remain physical-UART-only, no user operation acquires a broker client or writer lease, existing Phase 8A network authentication and revocation behavior remains functional, and serial/broker counters show no unexplained loss. Record memory, web counters, and ssh counters before and after the run.

Historical Phase 8B baseline — role-aware HTTPS and SSH authentication

These procedures preserve the Phase 8B regression baseline. Complete the Phase 8A bootstrap first and retain two disposable accounts, one user and one admin. At that baseline both roles received the serial/status interface; current Phase 8C firmware instead routes SSH administrators to the administration shell.

1. HTTPS passwords and fail-closed behavior

For both roles, authenticate to /, /api/status, and the browser terminal with the account password. Confirm the full 16-character username and a 64-character password work, while absent, malformed, oversized, wrong-user, and wrong-password Basic headers fail without exposing which component was wrong. After bootstrap, confirm the value from web credentials show no longer authenticates unless it independently equals a current user password.

On a disposable malformed-database image, confirm HTTPS returns authentication-service unavailable and never falls back to the legacy credential. Recover through physical UART0 only. Verify web status reports user-database authentication and active WebSocket sessions show account, user role, and password method without secrets.

2. SSH password and public-key login

For both user and admin, verify password SSH login reaches the same broker-backed serial stream as before. Confirm ssh sessions shows the account, user role, password method, and writer/observer role. An administrator must not receive UART0 commands yet.

Import one Ed25519 key and one ECDSA P-256 key through user key add, then verify each matching private key authenticates without a password and ssh sessions reports public-key. Confirm an unregistered key, wrong username, unsupported RSA key, certificate key, and three failed completed attempts are rejected. Normal client probe-then-sign behavior must consume one completed attempt, not two; unsigned probes alone must never create an authenticated session or broker client.

3. WebSocket ticket binding

Authenticate separately as two users and request tickets. Confirm each ticket is accepted exactly once, expires after 30 seconds, and cannot be reused or altered. Change one account after ticket issuance but before WebSocket connection; its ticket must fail closed without creating a broker client, while the other account's ticket still works. Repeat with delete followed by recreation of the same username and confirm the old ticket remains invalid.

4. Targeted session revocation

Connect simultaneous WebSocket and SSH sessions for two different users and place one affected session in writer state. One mutation at a time, exercise password change, role change, SSH-key add/delete/clear, and account deletion. Required results:

1. Outstanding tickets and active sessions for the changed username are revoked promptly and any writer lease is released.
2. The stale session cannot inject additional serial bytes or reacquire writer ownership.
3. The unrelated user's WebSocket/SSH sessions remain connected and retain their expected broker roles.
4. New authentication succeeds only with the updated password/key/role state.
5. Deleting and recreating the same username does not revive old tickets or sessions.

Run web status, ssh sessions, web counters, ssh counters, and broker clients around each mutation. Also rotate web credentials rotate --force after administrator bootstrap and confirm it changes only recovery material: role-based HTTPS/SSH sessions remain connected and user passwords are unchanged. web reset --force restarts HTTPS because the TLS identity changes, but must not revoke unrelated SSH sessions.

5. Concurrency regression

With USB CDC, two role-based network users, one WebSocket terminal, one SSH terminal, and UART1 traffic active, alternate writer ownership and mutate one account. Confirm binary transparency, observer isolation, bounded authentication/handshake behavior, UART0 responsiveness, and no unexpected disconnect of the unaffected user. Record memory, broker, web, SSH, and serial counters before and after. Repeat after reboot to verify passwords, roles, keys, and authentication methods persist.

Phase 8C SSH administrative shell

Use one disposable admin and one disposable user. Keep UART0 attached throughout. The SSH server still accepts only shell sessions: exec, subsystem/SFTP/SCP, forwarding, and unauthenticated connections must remain rejected.

1. Route separation and normal shells

1. Connect as the normal user and confirm the existing broker-backed UART1 serial stream, broker client, and writer/observer behavior are unchanged.
2. Connect as the administrator and confirm the admin@serial-tool> prompt appears. Run help, status, memory, serial status, wifi status, web status, broker status, and broker clients. Compare representative output with UART0 and confirm both routes execute the same registered command implementations.
3. From UART0 run ssh sessions and broker clients. The user session must show route=broker; the admin session must show route=admin-console, broker=0, broker-role=n/a, and no writer lease. The admin session must not start UART1 or alter broker client/writer counts merely by connecting.

2. Bounded command processing

Exercise printable input, backspace, Ctrl-C, CR/LF, an empty line, and a line longer than the documented limit. Confirm the command line is bounded, overflow is discarded through a clear diagnostic, and a new prompt remains usable. Run help, user list, and broker clients in a normal ANSI terminal and confirm every line starts in column zero: canonical LF output must be normalized to CRLF without doubling handlers that already emit CRLF.

Run at least five distinct commands, then use Up/Down to navigate the four-entry per-session history, return to a saved draft with Down, and confirm older entries are bounded out. Verify history does not survive reconnect and is not shared with a second administrator. Exercise Tab on root and nested prefixes such as us, user l, wifi ap sh, wifi next, and ssh host-key i; confirm the same hints are offered by UART0 and SSH. For an ambiguous prefix, confirm SSH prints the matching candidates and redraws the unchanged input line, matching UART0's candidate set; unique/common prefixes must redraw cleanly without inserting escape-sequence bytes into the command. Type a command, use Left/Right to insert and delete characters in its middle, then use Home/End and the terminal's Pos1/Ende keys; verify cursor placement, Backspace, and Delete change exactly the intended byte before submission.

Run an unsupported command and confirm it is rejected without affecting UART0 or the serial broker. Run the full root help output to exercise output-ring draining. With the SSH client temporarily unable to read output, confirm the worker applies input backpressure rather than accepting an unbounded command/output backlog; inspect ssh counters for admin-console admission and input-backpressure values.

3. Remote account administration

Run user list, user show <name>, user add <name> user --generate, user password <name> --generate, user role <name> admin --force, and the key delete/clear operations from the administrative shell. Confirm generated passwords appear once only on that authenticated channel, affected account sessions are revoked, and unrelated sessions remain connected.

Import both supported key types through the remote form:

user key add <username> ssh-ed25519 <base64-blob>
user key add <username> ecdsa-sha2-nistp256 <base64-blob>

Confirm the full ECDSA P-256 command is accepted, fingerprints appear in user show, a duplicate on the same account is rejected, and the same key can be imported for a second account. Verify subsequent private-key SSH login uses the selected SSH username.

4. Interactive administration, lifecycle actions, and revocation

Confirm user bootstrap and user recover --force remain unavailable from SSH and continue to work through physical UART0. Also confirm that an administrator cannot run user password <own-username> --generate over its own SSH session; entered replacement passwords and generated passwords for other accounts remain available. From admin SSH, exercise manually entered user passwords and public keys, Wi-Fi station/AP secret entry, AP secret display, legacy web recovery credential display/rotation, HTTPS certificate rotation/reset, and both ping and wifi ping. Hidden characters must not echo or enter command history; visible key input must support Backspace and Ctrl-C; ping lines must remain ordered and correctly attributed to the invoking SSH session.

Exercise ssh disconnect for another session and the current session. For the other-session case, confirm the acknowledgement drains, the target closes, and the source shell returns. Separately test reboot, ssh stop, ssh host-key rotate --force, and ssh reset --force: each must acknowledge scheduling, stop accepting another command, drain output, and then close/reboot as appropriate. Reconnect after key replacement and verify the new fingerprint. Simulate an unread SSH output window and confirm the destructive action cancels after its bounded drain timeout rather than remaining pending forever.

While an administrative command is queued or running, use UART0 to change that admin's role/password/key or delete it. Confirm no second remote administrative command runs after the mutation, the SSH session is revoked promptly, and queued output is not delivered to a reused SSH slot. Repeat with a different account mutation and confirm the administrator remains connected.

Finally, issue commands concurrently from UART0 and admin SSH, including user list, long help output, and one UART0 interactive password or key prompt while an SSH command waits. Confirm the single dispatcher serializes all esp_console_run() calls, UART0 retains its line editing/history/completion, prompt input is consumed only from UART0, outputs are not mixed between transports, and there is no stack overflow, corrupted argument parsing, database damage, or broker disruption.