sasa/wireguard-admin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add integration tests based on VM

Browse Source
This commit is contained in:
sasa
2026-10-26 22:34:00 +02:00
parent f4a6540df9
commit 7d0f006601
22 changed files with 3826 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/VM_ACCEPTANCE.md
+392
View File
@@ -0,0 +1,392 @@
# Disposable systemd VM acceptance contract
Status: Milestone 9 is paused at the user's request. The exact disposable VM
was powered off after a guarded first start and an installer progress screen;
read-only `list runningvms` was empty afterward. Debian installation completion,
guest provisioning, reset, cleanup, and systemd acceptance are unverified.
This contract supplements [Testing and Acceptance](TESTING.md) and the M9
criteria in [PLANS.md](../PLANS.md). Product behavior is governed by
[SPEC.md](SPEC.md), [Runtime observation and verification](RUNTIME.md), and
[ADRs 0001–0003](adr/).
## Selected target
The selected target is exactly one new VirtualBox VM:
| Field | Required value |
| --- | --- |
| Provider | Oracle VirtualBox, local `VBoxManage` 7.2.8_Gentoo |
| VM name | `wga-m9-debian137-20261025` |
| Guest base | Official Debian 13.7.0 (Trixie) amd64 netinst ISO |
| ISO path | `/media/data/wm/Virtualbox/wga-m9-lab/debian-13.7.0-amd64-netinst.iso` |
| ISO SHA-256 | `a7ef94ac2fb9a7fec454552abd629b7cc9d5155c886165a45649f5ce6167e355` |
| VirtualBox OS type | `Debian13_64` |
| Lab root | `/media/data/wm/Virtualbox/wga-m9-lab` (owner `sasa`, mode `0700`) |
| Networking | One NAT NIC only, no forwarding; all other NICs disabled |
| VM UUID | `b14c850b-a21e-4a56-acca-785590f0c4ac`, pinned in the mode-`0600` identity manifest |
| Guest marker | `/etc/wga-m9-lab-marker`, exact content `wga-m9-debian137-20261025` after provisioning |
The ISO is present, has the stated SHA-256, and was obtained from Debian's
[official amd64 netinst directory](https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/).
VBoxManage command forms were checked against the local 7.2.8 help and Oracle's
[VirtualBox 7.2 VBoxManage manual](https://docs.oracle.com/en/virtualization/virtualbox/7.2/user/vboxmanage.html).
Read-only `VBoxManage unattended detect --iso=... --machine-readable` reported
`OSTypeId=Debian13_64`, `OSVersion=13.7.0 Trixie`, and
`IsInstallSupported=on`, but then exited 1 with `NS_ERROR_NOT_IMPLEMENTED`.
The later bounded `unattended install --dry-run --start-vm=none` succeeded for
this exact VM with private synthetic password files, a fully qualified `.invalid`
hostname, and the local Guest Additions ISO. The dry run made no VM changes;
actual unattended preparation and guest boot remain unverified. VirtualBox's
dry-run output contains plaintext password fields, so the wrapper suppressed
them and displayed only redacted output. Any actual unattended step needs a
reviewed exact-target procedure and post-prepare media inspection before start.
The approved read-only inventory outside the sandbox found `/dev/vboxdrv`
available and `VBoxManage` 7.2.8 working. It initially listed Windows 7 and
Windows 11 VMs; neither is a test target or was modified. The create-only run
registered only the new Debian target and stopped before configuration. The
default sandbox does not expose the host driver/device, so host inventory and
later VBoxManage operations must use the approved outside-sandbox path. The
guard matches the exact new name/UUID and does not inspect existing VMs'
per-VM files.
## Guarded host operations
[`tests/vm/guard.py`](../tests/vm/guard.py) is a narrow standard-library Python
guard, not a general VirtualBox wrapper. Its default `check`, `start`, and
`authorize-guest` actions are outside Cargo's default test targets. The guard:
- Requires the exact lab root to be a real directory owned by the current user
with mode `0700`, and `target-identity.json` to be a regular current-user file
with mode `0600`.
- Requires the manifest to pin the exact VM name, current owner UID, one valid
UUID, and host marker `<VM name>:<UUID>`.
- Uses `VBoxManage list vms` to require that exact name occur once with the pinned
UUID, then `showvminfo <UUID> --machinereadable` to compare UUID/name/marker.
- Requires the VM configuration and its one VDI to reside below the lab root,
the VDI to be named exactly `wga-m9-debian137-20261025.vdi`, and any mounted
installer ISO to be the pinned Debian ISO in that root.
- Requires a complete inventory of all eight NIC slots, with only NIC 1 in NAT
mode. NAT forwarding and all VirtualBox shared folders are refused.
- Performs every identity, path, and isolation check before `startvm`. A missing,
duplicate, malformed, or mismatched identity fails closed. The only start
target is the pinned UUID, in the GUI frontend for console installation.
- Before guest authorization, repeats all host checks, requires a current-user
password file with mode `0600`, and reads only the guest marker via
`VBoxManage guestcontrol ... run /usr/bin/cat` with `--wait-stdout` and a
15-second guest process timeout. Host stdout/stderr capture is also capped at
1 MiB while the process runs. The password is supplied using `--passwordfile`;
it is never placed in argv. Marker output is compared exactly and is not
echoed. This read is unprivileged inside the guest.
The host marker is a pre-install bootstrap identity: it binds the created
VirtualBox object and protected manifest before the first start. It is not proof
of guest identity. The guest marker is installed only after Debian setup and is
checked separately after boot. Do not perform root provisioning, run application
acceptance, or access the guest system bus until `authorize-guest` succeeds for
the exact UUID. The guest marker check is deliberately read-only; it does not
grant the caller a general guest command runner.
The exact current focused checks are:
```sh
python3 -m unittest discover -s tests/vm -p 'test_*.py' -v
python3 -m py_compile tests/vm/guard.py tests/vm/test_guard.py
git diff --check
```
They use a temporary synthetic VM inventory and fake VBoxManage responses. They
do not call VBoxManage, start a guest, contact a host service, or read any
registered VM configuration. The behavioral cases cover normal exact identity,
missing/duplicate/wrong UUID, wrong name, manifest/root permissions, storage
escape, NIC mode and incomplete inventory, NAT forwarding, shared folders,
host/guest marker mismatch, no `startvm` on refusal, and exact UUID start scope.
The guard supports these later commands once the selected VM has been created,
its manifest and marker have been recorded, and the coordinator has reviewed the
mutation step:
```sh
python3 tests/vm/guard.py check
python3 tests/vm/guard.py start
python3 tests/vm/guard.py authorize-guest --password-file /path/to/private/guest-password
```
The placeholder password path above must name a private file owned by the
current user with mode `0600`. Do not put a password value in the command line,
environment, terminal history, or output. The `start` command is an implemented
guarded operation but has not been run. Running it requires the approved
outside-sandbox path because the default sandbox does not expose the host
VirtualBox device.
## Create-only procedure (ran once; do not rerun)
[`tests/vm/create.py`](../tests/vm/create.py) exposes only the fixed create
operation; it accepts no target, path, or action override. After confirming the
lab root owner/mode, exact ISO digest, absence of both the identity manifest and
the exact target VM path (using `lstat`, so dangling symlinks also count as
occupied), and a well-formed VirtualBox inventory without the selected name, it
issues bounded VBoxManage calls to:
1. Register `wga-m9-debian137-20261025` as `Debian13_64` below the lab root.
2. Configure 1024 MiB RAM, one CPU, NIC 1 in NAT mode, NICs 2–8 disabled, audio
disabled, USB disabled, and DVD then disk boot order.
3. Create one dynamically allocated standard 8192 MiB VDI, attach it on SATA
port 0, and attach the pinned Debian ISO on SATA port 1.
4. Write the actual VM UUID and `<name>:<UUID>` host marker in
`target-identity.json` with mode `0600`, set the same marker as the VirtualBox
description, then run the existing read-only host guard.
Run this exact command only after coordinator review of the current host state
and the scoped mutations:
```sh
python3 tests/vm/create.py
```
The command must run through the approved outside-sandbox path because the
default sandbox cannot access the VirtualBox host driver. Its effects are VM
registration, storage/network/device/boot configuration, ISO attachment, and
the protected host identity manifest. It does not start the VM, install Debian,
use guest control, create snapshots, or delete/clean up resources. VBoxManage
calls have 30-second timeouts and 1 MiB per-stream output limits. Errors omit
VBoxManage output; after registration they report the exact target name and
known UUID. A failed procedure retains any partial target and its identity for
inspection; it performs no automatic rollback or broad cleanup. The final guard
must pass before the script reports success. The first run registered the UUID
above, then stopped when strict parsing rejected VirtualBox's indented
`rec_screen0` recording marker. The parser was narrowly repaired and
independently re-reviewed. The target path is now occupied, so `create.py` must
not be rerun; use the reviewed resume procedure below.
## Resume the registered partial target (ran once; do not rerun)
The earlier create attempt registered the selected target before stopping at
the initial machine-readable parse. The exact retained target is
`wga-m9-debian137-20261025` with UUID
`b14c850b-a21e-4a56-acca-785590f0c4ac`; it is powered off and has only its
`.vbox` configuration file. Its observed NIC inventory was NIC 1 in NAT with
NICs 2–8 disabled. It had no identity manifest or VDI. That prior state was
the sole input accepted by [`tests/vm/resume.py`](../tests/vm/resume.py).
The resume-only procedure is fixed to that UUID and lab root. Before its first
mutation it checks the private root, exact pinned ISO path and SHA-256, absent
manifest and target VDI, unique registered name/UUID identity, exact `.vbox`
path, powered-off state, empty host description, default memory/CPU settings,
the observed NIC inventory (NIC 1 in NAT and NICs 2–8 disabled), no attached
media or storage controller, no shared folders, and no NAT forwarding. It
refuses an unexpected adapter or preexisting state.
After preflight it configures memory/CPU, NAT-only networking, disabled audio
and USB, boot order, one standard 8192 MiB VDI, and the pinned ISO; then it
writes the protected host manifest, sets the host marker, and runs the existing
read-only guard. It does not create, start, unregister, delete, or clean up a
VM. A failure after mutation retains the exact target and reports its pinned
UUID for inspection; no rollback is attempted.
The focused command is:
```sh
python3 -m unittest discover -s tests/vm -p 'test_resume.py' -v
```
It uses only a temporary synthetic lab and fake VBoxManage responses. It does
not call VBoxManage or establish host/VM evidence. The implementation command
was `python3 tests/vm/resume.py`; it ran once via the approved outside-sandbox
path. It configured the exact target and wrote the host marker/manifest. Its
final guard call initially refused observed VirtualBox storage metadata;
focused repairs and independent re-review closed those parser and guard
findings. A subsequent read-only `python3 tests/vm/guard.py check` passed
against the actual powered-off target. Do not rerun the resume command on this
occupied state.
Do not infer `unregistervm --delete`, snapshot restore, guest-control, or broad
cleanup commands from the CLI help. No provisioning command is documented
because Debian installation, Guest Additions/management channel, the unprivileged
`wga` account, and guest marker placement have not been validated. No reset or
deletion command is documented because a baseline snapshot and exact owned
resource IDs do not yet exist. Existing registered VMs must remain untouched.
## Unattended preparation (ran once; do not rerun)
[`tests/vm/install_prepare.py`](../tests/vm/install_prepare.py) is pinned to the
VM UUID above and accepts no target, path, credential, or action overrides. It
first checks the current user's `/tmp/wga-m9-private` directory is a real mode
`0700` directory and its nonempty `wga-password` and `root-password` files are
current-user-owned regular mode-`0600` files. These are synthetic install
credentials only; password contents never enter argv or output. It verifies the
pinned Debian ISO and `/usr/share/virtualbox/VBoxGuestAdditions.iso` SHA-256
digests, then calls the existing exact-target read-only guard and additionally
requires the VM to be powered off, with the expected VDI at `SATA-0-0`, the
pinned installer ISO at `SATA-1-0`, and every other reported SATA attachment
slot set to `none` or `emptydrive`. This accepts VirtualBox 7.2.8's real
`showvminfo` inventory (`SATA-2-0` through `SATA-29-0` are `none`) while
rejecting any additional attached media. It refuses before mutation if any
check fails.
The script runs a bounded `VBoxManage unattended install <pinned UUID>` dry run
first with `--iso=<pinned Debian ISO>` and
`--admin-password-file=/tmp/wga-m9-private/root-password`, using the synthetic
`wga` account and password file, `.invalid` FQDN, `en_US`, `UTC`, US country,
minimal package adjustment, and the pinned Guest Additions ISO. The root
credential option is `--admin-password-file`; `--root-password-file` is not a
valid option for the local VBoxManage 7.2.8. Only a successful dry run permits
the same bounded command without `--dry-run`. Both commands request
`--start-vm=none`; the script has no start, cleanup, or retry operation.
VBoxManage output is captured privately because its dry-run output contains
plaintext password fields; errors and status output omit captured text. The
command has a 120-second deadline and 1 MiB per-stream capture limit.
After independent review, the command ran once through the approved
outside-sandbox VBoxManage path:
```sh
python3 tests/vm/install_prepare.py
```
The command exited 0 and left the VM powered off. Read-only `showvminfo`
inspection found the exact VDI at SATA 0 and a VirtualBox-generated auxiliary
`.viso` at SATA 1, both in the private target folder. The generated installer
files are mode `0600`; the target folder is mode `0700`. The normal host guard
refuses the `.viso`, as designed, so the first start requires a separate guard
for this exact installer state and independent review. Do not rerun preparation
on the occupied state. Guest boot, Guest Additions setup, and guest provisioning
are not established by this result or the synthetic tests.
The focused behavioral tests use only temporary synthetic files and a fake
VBoxManage runner:
```sh
python3 -m unittest discover -s tests/vm -p 'test_install_prepare.py' -v
```
## First start of the prepared installer (ran once; VM stopped)
[`tests/vm/install_boot.py`](../tests/vm/install_boot.py) adds a separate,
fixed-target `check`/`start` command for the one powered-off installer state.
The ordinary [`guard.py`](../tests/vm/guard.py) deliberately continues to
reject the generated `.viso`; this installer guard does not loosen that normal
guard. It reuses the strict machine-readable parser and checks the exact
current-user-owned mode-`0600` identity manifest, pinned UUID and matching host
description, exact target name/configuration path, and current-user-owned
mode-`0700` target folder. It requires the VDI at SATA 0 and exactly
`Unattended-b14c850b-a21e-4a56-acca-785590f0c4ac-aux-iso.viso` at SATA 1;
SATA ports 2–29 must all be `none`. The `.viso` and any generated
`isolinux`/`preseed`/`postinstall` files are checked as current-user-owned
mode-`0600` regular files with exactly one hard link. The VM configuration and
VDI are also required to be regular current-user-owned files with exactly one
hard link. The guard never opens or prints installer-file contents.
The VM must remain powered off during preflight, have only NIC 1 in NAT mode,
and have no forwarding rules or shared folders. Symlinked paths, extra media,
unknown attachment metadata, and incomplete or mismatched identity refuse
before start.
The exact commands are:
```sh
python3 tests/vm/install_boot.py check
python3 tests/vm/install_boot.py start
```
`check` only reads the protected manifest, filesystem metadata, and bounded
VirtualBox inventory; it does not change VM state. `start` repeats all checks,
then issues only `VBoxManage startvm b14c850b-a21e-4a56-acca-785590f0c4ac
--type=headless`. It accepts no UUID, root, media, credential, or action
override. Each VBoxManage call uses the existing guard's 15-second deadline and
1 MiB per-stream output cap. Output is not copied into errors. If the start call
fails or times out, the result explicitly says the VM may have started and
requires a separate read-only inspection; there is no retry, cleanup, or
rollback. A successful command means only that VirtualBox accepted the start
request. It does not prove guest boot, Debian installation, or Guest Additions.
Focused fake-runner coverage is:
```sh
python3 -m unittest discover -s tests/vm -p 'test_install_boot.py' -v
```
These tests create synthetic private files and never call VBoxManage. Both
commands ran once against the actual VM after independent review. `check`
passed; `start` returned success; VirtualBox listed only the pinned target as
running. A private console screenshot showed Debian's base-system installer at
6%. The user then stopped Milestone 9, so the exact target was powered off and
read-only `list runningvms` returned no running VMs. Do not restart it without
a new explicit request. Installation completion and guest boot remain
unverified.
After installation, provision only synthetic lab packages,
fixed reviewed service units, synthetic WireGuard keys/configuration, and the
guest-local marker. Keep all server/client data, outputs, and working backups
under a guest-only root-owned `0700` directory; secret files are `0600`. The
guest-local marker must be readable by the unprivileged `wga` account for the
read-only marker gate. This work belongs to the next task after target creation
and guard review; it is not claimed as implemented here.
## Isolation and acceptance boundary
- Use synthetic server/client keys, configuration, settings, and vault contents
generated inside the guest. Preserve imported originals. Do not copy or read
production keys, configuration, backups, or credentials.
- The VM may have only one NAT management adapter, with no forwarding rules;
do not use bridged, host-only, internal, NAT-network, USB network, or physical
adapters. NAT-only VM egress still traverses the host's ordinary network; do
not claim host-independent network isolation. No host D-Bus, `/etc`, `/run`,
`/var`, WireGuard interface, service control, or general shared folder may be
mounted or forwarded into the guest.
- A guest system D-Bus and PID 1 must be guest-local. Keep console management
independent of the VPN under test. Tunnel probes target only synthetic
addresses reachable through the test tunnel, never the NAT management path.
- Provision only fixed, reviewed lab units/drop-ins and fixture hooks. Imported
configuration text remains data. Preserve administrator settings and prove
preflight refusal using synthetic fixtures rather than silently rewriting
`SaveConfig`.
- Exercise the public `Application::restart` flow with the real zbus service
adapter and Rustix observer. A test seam may inject persistence failures only
where already established; it may not replace service submission or runtime
observation as evidence of systemd behavior. Alpine fixture apply remains
separate evidence.
- Run the actual release executable separately from any `cfg(test)` driver.
Capture source/dirty identity, compiler/locked dependencies, target, artifact
SHA-256, and dynamic-library requirements for both artifacts. Prove each
executable starts in the guest.
- Keep secrets out of argv, environment variables, image metadata, logs,
transcripts, and evidence. Use bounded typed control, private files/channels,
monotonic timeouts, and bounded output. A timeout after service submission
means it may still complete; it is not cancellation.
- Before every scenario, reset to a recorded clean guest state and verify both
the pinned host identity and guest marker. Afterward, stop only resources
created by that run and verify their exact IDs. If cleanup is uncertain,
preserve IDs and report them; never use broad host cleanup. Preserve the
original failure if cleanup also fails.
## Required scenario evidence
For every run, record UTC start/end, guest and base identities, kernel/systemd/
D-Bus/WireGuard versions, both marker results, source/artifact identities, exact
commands, exit status and bounded redacted outcomes, backup/archive identities,
reset/cleanup result, and the acceptance criteria supported. Keep raw console
output private; evidence must contain no secrets or complete client config.
Record each scenario independently so failures cannot be masked by unrelated
fixture behavior.
| Scenario | Required observable evidence |
| --- | --- |
| Successful apply and fast job | Existing synthetic client has positive tunnel-only IPv4 traffic. Through one `Application`, create and Commit a client; show saved/runtime divergence and that Commit did not submit a service job. Cancel preview with no backup/job/mutation. Confirm restart: pre-restart backup completes first, real zbus job for the resolved primary unit reaches `done`, Rustix observes the expected interface and complete matching peer/routes, and durable verification is written. Verify new tunnel traffic, reopen and show verification is historical. Repeat with fast completion and a configured alias resolving to the primary unit; retain the completion signal. Include application outcomes, not journal inference alone. |
| Deletion and reservations | Establish positive traffic before deleting/Committing. Show the client remains in runtime and its address remains reserved until successful real restart and complete verification. Then show it absent, tunnel-only traffic fails within a bounded probe, durable reservation releases, and old backups remain. |
| Failure and deadline | Using fixed guest-only units/drop-ins, show terminal job failure and submitted-job deadline expiry with bounded results, accurate submitted/may-still-complete status, no automatic retry/rollback, no false Verified state, and retained reservations. Also show unresolved unit fails before submission and backup failure prevents restart. After late completion, explicitly observe and reset the guest; timeout is not cancellation. |
| Job succeeds, verification fails | A dedicated fixed fixture service completes while leaving the interface absent or peers mismatched. Show the real adapter reports verification failure and reservations remain. Separately inject saved-state write failure only through the established persistence seam after real restart/observation; show runtime changed without durable Verified state and `RecoveryRequired` blocks unsafe mutations. |
| Restore and manual recovery | Preview and confirm a known-good archive without vault password. Show precautionary backup, full saved settings/config/vault replacement, and unchanged runtime before explicit restart. Then run real restart/verification and expected tunnel check. Demonstrate documented manual recovery; no hidden rollback. Confirm permissions and reopen behavior. |
| Release executable smoke | In guest console/PTY, prove identified release binary starts from its private directory and settings resolve beside it. Exercise current-client/statistics view and one controlled Commit/restart through the real worker path; confirm exit/cancel restores terminal. Help/version work as non-root without settings; non-root management is rejected. Record output/backup permissions. |
The final M9 evidence must map these observations to M9-A1–A9 and the
specification-to-evidence matrix. State the tested scope, including IPv4 tunnel
coverage, synthetic dual-stack tests, unverified real IPv6 behavior, and
QR/device interoperability limits. A mocked bus, Alpine fixture apply, missing
prerequisite, skipped test, or guest boot alone does not satisfy real systemd
acceptance.
## Next action
None while Milestone 9 is paused by the user. The disposable VM is powered off;
do not resume installation, provision it, or run acceptance without a new
explicit request. No M9-A1–A9 check is complete.