Synthetic input & virtual devices for test-gen (keyboard/pointer/touch/gesture · mic · webcam · USB/HDMI/Ethernet/Wi-Fi/Bluetooth hot-plug)
develop specs/develop/synthetic-input-and-virtual-devices.kmd
The Koder Stack test generators MUST be able to drive EVERY input a real user can produce on their computer, and to simulate the connection / disconnection of every device class — so parity tests (visual, animation, audio) and flow tests actually EXERCISE the flows end-to-end instead of stopping at a debug hook. Defines the normative set of input modalities and device-event classes, the Linux-native, self-hostable virtual driver for each (uinput, wlroots virtual-input, PipeWire virtual source, v4l2loopback, mac80211_hwsim, BlueZ hci_vhci, dummy/veth NICs, QEMU device hot-plug), the harness contract, isolation/safety rules, and the generation contract. This SUPERSEDES the interim "input injection is blocked → debug-trigger hook" limitation in animation-parity § R2.1 (the hook stays as a fallback). Runs on the s.khost1 VM/privileged-container layer, never on the owner's real hardware.
When this spec applies
Primary triggers
- Author or extend synthetic-input / virtual-device drivers for test-gen
All triggers
- A parity/flow test needs to send keyboard/mouse/touch/gesture input
- A test needs to feed microphone audio or a webcam photo/video
- A test needs to simulate plugging/unplugging USB/HDMI/Ethernet/Wi-Fi/Bluetooth
- Build or extend the s.khost1 test harness's virtual-device layer
Specification body
Spec — Synthetic input & virtual devices for test-gen
Why
Parity tests are only as good as the flows they can trigger. The animation-parity spec had to fall back to a debug-trigger hook because headless input injection was unavailable on the dev harness. That hook is fine as a unit-level shim, but it bypasses the real input path — it cannot prove the pointer actually hit the hot corner, that a drag-and-drop landed, that plugging a USB stick opens the file manager, or that connecting a Bluetooth headset re-routes audio. The Stack test generators MUST be able to produce the full range of real user and hardware input, on a virtual seat, deterministically.
Scope
The harness MUST provide a virtual driver for every modality below, usable by generated tests across all Koder surfaces (native shell/compositor, Flutter, web, Android, CLI/TUI) and by all parity test types (visual, animation, audio) — and the test-gen MUST be able to script them.
Runs on the s.khost1 VM / privileged-container layer
(policies/heavy-work-isolation.kmd); several drivers also work in a
privileged container. Never drives the owner's real hardware (R6).
R1 — Input & device modalities (the normative matrix)
Each modality has a Linux-native, self-hostable virtual driver. "Where" notes the minimum host: C = privileged container, VM = needs the s.khost1 VM (QEMU) layer.
| # | Modality | Virtual driver (primary) | Where | Notes |
|---|---|---|---|---|
| 1 | Keyboard (keys, chords, repeat, layouts, IME) | uinput virtual kbd / wlroots virtual-keyboard-v1 | C | full scancode + modifier control |
| 2 | Pointer (move, click, multi-button, scroll/wheel) | uinput rel/abs pointer / wlroots virtual-pointer-v1 | C | absolute mode for hot-corner hits |
| 3 | Drag & drop | sequenced pointer down→move→up (1/2) + DnD protocol | C | inter-window DnD on Wayland |
| 4 | Touch / multitouch / gestures (pinch, swipe, edge) | uinput multitouch slots (ABS_MT_*) | C | drives mobile/touch surfaces |
| 5 | Stylus / tablet (pressure, tilt) | uinput tablet (ABS_PRESSURE/TILT) | C | for note/draw surfaces |
| 6 | Microphone audio in | PipeWire/PulseAudio virtual source fed a wav | C | "virtual mic" → wake-word, voice, calls |
| 7 | Webcam photo | v4l2loopback /dev/videoN fed a still image | C | app sees a camera with a fixed frame |
| 8 | Webcam video | v4l2loopback fed a video stream (kodec) | C | recording/streaming flows |
| 9 | USB connect/disconnect | QEMU device_add usb-* / usbip / vhci-hcd | VM | storage, HID, serial; udev hot-plug fires |
| 10 | HDMI / monitor hot-plug (connect, EDID, resolution) | wlroots headless create/destroy output / QEMU display hot-plug / virtual EDID | C/VM | multi-monitor, hot-plug, mode change |
| 11 | Ethernet (cable up/down, link speed) | dummy/veth NIC + ip link set … carrier on/off | C | carrier flap, DHCP flows |
| 12 | Wi-Fi (scan, associate, signal, roam) | mac80211_hwsim virtual radios + hostapd | C | drives the Kolide Wi-Fi tile (kolide#044c) end-to-end |
| 13 | Bluetooth (pair, connect, profiles) | BlueZ hci_vhci / btvirt virtual HCI | C | BT tile (kolide#044d), audio re-route |
| 14 | Battery / power / lid / brightness | upower/sysfs test backend / ACPI events in VM | C/VM | low-battery sound, lid-close flows |
| 15 | Removable media mount | virtual USB block (9) + udisks | VM | "plug drive → file manager opens" |
The matrix is the contract; the registry
registries/synthetic-input-coverage.md tracks which are implemented and
on which harness.
R2 — Harness contract (the driver API)
The harness exposes one scripting surface the test-gen targets — a CLI
koder-vdev (+ a thin lib binding) with verbs per modality:
koder-vdev seat create # uinput kbd+pointer+touch on a virtual seat
koder-vdev key "Super+a" # chords, hold/release, type "text"
koder-vdev pointer move 0 0 ; pointer click left
koder-vdev drag from <x,y> to <x,y> [window]
koder-vdev mic play sample.wav # virtual source
koder-vdev cam still photo.png | cam video clip.webm
koder-vdev usb add storage img.raw | usb del <id>
koder-vdev hdmi add --edid 4k | hdmi del <id>
koder-vdev net eth carrier off | wifi connect <ssid> --signal -55 | bt pair <addr>
koder-vdev marker <label> # shared trigger marker (AV timeline)
- Every action emits a trigger marker on the same timeline the animation (§R2.1) and audio (§A2) prongs consume, so input → motion → sound stay frame/sample-aligned.
- Actions are synchronous + acknowledged (return when the event has been delivered/settled), never fire-and-pray with a sleep.
- The driver set is provisioned per-test from a declared requirement block (R4), created on setup and torn down on teardown (R6).
R3 — Determinism
- Absolute pointer/touch coordinates (not relative drift) so a hot-corner or edge-gesture hit is exact and repeatable.
- Virtual Wi-Fi/BT/USB present fixed identities (SSID, MAC, serial, signal) provided by the test — no real-world variability.
- Webcam/mic feed fixed media (committed fixtures), so vision/audio flows are reproducible.
- All randomness (e.g. signal jitter) is seeded by the test, never live.
R4 — Generation contract
/k-test MUST let a generated flow/parity test declare the inputs &
devices it needs, and the harness provisions them. A test header block:
[requires]
seat = ["keyboard", "pointer", "touch"]
mic = "fixtures/wake-ok.wav"
camera = { photo = "fixtures/face.png" }
usb = [{ class = "storage", image = "fixtures/stick.raw" }]
wifi = [{ ssid = "Koder-Test", signal = -55 }]
bt = [{ addr = "AA:BB:CC:00:11:22", profile = "a2dp" }]
hdmi = [{ edid = "4k60" }]
- The generator emits the
koder-vdevcalls to set up/tear down and the assertions on the surface's reaction (visual via animation/visual specs, audio via audio spec, plus state assertions — "file manager opened", "Wi-Fi tile shows connected"). - Where a modality is not yet implemented on the target harness
(registry), the generator emits the test as skipped with a reason
pointing at the gap — never silently dropped (
/k-goRegra 15 spirit).
R5 — Integration with the parity specs
- Replaces the animation-parity §R2.1 debug-hook as the primary trigger: real pointer/touch fires the hot-corner ripple, real clicks open popovers. The debug hook remains a fallback for environments without a virtual seat (and for pure unit prongs).
- Powers device-driven flows the parity engine must verify: plug USB → file manager + mount sound; connect Wi-Fi via the Kolide tile (kolide#044c) → status icon + connect chime; pair BT headset (kolide#044d) → audio re-route; HDMI hot-plug → multi-monitor relayout.
- Feeds the audio spec's virtual mic and the visual/animation specs' pointer/touch.
R6 — Isolation & safety (hard rules)
- ❌ Never drive the owner's real input/devices. All virtual devices
live on the harness's virtual seat / inside the s.khost1 VM, scoped to
the test's namespace/seat, and are destroyed on teardown (even on
failure — teardown is
finally). - ❌ Never bring a virtual NIC/Wi-Fi/BT onto a real network or pair with a real device — virtual radios (mac80211_hwsim) and HCIs (hci_vhci) are self-contained.
- ❌ Never run on the owner's laptop (hostile seat; would hijack the real
pointer/keyboard) — s.khost1 only (
heavy-work-isolation.kmd). - A leaked/undeleted virtual device is a test failure (a teardown audit asserts the seat is clean after each test).
R7 — Where each runs (build-out order)
- Container-friendly first (uinput, wlroots virtual-input, virtual mic, v4l2loopback, dummy/veth, mac80211_hwsim, hci_vhci) — most of the matrix, unblocks keyboard/pointer/touch/mic/cam/wifi/bt/ethernet.
- VM layer (USB/HDMI hot-plug, ACPI/power, removable media) via QEMU
device_add/display hot-plug on s.khost1.
Tests of the test contract
| ID | Test |
|---|---|
| T1 | koder-vdev seat create + pointer move 0 0 fires the Kolide hot-corner ripple via the REAL input path (not the debug hook) — animation prong observes it. |
| T2 | key "Super" opens the launcher; type "..." filters results. |
| T3 | drag from→to performs an inter-window drag-and-drop and the drop is received. |
| T4 | mic play wake-ok.wav triggers the wake-word ack (specs/voice/); audio spec A1 observes the ack sound. |
| T5 | cam still face.png makes a camera-using surface see the fixture frame. |
| T6 | usb add storage fires a udev hot-plug → file manager opens + mount sound (audio A1). |
| T7 | wifi connect Koder-Test --signal -55 drives the Kolide Wi-Fi tile (kolide#044c) to connected + the correct signal icon (kolide#044f). |
| T8 | bt pair … --profile a2dp connects via the BT tile (kolide#044d) and re-routes audio. |
| T9 | hdmi add --edid 4k60 hot-plugs a second output; the compositor relayouts. |
| T10 | eth carrier off/on flaps the link; the network state + DHCP flow react. |
| T11 | Isolation: after any test, the teardown audit finds zero leftover virtual devices and no real-device interaction. |
| T12 | A test requiring an unimplemented modality is emitted skipped with a gap reason, not silently dropped. |
R8 — Existing gaps (snapshot at ratification)
At ratification (2026-06-07) the harness has no synthetic-input or
virtual-device layer — the animation-parity prong relies on the debug hook,
and the Kolide Wi-Fi/BT/network flows are tested only at the pure-list
level (kolide#044c/#044d normalize functions), never end-to-end through a
virtual radio. koder-vdev does not exist yet. Track build-out in
registries/synthetic-input-coverage.md (new — created with this spec).
Open follow-ups
- Build
koder-vdev+ the container-friendly drivers (R7 step 1) — KSTACK-180. - Add the VM-layer drivers (USB/HDMI/power) on s.khost1 (R7 step 2).
- Wire the parity specs' triggers to
koder-vdev(replace the debug hook as primary).
References
specs/develop/animation-parity-tdds.kmdspecs/develop/audio-parity-tdds.kmdspecs/develop/visual-regression-tdds.kmdspecs/develop/gesture-tdds.kmdpolicies/heavy-work-isolation.kmdpolicies/headless-first.kmdpolicies/self-hosted-first.kmd