Skip to content

Synthetic input & virtual devices for test-gen (keyboard/pointer/touch/gesture · mic · webcam · USB/HDMI/Ethernet/Wi-Fi/Bluetooth hot-plug)

develop specs/develop/synthetic-input-and-virtual-devices.kmd

The Koder Stack test generators MUST be able to drive EVERY input a real user can produce on their computer, and to simulate the connection / disconnection of every device class — so parity tests (visual, animation, audio) and flow tests actually EXERCISE the flows end-to-end instead of stopping at a debug hook. Defines the normative set of input modalities and device-event classes, the Linux-native, self-hostable virtual driver for each (uinput, wlroots virtual-input, PipeWire virtual source, v4l2loopback, mac80211_hwsim, BlueZ hci_vhci, dummy/veth NICs, QEMU device hot-plug), the harness contract, isolation/safety rules, and the generation contract. This SUPERSEDES the interim "input injection is blocked → debug-trigger hook" limitation in animation-parity § R2.1 (the hook stays as a fallback). Runs on the s.khost1 VM/privileged-container layer, never on the owner's real hardware.

When this spec applies

Primary triggers

All triggers

Specification body

Spec — Synthetic input & virtual devices for test-gen

Why

Parity tests are only as good as the flows they can trigger. The animation-parity spec had to fall back to a debug-trigger hook because headless input injection was unavailable on the dev harness. That hook is fine as a unit-level shim, but it bypasses the real input path — it cannot prove the pointer actually hit the hot corner, that a drag-and-drop landed, that plugging a USB stick opens the file manager, or that connecting a Bluetooth headset re-routes audio. The Stack test generators MUST be able to produce the full range of real user and hardware input, on a virtual seat, deterministically.

Scope

The harness MUST provide a virtual driver for every modality below, usable by generated tests across all Koder surfaces (native shell/compositor, Flutter, web, Android, CLI/TUI) and by all parity test types (visual, animation, audio) — and the test-gen MUST be able to script them.

Runs on the s.khost1 VM / privileged-container layer (policies/heavy-work-isolation.kmd); several drivers also work in a privileged container. Never drives the owner's real hardware (R6).

R1 — Input & device modalities (the normative matrix)

Each modality has a Linux-native, self-hostable virtual driver. "Where" notes the minimum host: C = privileged container, VM = needs the s.khost1 VM (QEMU) layer.

#ModalityVirtual driver (primary)WhereNotes
1Keyboard (keys, chords, repeat, layouts, IME)uinput virtual kbd / wlroots virtual-keyboard-v1Cfull scancode + modifier control
2Pointer (move, click, multi-button, scroll/wheel)uinput rel/abs pointer / wlroots virtual-pointer-v1Cabsolute mode for hot-corner hits
3Drag & dropsequenced pointer down→move→up (1/2) + DnD protocolCinter-window DnD on Wayland
4Touch / multitouch / gestures (pinch, swipe, edge)uinput multitouch slots (ABS_MT_*)Cdrives mobile/touch surfaces
5Stylus / tablet (pressure, tilt)uinput tablet (ABS_PRESSURE/TILT)Cfor note/draw surfaces
6Microphone audio inPipeWire/PulseAudio virtual source fed a wavC"virtual mic" → wake-word, voice, calls
7Webcam photov4l2loopback /dev/videoN fed a still imageCapp sees a camera with a fixed frame
8Webcam videov4l2loopback fed a video stream (kodec)Crecording/streaming flows
9USB connect/disconnectQEMU device_add usb-* / usbip / vhci-hcdVMstorage, HID, serial; udev hot-plug fires
10HDMI / monitor hot-plug (connect, EDID, resolution)wlroots headless create/destroy output / QEMU display hot-plug / virtual EDIDC/VMmulti-monitor, hot-plug, mode change
11Ethernet (cable up/down, link speed)dummy/veth NIC + ip link set … carrier on/offCcarrier flap, DHCP flows
12Wi-Fi (scan, associate, signal, roam)mac80211_hwsim virtual radios + hostapdCdrives the Kolide Wi-Fi tile (kolide#044c) end-to-end
13Bluetooth (pair, connect, profiles)BlueZ hci_vhci / btvirt virtual HCICBT tile (kolide#044d), audio re-route
14Battery / power / lid / brightnessupower/sysfs test backend / ACPI events in VMC/VMlow-battery sound, lid-close flows
15Removable media mountvirtual USB block (9) + udisksVM"plug drive → file manager opens"

The matrix is the contract; the registry registries/synthetic-input-coverage.md tracks which are implemented and on which harness.

R2 — Harness contract (the driver API)

The harness exposes one scripting surface the test-gen targets — a CLI koder-vdev (+ a thin lib binding) with verbs per modality:

koder-vdev seat create                       # uinput kbd+pointer+touch on a virtual seat
koder-vdev key  "Super+a"                     # chords, hold/release, type "text"
koder-vdev pointer move 0 0 ; pointer click left
koder-vdev drag  from <x,y> to <x,y> [window]
koder-vdev mic   play sample.wav              # virtual source
koder-vdev cam   still photo.png | cam video clip.webm
koder-vdev usb   add storage img.raw | usb del <id>
koder-vdev hdmi  add --edid 4k | hdmi del <id>
koder-vdev net   eth carrier off | wifi connect <ssid> --signal -55 | bt pair <addr>
koder-vdev marker <label>                     # shared trigger marker (AV timeline)
  • Every action emits a trigger marker on the same timeline the animation (§R2.1) and audio (§A2) prongs consume, so input → motion → sound stay frame/sample-aligned.
  • Actions are synchronous + acknowledged (return when the event has been delivered/settled), never fire-and-pray with a sleep.
  • The driver set is provisioned per-test from a declared requirement block (R4), created on setup and torn down on teardown (R6).

R3 — Determinism

  • Absolute pointer/touch coordinates (not relative drift) so a hot-corner or edge-gesture hit is exact and repeatable.
  • Virtual Wi-Fi/BT/USB present fixed identities (SSID, MAC, serial, signal) provided by the test — no real-world variability.
  • Webcam/mic feed fixed media (committed fixtures), so vision/audio flows are reproducible.
  • All randomness (e.g. signal jitter) is seeded by the test, never live.

R4 — Generation contract

/k-test MUST let a generated flow/parity test declare the inputs & devices it needs, and the harness provisions them. A test header block:

[requires]
seat   = ["keyboard", "pointer", "touch"]
mic    = "fixtures/wake-ok.wav"
camera = { photo = "fixtures/face.png" }
usb    = [{ class = "storage", image = "fixtures/stick.raw" }]
wifi   = [{ ssid = "Koder-Test", signal = -55 }]
bt     = [{ addr = "AA:BB:CC:00:11:22", profile = "a2dp" }]
hdmi   = [{ edid = "4k60" }]
  • The generator emits the koder-vdev calls to set up/tear down and the assertions on the surface's reaction (visual via animation/visual specs, audio via audio spec, plus state assertions — "file manager opened", "Wi-Fi tile shows connected").
  • Where a modality is not yet implemented on the target harness (registry), the generator emits the test as skipped with a reason pointing at the gap — never silently dropped (/k-go Regra 15 spirit).

R5 — Integration with the parity specs

  • Replaces the animation-parity §R2.1 debug-hook as the primary trigger: real pointer/touch fires the hot-corner ripple, real clicks open popovers. The debug hook remains a fallback for environments without a virtual seat (and for pure unit prongs).
  • Powers device-driven flows the parity engine must verify: plug USB → file manager + mount sound; connect Wi-Fi via the Kolide tile (kolide#044c) → status icon + connect chime; pair BT headset (kolide#044d) → audio re-route; HDMI hot-plug → multi-monitor relayout.
  • Feeds the audio spec's virtual mic and the visual/animation specs' pointer/touch.

R6 — Isolation & safety (hard rules)

  • Never drive the owner's real input/devices. All virtual devices live on the harness's virtual seat / inside the s.khost1 VM, scoped to the test's namespace/seat, and are destroyed on teardown (even on failure — teardown is finally).
  • ❌ Never bring a virtual NIC/Wi-Fi/BT onto a real network or pair with a real device — virtual radios (mac80211_hwsim) and HCIs (hci_vhci) are self-contained.
  • ❌ Never run on the owner's laptop (hostile seat; would hijack the real pointer/keyboard) — s.khost1 only (heavy-work-isolation.kmd).
  • A leaked/undeleted virtual device is a test failure (a teardown audit asserts the seat is clean after each test).

R7 — Where each runs (build-out order)

  1. Container-friendly first (uinput, wlroots virtual-input, virtual mic, v4l2loopback, dummy/veth, mac80211_hwsim, hci_vhci) — most of the matrix, unblocks keyboard/pointer/touch/mic/cam/wifi/bt/ethernet.
  2. VM layer (USB/HDMI hot-plug, ACPI/power, removable media) via QEMU device_add/display hot-plug on s.khost1.

Tests of the test contract

IDTest
T1koder-vdev seat create + pointer move 0 0 fires the Kolide hot-corner ripple via the REAL input path (not the debug hook) — animation prong observes it.
T2key "Super" opens the launcher; type "..." filters results.
T3drag from→to performs an inter-window drag-and-drop and the drop is received.
T4mic play wake-ok.wav triggers the wake-word ack (specs/voice/); audio spec A1 observes the ack sound.
T5cam still face.png makes a camera-using surface see the fixture frame.
T6usb add storage fires a udev hot-plug → file manager opens + mount sound (audio A1).
T7wifi connect Koder-Test --signal -55 drives the Kolide Wi-Fi tile (kolide#044c) to connected + the correct signal icon (kolide#044f).
T8bt pair … --profile a2dp connects via the BT tile (kolide#044d) and re-routes audio.
T9hdmi add --edid 4k60 hot-plugs a second output; the compositor relayouts.
T10eth carrier off/on flaps the link; the network state + DHCP flow react.
T11Isolation: after any test, the teardown audit finds zero leftover virtual devices and no real-device interaction.
T12A test requiring an unimplemented modality is emitted skipped with a gap reason, not silently dropped.

R8 — Existing gaps (snapshot at ratification)

At ratification (2026-06-07) the harness has no synthetic-input or virtual-device layer — the animation-parity prong relies on the debug hook, and the Kolide Wi-Fi/BT/network flows are tested only at the pure-list level (kolide#044c/#044d normalize functions), never end-to-end through a virtual radio. koder-vdev does not exist yet. Track build-out in registries/synthetic-input-coverage.md (new — created with this spec).

Open follow-ups

  • Build koder-vdev + the container-friendly drivers (R7 step 1) — KSTACK-180.
  • Add the VM-layer drivers (USB/HDMI/power) on s.khost1 (R7 step 2).
  • Wire the parity specs' triggers to koder-vdev (replace the debug hook as primary).

References