Testing

The three test tiers, goldens, real input, the nd-test locators, the MCP bridge and how nd test runs them.

Captured on macOS 27.0, 2026-10-14.

Every backend builds one semantic tree: role, name, value, test id, bounds, enabled, focused, checked, selected, expanded. Tests read and act on that tree, never on pixels or the OS accessibility API, so a locator written in tier 1 reads the same against a real window in tier 3.

The rules for writing tests as an agent are in for agents.

Tier 1: logic

#[nd::test] runs a test on TestBackend, which records backend ops and has a deterministic clock. The same test compiles for wasm32-unknown-unknown and runs in headless Chromium (just web-test).

#[nd::test]
fn starts_at_zero(cx: &mut TestApp) {
    let window = hello::open(cx).id();
    assert!(text(cx, window).contains("0 clicks"), "{}", text(cx, window));
}

#[nd::test]
fn add_one_counts_each_click(cx: &mut TestApp) {
    let window = hello::open(cx).id();
    click(cx, window, "add");
    click(cx, window, "add");
    assert!(text(cx, window).contains("2 clicks"), "{}", text(cx, window));
}

From examples/hello/tests/tier1.rs, line 7.

Locators work in tier 1 through nd_test::TestHarness, which wraps a TestApp and runs requests through the same handler the automation socket uses (see crates/nd-test).

Tier 2: backends and goldens

Tier 2 renders with a real backend inside the test process. Three kinds of check live here.

The appearance audit is covered below. Goldens compare a screenshot with a PNG in the example's tests/goldens/. A mismatch past the tolerance writes the difference beside the golden (web-light.diff.png, or <golden>.actual.png) and fails. To re-record:

• ND_BLESS=1: the web golden of hello (web_golden.rs). It runs only with ND_WEB_GOLDEN=1 and needs Chrome (ND_CHROME picks another).
• ND_UPDATE_GOLDENS=1: the macOS goldens of the gallery and overrides examples, which capture the composited window including glass (goldens.rs).

let bless = std::env::var_os("ND_BLESS").is_some_and(|v| v == "1");

From examples/hello/tests/web_golden.rs, line 92.

The third kind is the AppKit conformance tests: harness = false binaries, one process per test, that open offscreen windows and check each component's native mapping (an NSMenu, an NSTableView). Glass only appears in tier 3 captures.

Fluent goldens on Windows can only be recorded on Windows. The Windows backend has not run yet, so none exist.

Appearance audit

nd::audit::Audit renders every window you name in light, dark and two matugen themes on each backend the target links. It reads back every text run and fill that was painted and fails when a colour is not a theme token (or a state derived from one), or when text has less than WCAG contrast against what it sits on: 4.5:1, or 3:1 for large text. Each backend is its own test, such as appearance_audit::appkit. Matugen themes run on GPUI only.

fn main() {
    nd::audit::Audit::new("hello", hello::app).window("main", |cx| hello::open(cx).id()).run()
}

From examples/hello/tests/appearance.rs, line 2.

The test target is declared with harness = false in Cargo.toml. Audit::allow lists brand colours the theme names on purpose; contrast is still checked. nd check runs the audit for every package that has tests/appearance.rs.

Tier 3: real input

Tier 3 starts the app as its own process with NATIVE_AUTOMATION=1 and drives it over the automation socket. The child prints ND_AUTOMATION_LISTENING path=<socket> on stderr and launch_app waits for it. Each launch gets its own socket and ND_APP_DATA_DIR, so parallel launches share no path.

use nd_test::{LaunchOptions, Role, expect, launch_app};

#[test]
fn real_add_one_updates_the_count() {
    let app = launch_app(LaunchOptions::new(env!("CARGO_BIN_EXE_hello"))).expect("launch hello");
    expect(&app.get_by_role(Role::Text).name("0").exact()).to_be_visible().unwrap();
    app.get_by_test_id("add").click().unwrap();
    expect(&app.get_by_role(Role::Text).name("1").exact()).to_be_visible().unwrap();
}

From examples/hello/tests/locators.rs, line 7.

Test names that start with real_, or live in a real module, are tier 3. nextest runs them one at a time (except on Linux), and nd test --tier real selects only them. connect_app attaches to a host somebody else started.

Locators and expect

app.get_by_role, get_by_test_id, get_by_text, get_by_label, get_by_placeholder and locator("role=button[name=\"Save\"] >> nth=1") return lazy locators. An action re-resolves the locator against the current tree and waits until the node is visible, enabled, has real geometry and has stopped moving. expect(&locator) has to_be_visible, to_have_text, to_contain_text and the like, and polls until the session timeout (5 s, change it with Session::with_timeout). The API blocks; there are no futures.

Selector parts chain with >>: testid=, role=, text=, label=, type=, has-text=, has=(...), nth=, first, last. The grammar is in selector.rs.

Tier 3 adds app.cursor, app.capture and app.record for the real cursor, screenshots and recordings.

ndshell

ndshell is the test-only helper that tier 3 stands on. Apps never link it.

• The machine lock. Real HID input, activation and screen capture share one window server, so tests that use them hold an advisory flock (real-input.lock under ~/Library/Caches/nd/ on macOS, $XDG_CACHE_HOME/nd/ or ~/.cache/nd/ on Linux, %LOCALAPPDATA%\nd\ on Windows; ND_LOCK_DIR moves it). A second run, in any worktree, queues and prints who holds it. ndshell lock status shows the holder and ndshell lock -- <command> runs a command under the lock. nd_test::real_input(purpose) takes it from a test. launch_app takes it by default on macOS.
• Preflight (macOS). ndshell preflight checks that no window manager that intercepts synthetic clicks runs (OmniWM, yabai, AeroSpace, Amethyst), that the process may post events, and that no modifier key is stuck.
• Grant (macOS). ndshell grant <binary> writes Screen Recording, Accessibility and PostEvent rows into the system TCC database, keyed to the signing identifier so they survive rebuilds. It needs SIP off and root. Grant the process TCC treats as responsible: the terminal, or sshd-keygen-wrapper over ssh, not the test binary.
• Hyprland runner (Linux). ndshell::hyprland::Hyprland::start boots a private headless Hyprland per test in its own runtime directory, with a virtual pointer and keyboard and output capture, so Linux tier 3 tests run in parallel. It nests on a headless labwc for a render node. ND_HYPRLAND, ND_LABWC and ND_RENDER_NODE override the binaries and the device; ND_KEEP_HYPRLAND=1 keeps the logs. An example is real_overlays.rs.
• Windows agent. ndshell agent is meant to run in the interactive session, since processes started over ssh cannot post input. The Windows backend has not run yet, so this path is untested.

The incident tests of hello launch the real binary with ND_DEBUG_PANIC, ND_DEBUG_HANG_MS and ND_DEBUG_CRASH and assert on the bundle (incidents.rs, diagnostics).

MCP bridge

nd-mcp lets an agent drive a running app. It has three tools:

• execute({code}): runs JavaScript in QuickJS with app bound to the locator API.
• snapshot: returns the compact semantic tree, one line per node with a [ref=eN].
• reset: relaunches the app, or reconnects to the socket.

nd-mcp --example hello
nd-mcp --socket "$ND_AUTOMATION_SOCKET"

--bin <path> (env ND_MCP_BIN) launches any binary with automation on; arguments after -- go to it.

nd test

nd test maps --tier to a nextest profile from .config/nextest.toml (which nd new writes for your app too) and runs cargo nextest run.

Every test is its own process. Tests that hold the machine lock sit in one nextest test group, so they also queue behind each other inside a run. A test that takes the lock but is missing from the group wastes a parallel slot; keep the two in step.

nd test -p gallery forms
nd test --tier all

Real-input tests also run from the repository with cargo nextest run --profile real -p hello.