Skip to content

Commit 26baabe

Browse files
branchseerclaude
andauthored
refactor(tests): adopt libtest-mimic and custom snapshot_test crate (#339)
## Why Before this PR, both `e2e_snapshots` and `plan_snapshots` used `harness = false` with a hand-rolled `fn main()` that: - Parsed one positional arg as a substring filter (`args().nth(1)`), and nothing else — no `--list`, no `--exact`, no `--ignored`, no `--test-threads`. - Iterated fixtures sequentially in a single thread, regardless of machine parallelism. - Printed fixture names to stdout as progress, mixed into `insta`'s own stderr diff output, with no structured "passed / failed / filtered out" summary. Two problems fall out of that: - **AI-agent friction.** Agents (Claude Code, Cursor, etc.) drive tests through `cargo test` and expect the built-in harness contract: filter a subset with `--exact`, enumerate with `--list`, read the standard "N passed; M failed" summary to know what happened. Custom output and missing flags force the agent to grep through ad-hoc progress prints or re-run the full suite just to isolate one fixture. Moving to `libtest-mimic` removes that friction — the harness behaves exactly like a normal Rust test binary. - **No parallelism.** `plan_snapshots` in particular is CPU-bound and embarrassingly parallel per fixture, but the old harness ran it single-threaded. `libtest-mimic` runs fixtures in parallel by default (via `available_parallelism()`), using the hardware the CI already pays for. (E2E tests spawn PTY children and send signals; on Linux we pin them back to single-threaded — see the "Platform-specific fixes" section below.) ## Summary Two-part migration of the e2e and plan snapshot test harnesses: ### 1. Adopt `libtest-mimic` for proper `cargo test` integration Migrating to `libtest-mimic` gives: - Proper `cargo test` flag support: `--filter`, `--list`, `--exact`, `--ignored`, `--test-threads` - Standard test runner output formatting (`running N tests` / `test X ... ok` / `test result: ok. ... passed; ... failed`) - Parallel fixture execution by default ### 2. Replace `insta` with a custom `snapshot_test` crate Created `crates/snapshot_test` with two methods that return `Result<(), String>` — containing a unified diff — so failures integrate cleanly with `libtest-mimic::Failed`: - `check_snapshot(name, actual)` — compares raw text - `check_json_snapshot(name, comment, value)` — serializes to pretty JSON, prepends `// {comment}` header, writes to `.jsonc` Key behaviors: - Drops the `insta` dependency and its transitive deps - Unified diff appears directly in the `failures:` section of test output (see [`crates/snapshot_test/README.md`](crates/snapshot_test/README.md) for why `insta` didn't fit) - Plan JSON snapshots now use `.jsonc` extension with `// run ...` command headers (e.g. `// run build --recursive`); e2e snapshots keep `.snap` - Existing `insta` YAML headers stripped from all 310 snapshot files - `UPDATE_SNAPSHOTS=1` env var accepts new output in-place (replaces `INSTA_UPDATE`) - Snapshot directories excluded from `oxfmt` via `.oxfmtrc.json` to prevent trailing-comma insertion in `.jsonc` files ### Platform-specific fixes - **Windows**: CRLF line endings are normalized to LF when reading snapshot files - **Linux**: e2e tests forced to single-threaded (`test_threads = 1`), matching the pre-PR custom-harness behavior. `libtest-mimic` defaults to `available_parallelism()`, which caused PTY/signal-routing races in the `ctrl-c` test. macOS and Windows keep parallel execution. ## Test plan - [x] `cargo test -p vite_task_plan --test plan_snapshots` — 45 tests pass - [x] `cargo test -p vite_task_bin --test e2e_snapshots` — 36 tests pass - [x] `cargo test -- --list` lists all fixture names - [x] `cargo test -- --exact <name>` filters correctly - [x] Tampered `.snap`/`.jsonc` file produces a unified diff in the failure message, pointing to the `.snap.new` / `.jsonc.new` file - [x] `UPDATE_SNAPSHOTS=1` accepts new output in-place and clears `.new` files - [x] CI stability: 5 consecutive green runs across Linux, macOS (arm64 + x86_64), Windows, and musl 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent b2e2e2c commit 26baabe

373 files changed

Lines changed: 835 additions & 2730 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.oxfmtrc.json

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
11
{
22
"singleQuote": true,
3-
"ignorePatterns": ["crates/fspy_detours_sys/detours", "crates/vite_task_graph/run-config.ts"]
3+
"ignorePatterns": [
4+
"crates/fspy_detours_sys/detours",
5+
"crates/vite_task_graph/run-config.ts",
6+
"**/fixtures/*/snapshots"
7+
]
48
}

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ cargo test # All tests
4747
cargo test -p vite_task_bin --test e2e_snapshots # E2E snapshot tests
4848
cargo test -p vite_task_plan --test plan_snapshots # Plan snapshot tests
4949
cargo test --test e2e_snapshots -- stdin # Filter by test name
50-
INSTA_UPDATE=always cargo test # Update snapshots
50+
UPDATE_SNAPSHOTS=1 cargo test # Update snapshots
5151
```
5252

5353
Integration tests (e2e, plan, fspy) require `pnpm install` in `packages/tools` first. You don't need `pnpm install` in test fixture directories.

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ cargo test # All tests
4545
cargo test -p vite_task_bin --test e2e_snapshots # E2E snapshot tests
4646
cargo test -p vite_task_plan --test plan_snapshots # Plan snapshot tests
4747
cargo test --test e2e_snapshots -- stdin # Filter by test name
48-
INSTA_UPDATE=always cargo test # Update snapshots
48+
UPDATE_SNAPSHOTS=1 cargo test # Update snapshots
4949
```
5050

5151
Integration tests (e2e, plan, fspy) require `pnpm install` in `packages/tools` first. You don't need `pnpm install` in test fixture directories.

0 commit comments

Comments
 (0)