Skip to content

feat(coverage): rivet coverage --aggregate — file-based cross-repo V&V matrix (#188 sub-issue 3)#270

Merged
avrabe merged 1 commit into
mainfrom
feat/issue-188-vv-matrix-aggregator
May 15, 2026
Merged

feat(coverage): rivet coverage --aggregate — file-based cross-repo V&V matrix (#188 sub-issue 3)#270
avrabe merged 1 commit into
mainfrom
feat/issue-188-vv-matrix-aggregator

Conversation

@avrabe
Copy link
Copy Markdown
Contributor

@avrabe avrabe commented May 12, 2026

Summary

Picks up sub-issue 3 of the 2026-04-26 decomposition of #188 — the cross-repo aggregator — in the form that triage comment already endorsed:

walks an org (probably via a config file listing repos rather than GitHub API at first), composes the matrix.

This PR is the file-based form: each repo's CI runs rivet coverage --matrix --format json and uploads the result; a top-level job merges those files:

rivet coverage --aggregate rivet.json loom.json sigil.json ... --format markdown

No GitHub API access is needed, so it works inside the org-scoped triage agent and as an ordinary CI artifact-collection job.

Sub-issues 1 (vv-coverage schema, #232) and 2 (rivet coverage --matrix, #243) are already merged; this is the next in-scope chunk.

Behaviour

  • --aggregate <FILE>... (repeatable / multi-value) on rivet coverage. Implies matrix mode; the local project is not read. Conflicts with --tests at the clap layer.
  • Inputs are the JSON envelope render_matrix_json emits ({command, columns, repos:[{id, repo, techniques_applied, techniques_gated_in_ci, notes, cells}]}).
  • Output formats: text (default), markdown, html, json — the same renderers as --matrix.
  • Columns = sorted union of every input's techniques. Cell legend unchanged (· absent, applied, applied + CI-gated).
  • Duplicate (repo, id) rows are coalesced (first occurrence wins), so re-running the aggregator over overlapping inputs is idempotent.
  • The merged JSON re-feeds the aggregator unchanged (round-trips).
  • Bad / wrong-shaped input fails with a diagnostic naming the file (reading matrix JSON <path>, parsing matrix JSON <path>, <path>: ... missing 'repos' array).

#188 acceptance status

  • Extend rivet schema to capture technique-adoption status per repo — schemas/vv-coverage.yaml / repo-status (feat(schemas): vv-coverage — repo-status type for V&V technique tracking (#188) #232, merged).
  • New subcommand rivet coverage --matrix emits markdown / HTML / JSON — feat(coverage): rivet coverage --matrix — V&V matrix from repo-status artifacts #243, merged.
  • Aggregator that walks the pulseengine org and composes a cross-repo matrix (CI-friendly) — this PR, file-based form (--aggregate). The GitHub-org-API form, if still wanted, is a follow-up; the triage comment explicitly scoped the file-based version first.
  • Published dashboard (GitHub Pages / rivet-web component) — sub-issue 4, separate PR.
  • Integration: pulseengine.eu surfaces the live matrix — out of scope for a repo-scoped run.
  • Per-technique trend line — stretch.
  • Per-standard mapping view — stretch.

So this PR does not close #188; sub-issue 4 and the two stretch goals remain. Filed as draft for the maintainer's call on whether the file-based aggregator fully satisfies the "CI-friendly aggregator" bullet or whether a GitHub-API variant is also wanted.

Test plan

  • cargo test -p rivet-cli — green (all suites, incl. 3 new coverage_aggregate_* tests and the existing coverage_matrix_* tests).
  • cargo clippy -p rivet-cli --tests — clean.
  • cargo fmt -p rivet-cli --check — clean.
  • cargo run -p rivet-cli -- validate — 6 errors / 140 warnings, unchanged from origin/main (the 6 live in the spar: external fixture).
  • Manual smoke: merge two single-repo JSONs across all four formats; dedup on repeated input; re-feed merged JSON; bad-input diagnostics.
  • CI on this PR.

Note on process

Consulted https://pulseengine.eu/blog/ at the start of this run and again before opening this PR — no posts there describe branch/PR/test conventions beyond the spec-driven / oracle-gated workflow already followed (acceptance-criteria-as-oracle, one PR per issue, rivet validate gate, artifact trailers).

🤖 Generated with Claude Code — issue-triage agent run 2026-05-12.


Generated by Claude Code

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 12, 2026

📐 Rivet artifact delta

No artifact changes in this PR. Code-only changes (renderer, CLI wiring, tests) don't touch the artifact graph.

Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rivet Criterion Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite Current: 5665cb5 Previous: 01c84a9 Ratio
traceability_matrix/1000 60232 ns/iter (± 303) 44286 ns/iter (± 617) 1.36

This comment was automatically generated by workflow using github-action-benchmark.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…V matrix

Implements sub-issue 3 of the V&V coverage matrix decomposition on #188:
the cross-repo aggregator. Rather than reaching into the GitHub org API,
each repo's CI emits `rivet coverage --matrix --format json` and a
top-level job merges the files with `rivet coverage --aggregate a.json
b.json ... --format {text,markdown,html,json}`.

- Duplicate `(repo, id)` rows are coalesced (first wins) so re-running
  the aggregator over overlapping inputs is idempotent.
- The merged JSON uses the same envelope as the per-repo command, so it
  re-feeds the aggregator unchanged.
- Bad / wrong-shaped inputs fail with a diagnostic that names the file.

3 integration tests: markdown merge + column union, JSON round-trip +
dedup, bad-input diagnostics. Existing `coverage --matrix` tests, clippy,
fmt, and `rivet validate` baseline all unchanged.

Implements: REQ-007
Refs: FEAT-001
@avrabe avrabe force-pushed the feat/issue-188-vv-matrix-aggregator branch from d311e5c to 5665cb5 Compare May 15, 2026 14:24
@avrabe avrabe marked this pull request as ready for review May 15, 2026 14:24
@avrabe avrabe merged commit 01c84a9 into main May 15, 2026
9 of 15 checks passed
@avrabe avrabe deleted the feat/issue-188-vv-matrix-aggregator branch May 15, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

V&V coverage matrix dashboard via rivet coverage (per-commit, per-repo, per-technique)

2 participants