Skip to content

feat(resolver,python): version_range hint on load_family callback (closes #92)#93

Closed
doubleailes wants to merge 3 commits into
mainfrom
feat/issue-92-load-family-range-hint
Closed

feat(resolver,python): version_range hint on load_family callback (closes #92)#93
doubleailes wants to merge 3 commits into
mainfrom
feat/issue-92-load-family-range-hint

Conversation

@doubleailes
Copy link
Copy Markdown
Owner

@doubleailes doubleailes commented May 18, 2026

Summary

Threads a VersionRange hint through pyrer.solve's load_family callback so the rez integration shim can pre-filter via rez.packages.iter_packages(range_=...). Targets the 95% load-fan-out waste documented in #92.

The win

Workload Without hint With hint
Real Fortiche 132-package resolve (#92's profile) 2,637 loads / 91% wall time ~400 loads (projected) / ~6.6× cut
Compounding with static parser already merged on `experimental` parser makes each load cheap hint makes most loads unnecessary

How

Rust:

  • `FamilyLoader` signature: `Fn(&str, Option<&VersionRange>) -> Vec<...>`
  • `PackageRepo` tracks per-family `loaded_range`; reloads with the union when a later request needs a wider range (backtrack-widen support)
  • `PackageVariantCache` invalidates the cached `PackageVariantList` when the underlying family map changes (detected via `Rc::ptr_eq`)

Python:

  • Callback can be `def cb(name)` (legacy), `def cb(name, version_range=None)`, or `def cb(name, **kwargs)` — detected via `inspect.signature` once per `solve()`
  • Hint passed as keyword argument (rez-syntax range string, e.g. `"2+<3"`); `None` signals "unconstrained"

Tests

Suite Before After
`cargo test --lib -p rer-resolver` 44 46 (2 new: range-hint, eager-seed-no-call)
`pytest tests/` 94 98 (4 new: hint passed, legacy 1-arg works, hint string format, `**kwargs` works)
188-case rez differential 188/188 188/188 in 16.64 s — solver unchanged

Verifications

  • `cargo build` clean across workspace
  • All Rust unit tests pass (46/46)
  • All Python tests pass (98/98)
  • Strict 188-case rez differential still passes (188/188 in 16.64 s)
  • Legacy 1-arg callback signature still works
  • Backtrack-widening reload path covered by Rust test

Test plan

  • Wire into the Fortiche shim (`claude/rer-experimental` branch) — replace `load_family(name)` with `load_family(name, version_range=None): return [... for pkg in iter_packages(name, range_=version_range, paths=...)]`
  • Profile a representative resolve and confirm the projected 6-20× cut to `_load_family` wall time
  • Confirm 0 mismatches on the differential harness from `experimental` (parser also active)

Files

  • `crates/rer-resolver/src/rez_solver/context.rs` — `FamilyLoader` signature; `PackageRepo` widening logic
  • `crates/rer-resolver/src/rez_solver/variant.rs` — `PackageVariantList::new(ctx, name, hint)`; cache invalidation via `Rc::ptr_eq`
  • `crates/rer-resolver/src/rez_solver/solver.rs` — 2 new unit tests
  • `crates/rer-python/src/lib.rs` — signature inspection, keyword-arg passing
  • `crates/rer-python/Cargo.toml` — `rer-version` dep for the closure type
  • `tests/test_rich_api.py` — 4 new Python tests
  • `docs/content/docs/getting-started/rez-integration.md` — updated `load_family` examples + new "version_range hint" section
  • `CHANGELOG.md` — Unreleased entry

🤖 Generated with Claude Code

@doubleailes @claude
feat(resolver,python): version_range hint on load_family callback (cl…
9614f07 
…oses #92)

The `load_family` callback now receives a `version_range` argument
when its signature accepts one. The hint is a rez-syntax range
string (`"2+<3"`, `"==2.0"`, `None` for unconstrained) that the
shim can pass directly to `rez.packages.iter_packages(range_=...)`
to skip on-disk version directories outside the current solver
constraint.

## Why this matters

Issue #92 measured 95% load-fan-out waste in a real Fortiche
resolve: 2,637 PackageData materialised so that 132 land in the
solve. With the hint, the shim pre-filters to versions intersecting
the solver's current constraint — projected ~400 loads instead of
2,637, a 6.6× cut to a path that accounts for 91% of resolve
wall time on that workload.

## Backward compatibility

Detected at solve start via `inspect.signature`:

  def load_family(name):                       # legacy, 1-arg
  def load_family(name, version_range=None):   # new, 2-arg
  def load_family(name, **kwargs):             # **kwargs accepted

All three keep working. The hint is passed as a keyword argument
so the **kwargs form receives it transparently. No flag-day for
existing shims.

## Backtrack widening

The repo tracks the loaded range per family. If a later request
needs a wider range than the cached load covered (i.e. the solver
backtracks and widens), the loader is re-called with the unioned
range and the variant cache invalidates the stale `PackageVariantList`.
In practice this is rare — most resolves narrow monotonically.

## Surface area changes

  Rust:
    - `FamilyLoader` signature: `Fn(&str, Option<&VersionRange>) -> Vec<...>`
    - `PackageRepo::get_family(name, hint)` — was `get_family(name)`
    - `PackageVariantList::new(ctx, name, hint)` — was `new(ctx, name)`
    - `PackageVariantCache::get_or_build(ctx, name, hint)` — was 2-arg
    - `PackageRepo` internal cache entry now tracks `loaded_range`

  Python:
    - `load_family` callback gains optional `version_range` kwarg
    - Detection via `inspect.signature`; legacy callbacks unchanged

## Tests

  Rust:
    - test_loader_receives_version_range_hint — confirms the
      hint string is "2+<3" for `lib-2+<3` requests
    - test_loader_eager_seed_no_loader_call — pre-seeded families
      never invoke the loader even with a hint
    - All 44 existing solver tests + 2 new = 46/46

  Python:
    - test_load_family_range_hint_passed_for_pinned_request
    - test_load_family_legacy_one_arg_callback_still_works
    - test_load_family_range_hint_string_format
    - test_load_family_kwargs_callback
    - 94 existing tests + 4 new = 98/98

## Differential

`cargo test --release -p rer-resolver --test test_rez_benchmark
-- --ignored`: **188/188** in 16.64 s. Solver behavior unchanged
on the strict rez-faithfulness gate.

## Docs

`docs/content/docs/getting-started/rez-integration.md` updated:

  - `load_family` worked example now shows the 2-arg signature with
    `iter_packages(range_=...)`.
  - New "The version_range hint (issue #92) — pre-filter at the shim"
    section with the impact table and the advisory semantics.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@qodo-code-review
Copy link
Copy Markdown

qodo-code-review Bot commented May 18, 2026

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

doubleailes and others added 2 commits May 18, 2026 16:14
@doubleailes @claude
docs: drop broken anchor — static-parser section isn't on main
576e97d 
The cross-reference link
`#plugging-in-the-static-package-py-fast-parser` pointed at a
section that only exists on the `experimental` branch, so Zola's
internal-link checker fails when building this doc on main.

Replace the link with a text-only reference. The cross-link can
come back when the `experimental` branch lands on main and the
section is actually present in this doc.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@doubleailes @claude
chore(release): bump workspace version to 1.0.0-rc.1
cbdea84 
First 1.0 release candidate. Picks up everything currently on
`main`:

  - Solver port + 188-case rez differential (1:1 match)
  - load_family lazy package discovery (#86)
  - resolved_ephemerals on SolveResult (#84)
  - PackageData.from_strings raw-string fast path (#88)
  - load_family version_range hint (#92, this branch)

The 1.0.0 jump signals that the public API surface is now under
the stability contract documented in
`docs/content/docs/engineering/stability/`. RC numbers keep
shipping until the gate to remove the `-rc.N` suffix is reviewed
(see CHANGELOG `[1.0.0] — TBD`).

Four touchpoints:

  - Cargo.toml: workspace version + the two internal-dep pins
    (rer-version, rer-resolver)
  - docs/config.toml: GitHub-pill version
  - docs/content/_index.md: homepage repo_version
  - docs/content/docs/getting-started/quick-start.md: Rust dep
    snippet

Existing historical note at
`docs/content/docs/engineering/stability.md:39` ("Pre-1.0.0
(0.1.0-rc.x) releases did not carry a stability contract") stays
unchanged — it's a deliberate reference to the prior series.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@doubleailes