Skip to content

Add analyze subset command#1301

Open
gayanW wants to merge 1 commit into
mainfrom
AIENG-367-1
Open

Add analyze subset command#1301
gayanW wants to merge 1 commit into
mainfrom
AIENG-367-1

Conversation

@gayanW
Copy link
Copy Markdown
Contributor

@gayanW gayanW commented May 15, 2026

Add new command analyze subset which can be used to demonstrate PTS effectiveness.

Usage:

$ smart-tests analyze subset --help
Usage: smart-tests analyze subset [OPTIONS] <SUBSET_ID>

Arguments:
  SUBSET_ID
      Subset ID to analyze

Options:
  --baseline-subset-id INT
      Baseline subset ID to compare against (auto-detected if not provided) [optional]

Example:

$ smart-tests analyze subset 2019138
→ Fetching subset data...
✓ Found 67 tests in subset

──────────────────────────────────────────────────────────────────────
Select Expected Tests
──────────────────────────────────────────────────────────────────────

Code files modified:
  - smart_tests/commands/inspect/model.py

Which tests do you expect PTS to prioritize?

? Select expected tests? Yes

Use Tab/Space to toggle selection, Enter when done

? Search and select expected tests: ['file=tests/commands/inspect/test_model.py']

✓ Selected 1 expected test(s):
  • file=tests/commands/inspect/test_model.py

──────────────────────────────────────────────────────────────────────
Step 2: Comparison Results
──────────────────────────────────────────────────────────────────────

Δ Rank      Subset Rank  Test Name                                  Reason                                                 Density
--------  -------------  -----------------------------------------  ---------------------------------------------------  ---------
↑42                   1  file=tests/commands/inspect/test_model.py  Changed file: smart_tests/commands/inspect/model.py   0.840091


✅ Tests Promoted:
- test_model.py promoted by 42 positions to rank #1

✅ Tests Prioritized:
- Expected test prioritized at rank #1 out of 67 (top 1.5%)

✅ Test Selection:
- PTS successfully picked expected tests with high confidence
- test_model.py: density 0.84 (very strong confidence)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant