SF-3777 Retrieve confidence data and calculate book level scores by pmachapman · Pull Request #3876 · sillsdev/web-xforge

pmachapman · 2026-05-12T19:50:28Z

This PR adds a card to the Serval Builds tab with Quality Estimation, allowing downloading of book and chapter data in a format very similar to SILNLP:

This change is

codecov · 2026-05-12T19:58:30Z

Codecov Report

❌ Patch coverage is 81.36201% with 52 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.16%. Comparing base (021000d) to head (d35360f).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...administration/build-confidences-export.service.ts	3.33%	29 Missing ⚠️
...p/serval-administration/serval-builds.component.ts	23.52%	12 Missing and 1 partial ⚠️
...c/app/serval-administration/base-export-service.ts	71.42%	0 Missing and 2 partials ⚠️
src/SIL.XForge.Scripture/Models/VerseRefData.cs	81.81%	1 Missing and 1 partial ⚠️
...SIL.XForge.Scripture/Services/MachineApiService.cs	98.42%	0 Missing and 2 partials ⚠️
...orge.Scripture/Controllers/MachineApiController.cs	95.83%	0 Missing and 1 partial ⚠️
...aAccess/SFDataAccessServiceCollectionExtensions.cs	50.00%	1 Missing ⚠️
src/SIL.XForge.Scripture/Models/DraftMetrics.cs	95.00%	1 Missing ⚠️
...XForge.Scripture/Services/PreTranslationService.cs	95.65%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3876      +/-   ##
==========================================
+ Coverage   81.05%   81.16%   +0.11%     
==========================================
  Files         630      635       +5     
  Lines       40690    40924     +234     
  Branches     6603     6633      +30     
==========================================
+ Hits        32983    33218     +235     
+ Misses       6673     6663      -10     
- Partials     1034     1043       +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions · 2026-05-13T04:17:40Z

📸 Screenshot diff deployed! (4 changes)

View the visual diff at: https://pr-3876--sf-screenshot-diffs.netlify.app

RaymondLuong3

It is exciting to be one step closer to introducing book and chapter level quality estimates to users!

@RaymondLuong3 reviewed 33 files and all commit messages, and made 7 comments.
Reviewable status: all files reviewed, 6 unresolved discussions (waiting on pmachapman).

src/SIL.XForge.Scripture/ClientApp/src/app/serval-administration/serval-builds.component.ts line 269 at r3 (raw file):

    } else {
      this.buildConfidencesExportService.exportCsv(rows, this.dateRange$.value);
    }

I think this should be exportRsv.

Code quote:

    } else {
      this.buildConfidencesExportService.exportCsv(rows, this.dateRange$.value);
    }

src/SIL.XForge.Scripture/ClientApp/src/app/serval-administration/serval-builds.component.ts line 278 at r3 (raw file):

    } else {
      this.buildConfidencesExportService.exportCsv(rows, this.dateRange$.value);
    }

I think this should be exportTsv

Code quote:

    } else {
      this.buildConfidencesExportService.exportCsv(rows, this.dateRange$.value);
    }

src/SIL.XForge.Scripture/ClientApp/src/app/translate/draft-generation/build-confidences/display-confidence.component.html line 11 at r2 (raw file):

    <div class="confidence red"><mat-icon>warning</mat-icon> Likely poor quality</div>
  }
}

Looks like this is likely to be shown to the user and needs to be localized. Oh yes, I see below you mention that this is only visible from the serval admin tab right now.

Code quote:

@switch (usabilityLabel) {
  @case ("Green") {
    <div class="confidence green"><mat-icon>check</mat-icon> Likely good quality</div>
  }
  @case ("Yellow") {
    <div class="confidence yellow"><mat-icon class="filled">circle</mat-icon> Likely moderate quality</div>
  }
  @case ("Red") {
    <div class="confidence red"><mat-icon>warning</mat-icon> Likely poor quality</div>
  }
}

src/SIL.XForge.Scripture/Services/MachineApiService.cs line 3100 at r3 (raw file):

                List<ScriptureSegmentUsability> _, // We do not require segment level confidence values
                List<ScriptureChapterUsability> usabilityChapters,
                List<ScriptureBookUsability> usabilityBooks

I know the name is already in Machine so I understand why the variables are named this way, but I have a hard time understanding the meaning of usabilityChapters and usabilityBooks. However, I'm not sure I have a good suggestion

Code quote:

                List<ScriptureSegmentUsability> _, // We do not require segment level confidence values
                List<ScriptureChapterUsability> usabilityChapters,
                List<ScriptureBookUsability> usabilityBooks

src/SIL.XForge.Scripture/Services/PreTranslationService.cs line 68 at r3 (raw file):

            // Get the references - old builds will only have Refs specified,
            // newer builds will have TargetRefs with Refs containing the same values
            List<string> references = [.. preTranslation.TargetRefs];

It seems strange that old builds use Refs, but this code is changing it from using TargetRefs to TargetRefs or Refs. This seems to me that we are introducing Refs as a new fallback. What am I missing?

Code quote:

            // Get the references - old builds will only have Refs specified,
            // newer builds will have TargetRefs with Refs containing the same values
            List<string> references = [.. preTranslation.TargetRefs];

test/SIL.XForge.Scripture.Tests/Services/MachineApiServiceTests.cs line 2335 at r3 (raw file):

        );

        using (Assert.EnterMultipleScope())

This is neat. I haven't encountered this before, but nice to see a way to identify multiple failures within a block.

Code quote:

       using (Assert.EnterMultipleScope())

pmachapman

@pmachapman made 5 comments and resolved 1 discussion.
Reviewable status: all files reviewed, 5 unresolved discussions (waiting on RaymondLuong3).

src/SIL.XForge.Scripture/ClientApp/src/app/serval-administration/serval-builds.component.ts line 269 at r3 (raw file):