Skip to content

[Quality-547] Update bert_tiny model to the retrained classifier#10529

Open
evelyn-with-warp wants to merge 1 commit intomasterfrom
evelyn/quality-543
Open

[Quality-547] Update bert_tiny model to the retrained classifier#10529
evelyn-with-warp wants to merge 1 commit intomasterfrom
evelyn/quality-543

Conversation

@evelyn-with-warp
Copy link
Copy Markdown
Contributor

@evelyn-with-warp evelyn-with-warp commented May 8, 2026

Description

Simply use retrained classifier to replace the previous one
The new classifier has the same same size as previous one, just further finetuned on prod data with user quickback as misfire groundtruth label
It could resolve the misfires reported in quality-547; For quality 543 on file paths misfires, some are still pending to resolve by improving heuristics.

Linked Issue

  • The linked issue is labeled ready-to-spec or ready-to-implement.
  • Where appropriate, screenshots or a short video of the implementation are included below (especially for user-visible or UI changes).

Testing

./script/presubmit
  • I have manually tested my changes locally with ./script/run

Screenshots / Videos

https://www.loom.com/share/94f99078f8614b2bbc05eab7f8d1a7d1
in this video we tested on the misfires listed in quality-547

  • left: warp dev
  • right: warp local build on this change

Agent Mode

  • Warp Agent Mode - This PR was created via Warp's AI Agent Mode

Reviewers:

@szgupta @vorporeal

@cla-bot cla-bot Bot added the cla-signed label May 8, 2026
@oz-for-oss
Copy link
Copy Markdown
Contributor

oz-for-oss Bot commented May 8, 2026

@evelyn-with-warp

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

Copy link
Copy Markdown
Contributor

@oz-for-oss oz-for-oss Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR swaps the bert_tiny.onnx model with a retrained version (bert_tiny_2026May.onnx) to reduce input-classifier misfires reported in Quality-547. The change renames the binary model file and updates the single path reference in Model::model_path(). The model is embedded at compile time via RustEmbed, and no other code references the old filename.

Concerns

  • The model file rename bakes the training date into the filename (bert_tiny_2026May.onnx). This is fine as a one-off, but if models are swapped regularly a versioning scheme (e.g. a monotonic version number or hash suffix) would be easier to maintain. This is non-blocking.
  • The PR description sentence "Simply use retrained classifier to replace the previously" appears truncated. Minor, but worth fixing for posterity.

Security

No security concerns. The change swaps a binary model asset and updates a static path literal. No new inputs, dependencies, secrets, or network surfaces are introduced.

Verdict

Found: 0 critical, 0 important, 1 suggestions

Approve

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

fn model_path(&self) -> &'static str {
match self {
Model::BertTiny => "bert_tiny.onnx",
Model::BertTiny => "bert_tiny_2026May.onnx",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 [SUGGESTION] Consider a versioning scheme that doesn't encode calendar dates (e.g. bert_tiny_v2.onnx). Date-suffixed names become ambiguous when multiple retrains happen in the same month and make it harder to correlate with training metadata.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@szgupta @vorporeal is this a real concern (as we won't refresh this often)? if so, could you suggest proper model versioning strategy for client app?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant