Problems to calculate FDP to the peptide level in a benchmarking study with different DL models in FragPipe.

[adrianmarman]

DBGear.java

Hello,

First of all, thank you very much for developing FDRBench. I have been testing it for entrapment-based FDP estimation in semi-enzymatic proteomics workflows and I would like to ask for your advice regarding a potential issue I encountered.

I modified the FDRBench source code to support a custom Tryp-N/LysN-like digestion strategy for semi-enzymatic searches. I am attaching:

the modified source code (DB.Gear file with Tryp-N),
the protein-level entrapment FASTA used for MSFragger searches,
the commands used for:
protein-level entrapment FASTA generation,
peptide entrapment mapping (.txt) generation,
paired FDP calculation.

The workflow was:

Generate paired entrapment proteins using FDRBench.
Use the generated protein FASTA for MSFragger searches.
Generate the peptide-level entrapment mapping file (.txt).
Run paired FDP estimation at peptide/precursor level.

The commands used were:

Protein-level entrapment FASTA generation:

java -jar fdrbench-0.0.4.jar
-level protein
-db 2026-05-04-reviewed-contam-UP000005640-spikein.fas
-o UP000005640_entrapment_paired.fasta
-a
-fix_nc c
-enzyme 9
-miss_c 2
-minLength 7
-maxLength 50
-check
-ns
-uniprot
-I2L
-export_db
-decoy

Peptide entrapment mapping generation:

java -jar fdrbench-0.0.4.jar
-I2L
-level peptide
-db UP000005640_entrapment_cleaned.fasta
-o UP000005640_entrapment_pep.txt
-uniprot
-fix_nc c
-enzyme 9
-miss_c 2
-minLength 7
-maxLength 50

Paired FDP calculation:

java -jar fdrbench-0.0.4.jar
-i FDRBench_exports/WF_C_1A.tsv.remove_invalid_peptides
-fold 1
-pep UP000005640_entrapment_pep.txt
-level peptide
-o WF_C_1A_fdp.csv
-score "score:1"

The searches themselves complete correctly and I obtain target, decoy, and entrapment identifications. However, during FDP calculation I repeatedly observe messages such as:

The number of entrapment hits is larger than k: 3 > k=1
Target: RVVVVDC
Entrapment: RVDVVVC

The process finishes without an explicit Java error, but no final FDP output (.csv/.tsv) is generated. Instead, an intermediate file such as:

WF_C_1A.tsv.remove_invalid_peptides

is created, and I am not entirely sure whether this indicates:

an issue in the paired peptide mapping logic,
instability of paired FDP estimation in highly redundant semi-enzymatic peptide spaces,
or a problem caused by my custom enzyme implementation.

My main question is:

Is paired peptide/precursor-level FDP estimation expected to become problematic for semi/non-specific searches with large redundant peptide spaces, even if the entrapment FASTA generation itself is correct?

Thank you very much for your time and for the excellent tool.

[wenbostar]

Hi @adrianmarman,

Thanks for the detailed write-up. The failure you're seeing isn't directly caused by your custom Tryp-N/LysN digestion — it's
a method limitation of the peptide-level paired FDP estimation when combined with a protein-level entrapment
database.

Why it doesn't work:

The paired method requires a 1:1 correspondence between each target peptide and a specific entrapment counterpart so
the two can be compared directly. That correspondence only exists when the entrapment database is built at the
peptide level. When the entrapment FASTA is generated at the protein level and then digested in the search engine,
each entrapment protein produces many peptides whose pairing back to target peptides is ambiguous — and a
peptide-level entrapment database is difficult to be reconstructed accurately from a protein-level one after the fact for the purpose
of detecting the pairing.

If you want to keep using the paired method at the peptide level, regenerate the entrapment database at the
peptide level rather than the protein level, then run the search and FDP estimation against that.

If you need to keep the protein-level entrapment FASTA (e.g., because you also want protein-level FDP), use one
of the non-paired peptide-level FDP methods instead — those don't depend on target↔entrapment pairing and work with
a protein-level entrapment DB.

Hope that helps — happy to clarify further if you'd like.

[adrianmarman]

Hi again, and thanks for the clarification.

I think I now better understand the issue with using a protein-level entrapment FASTA together with peptide/precursor-level paired FDP estimation.

However, I would like to ask whether the following workaround could still be considered a valid approximation for paired peptide-level FDP evaluation.

What I did was:

I generated the entrapment FASTA at the protein level using:
-level protein
-decoy
-fix_nc c
same enzyme settings as MSFragger
same missed cleavages
same peptide length constraints
I then used this FASTA directly in MSFragger for the database search.
After the search, I generated a new .pep.txt mapping file from the SAME entrapment FASTA used in the search, but:
removing decoys first
keeping only target and paired entrapment sequences
using exactly the same digestion parameters as MSFragger:
same enzyme
same missed cleavages
same peptide length range
same I2L settings
same cleavage behavior

So in practice, the peptide mapping file is reconstructed directly from the exact FASTA used during the search and with matching digestion settings.

My question is:

Would this still be considered invalid for paired peptide/precursor FDP estimation, or could it be considered an acceptable approximation since the peptide-level mapping is regenerated from the exact searched FASTA under matching digestion constraints?

I am attaching:

the FASTA used for MSFragger
the commands used
screenshots of the digestion settings
the exported TSV
the resulting FDPBench CSV output
terminal logs during processing

I would really appreciate your opinion on whether this approach is still fundamentally incompatible with the paired method, or whether it could be acceptable in practice.

Thanks again for your help.

[adrianmarman]

Thank you @wenbostar and sorry but GitHub doesn´t let me send you the files.

[wenbostar]

So in practice, the peptide mapping file is reconstructed directly from the exact FASTA used during the search and with matching digestion settings.

I don't know this could exactly construct the peptide pairing. A better way seems to be just to generate a peptide level entrapment database and perform msfragger search against the peptide level database without enzyme digestion (this is doable).

[wenbostar]

For your protein level entrapment database generation, you may want to set -fix_nc n when using Tryp-N. When using a protein level entrapment database, you can still use the combined and the lower bound FDP estimation methods for FDR control evaluation.

[adrianmarman]

Thank you again, this clarifies a lot.

I think I will follow the workflow closer to the paired peptide-level strategy described in the paper. Right now I am generating a peptide-level entrapment FASTA with FDRBench and then using that FASTA directly in FragPipe/MSFragger without allowing MSFragger to perform its own digestion, so the peptide pairing generated by FDRBench is preserved.

I am attaching both the current FragPipe/MSFragger GUI configuration and the command I am currently using to generate the digested entrapment FASTA:

java -jar fdrbench-0.0.4.jar
-level peptide
-db 2026-05-04-reviewed-contam-UP000005640-spikein.fas
-o UP000005640_entrapment_paired.fasta
-a
-fix_nc n
-enzyme 9
-miss_c 2
-minLength 7
-maxLength 50
-check
-ns
-uniprot
-I2L
-export_db
-decoy

I used these parameters because they match as closely as possible the digestion/search settings I originally used in MSFragger to obtain my current N-terminomics results, so I assume the resulting peptide space should remain very similar.

Right now I set the enzyme mode in MSFragger to "nonspecific" because my understanding is that MSFragger should not perform an additional digestion step if I want to preserve the peptide-level target/p-target pairing generated by FDRBench. However, I am still not completely sure this is the correct way to reproduce the paired workflow in practice.

For a bit more context about the project: I fine-tuned AlphaPeptDeep-based models to improve PSM rescoring for N-terminomics datasets (Tryp-N datasets specifically), and now I want to evaluate the reliability/calibration of the resulting identifications using your FDP/entrapment assessment strategy.

That is why I am trying to preserve the peptide-level pairing structure as faithfully as possible instead of using a standard protein-level entrapment workflow.

Thank you very much again for all the help.

*Enzyme 9 == Tryp-N. If you want, I can send you the DBGear.java with this customizated Tryp-N.

[adrianmarman]

*Sorry the -o the extension is with .txt (Not .FASTA)
in order to do the FDP calculation after of the seraching with FragPipe.

[wenbostar]

Does your modified code do semi-digestion with enzyme Tryp-N?

[adrianmarman]

No, only I´ve added the Tryp-N rules because our project is a fully enzymatic Tryp-N/LysN-like digestion.

[wenbostar]

First of all, thank you very much for developing FDRBench. I have been testing it for entrapment-based FDP estimation in semi-enzymatic proteomics workflows and I would like to ask for your advice regarding a potential issue I encountered.

I modified the FDRBench source code to support a custom Tryp-N/LysN-like digestion strategy for semi-enzymatic searches. I am attaching:

I'm a bit confused. From your first message, you mentioned it is a semi-enzymatic search.

[adrianmarman]

No no sorry I was wrong, It´s a fully enzymatic digestion with Tryp-N.

[wenbostar]

@fcyu , could you send a screenshot here about how to use a peptide level fasta file for search in fragpipe?

[fcyu]

Sure

Best,

Fengchao

[wenbostar]

Hi @adrianmarman , we have a GUI for FDRBench to perform both entrapment database generation and FDP estimation. Please let me know if you want to try the GUI for your analysis. I can add Tryp-N to the enzyme list.