Skip to content

fix(test-client-clis): detect EELS-style OutOfGasError in GasExhaustionTraceComparator#2879

Open
leolara wants to merge 1 commit into
ethereum:forks/amsterdamfrom
leolara:leolara/fix-oog-comparator-eels
Open

fix(test-client-clis): detect EELS-style OutOfGasError in GasExhaustionTraceComparator#2879
leolara wants to merge 1 commit into
ethereum:forks/amsterdamfrom
leolara:leolara/fix-oog-comparator-eels

Conversation

@leolara
Copy link
Copy Markdown
Member

@leolara leolara commented May 19, 2026

🗒️ Description

_is_out_of_gas_error in packages/testing/src/execution_testing/client_clis/trace_comparators.py used a substring check ("out of gas" in error.lower()) that matches geth-style trace error strings but misses the EELS EIP-3155 emitter's convention of writing the Python exception class name ("OutOfGasError") into the trace error field.

Because the substring check returned False for EELS-style errors, GasExhaustionTraceComparator silently classified every EELS trace as having no gas-exhaustion points. With EELS as the default t8n in this repo, both sides of a comparison return empty OOG sets and the comparator reports equivalent=True for trace pairs that may actually disagree on whether a frame OOG'd — the exact thing it is meant to catch.

The comparator is opt-in (TraceComparatorType.GAS_EXHAUSTION), so the bug is latent rather than active in the default fill flow, but it makes the comparator effectively inert for any opt-in caller running against EELS.

Fix: also accept the bare class-name form "OutOfGasError" (case-insensitive exact match) alongside the existing substring check, with a comment explaining why both conventions need to be recognized.

Tests: added coverage under TestGasExhaustionTraceComparator:

  • Parametrized positive test over "out of gas", "Out Of Gas", and "OutOfGasError".
  • Mixed-convention test (geth baseline vs. EELS current at the same line → equivalent), which is the exact mismatch this fix closes.
  • Two all-EELS tests covering same-line equivalence and different-line divergence.

🔗 Related Issues or PRs

N/A.

✅ Checklist

  • All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    just static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
Screenshot 2569-05-19 at 21 22 48

@codecov
Copy link
Copy Markdown

codecov Bot commented May 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.16%. Comparing base (852715c) to head (43f23cc).

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2879   +/-   ##
================================================
  Coverage            87.16%   87.16%           
================================================
  Files                  586      586           
  Lines                35792    35792           
  Branches              3364     3364           
================================================
  Hits                 31199    31199           
  Misses                3943     3943           
  Partials               650      650           
Flag Coverage Δ
unittests 87.16% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@leolara leolara marked this pull request as ready for review May 19, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants