feat(host-reth): replace ExEx backfill with direct DB reads#136
Open
feat(host-reth): replace ExEx backfill with direct DB reads#136
Conversation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduces three types for reading host-chain blocks from the reth DB: - `DbBlock`: owned block+receipts pair from the provider - `DbChainSegment`: newtype over `Vec<DbBlock>` implementing `Extractable` using the same `RecoveredBlockShim` transmute pattern as `RethChain` - `DbBackfill<P>`: batch reader that walks from a cursor to the finalized block, recording metrics per batch via `crate::metrics` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds `HostChain` enum with `Backfill(DbChainSegment)` and `Live(RethChain)` variants, both delegating to the inner `Extractable` impl. Promotes `DbChainSegment` to `pub` and re-exports both new types from the crate root. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace reth's built-in ExEx backfill with DbBackfill for startup catch-up. The notifier now runs a two-phase loop: phase 1 drains DB batches via DbBackfill, then phase 2 switches to live ExEx notifications. set_head initializes backfill instead of resolving a header directly, and set_backfill_thresholds configures DbBackfill batch size. Chain type changes from RethChain to HostChain enum. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
DbBackfill<P>reads blocks+receipts in batches up to finalized, then hands off to the ExEx stream for live blocksHostChainenum unifies backfill and live chain segments behindExtractableset_headdocumented and enforced as once-only across both notifier implementationsDetails
The ExEx backfill mechanism re-executes historical blocks on startup, which is slow and has caused memory issues. The reth DB already contains executed results, making re-execution unnecessary.
New types (
signet-host-reth):DbBlock/DbChainSegment— owned block+receipts from DB, implementsExtractableDbBackfill<P>— batch reader usingspawn_blockingfor MDBX readsHostChain— enum wrappingDbChainSegment(backfill) andRethChain(live)Notifier changes:
RethHostNotifier::next_notificationis now two-phase: drain DB backfill, then switch to ExExset_headcreates aDbBackfillinstead of calling ExExset_with_headreth-stages-typesdependency removedCross-crate:
HostNotifier::set_headtrait doc clarified as once-onlyRpcHostNotifier::set_headguards against repeated callsCloses ENG-1784
Test plan
cargo clippypasses (both--all-featuresand--no-default-features)RUSTDOCFLAGS="-D warnings" cargo docpassessignet-host-reth,signet-node-types,signet-host-rpc)signet-nodecompiles cleanly with newHostChaintype🤖 Generated with Claude Code