fix(session-replay-browser): suppress IDB transaction-timeout log cascade (SR-4356)#1751
Closed
lewgordon-amplitude wants to merge 1 commit into
Conversation
…cade (SR-4356)
Customer page was seeing hundreds of repeated 'transaction timed out' warns
when IDB wedged: every fire-and-forget addEventToCurrentSequence opened its
own readwrite tx and armed its own 5s watchdog, so the 5s mark produced a
burst of identical warns (devtools collapsed 429x). A self-amplifying
loop made it worse — logger.warn went through rrweb-plugin-console-record,
becoming a new rrweb event that fed back into addEventToCurrentSequence.
Fix in three layered changes in packages/session-replay-browser/src/events:
1. Add a tripped flag to SessionReplayEventsIDBStore. Public methods
short-circuit without opening a tx once tripped. Kills new txs at
the source so no new watchdogs are armed.
2. Route all failure logs through a centralised logFailure helper
gated on !tripped. Only the first failure that flips tripped logs;
later in-flight watchdogs / tx.done rejections stay silent.
3. Swap-first in switchToMemoryStore: replace store synchronously
before the recovery drain so new captures land on memory immediately,
even if the drain stalls. The drain itself uses a new
drainForFallback method that bypasses tripped, preserving the
ability to recover sequences on a write-side wedge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
bugbot run |
Session Replay Browser E2E Results
Details
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 9e4ca24. Configure here.
Collaborator
Author
|
Closing as I'm transitioning to Statsig. @stevenchien-amplitude @jpollock-ampl @jxiwang — would any of you want to pick this up? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Amplitude Logger [Warn]: Failed to store session replay events in IndexedDB: transaction timed out(devtools collapsed 429×). Self-amplifying loop via rrweb-plugin-console-record + Sentry made it worse.packages/session-replay-browser/src/events:trippedflag onSessionReplayEventsIDBStore— public methods short-circuit without opening a tx once tripped, so no more watchdogs are armedlogFailurehelper gated on!tripped— only the first failure logs; in-flight watchdogs / tx.done rejections stay silentswitchToMemoryStore— replacestoresynchronously before the recovery drain. The drain itself uses a newdrainForFallbackthat bypasses tripped, preserving recovery on write-side wedgesLinear: SR-4356
Test plan
drainForFallbackbypasses tripped, swap-first ordering, drain rejection swallowede2e/idb.spec.tsverifies that under sustained IDB put failure with 40+ rapid events, exactly 1 storage-failure warn is emitted (pre-fix would be dozens)🤖 Generated with Claude Code
Note
Medium Risk
Modifies session-replay event persistence and mid-session fallback behavior under IndexedDB failure; bugs here could impact event durability or recovery ordering, but changes are gated to failure paths and are heavily test-covered.
Overview
Prevents session replay from spamming console warnings and piling up stalled transactions when IndexedDB becomes unhealthy (SR-4356).
SessionReplayEventsIDBStorenow trips a permanent failure flag that makes public read/write methods short-circuit (no new transactions/watchdogs) and centralizes storage-failure logging so only the first failure logs; a newdrainForFallback()bypasses the trip to allow a best-effort final read.switchToMemoryStoreinevents-manageris reordered to swap to the in-memory store first, then attempt the best-effort IDB drain (swallowing drain failures), ensuring new captures aren’t blocked by a wedged IDB handle; unit/integration/e2e tests are updated/added to lock in burst-suppression and swap-first behavior.Reviewed by Cursor Bugbot for commit 9e4ca24. Bugbot is set up for automated code reviews on this repo. Configure here.