Switch challenge reads to sol_reward_disbursements#809
Merged
Conversation
rickyrombo
force-pushed
the
mp/challenges-read-from-sol-indexer
branch
from
efdaa39 to
6bd6ee5
Compare
Routes that joined challenge_disbursements now read from a compatibility view v_challenge_disbursements over the new Solana indexer's sol_reward_disbursements table, with user_id resolved via recipient_eth_address -> users.wallet. Python continues to dual-write challenge_disbursements until the discovery-provider service is decommissioned in a future change. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
rickyrombo
force-pushed
the
mp/challenges-read-from-sol-indexer
branch
from
6bd6ee5 to
e060a83
Compare
…sements
Restores the indexes the legacy challenge_disbursements had on user_id
and created_at. v_challenge_disbursements inlines into the route SQL, so
filters like cd.user_id = X push down to users.wallet -> probe
sol_reward_disbursements.recipient_eth_address, which would otherwise
seq-scan. Same for the default created_at sort on
/v1/challenges/disbursements and the date-range filter in
/v1/challenges/{id}/info.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
raymondjacobson
approved these changes
May 15, 2026
6 tasks
rickyrombo
added a commit
that referenced
this pull request
May 15, 2026
## Summary Step 1 of the purchases-domain cutover. This PR populates the new Go-indexer tables with the historical data they're missing today, adds the compatibility view + parallel notification trigger, but **leaves all readers on the legacy `usdc_purchases` table**. The route swap is a separate PR (step 2) that lands after this one is verified on production. The shape mirrors PR #809 (challenges cutover): bounded migration, view-based read translation, parallel trigger that dedupes via shared `group_id`. ## What's in this PR **Schema / migration** — `ddl/migrations/0199_backfill_sol_purchases.sql` - Adds `created_at TIMESTAMP DEFAULT NOW()` to `sol_purchases`. Same gap we hit on `sol_reward_disbursements` in #809. - Copies historical purchases from `usdc_purchases` into `sol_purchases`. `from_account` is resolved to the buyer's USDC user_bank via `usdc_user_bank_accounts` so the NOT NULL column has a real value. - Patches `created_at` on rows the Go indexer wrote before this migration (their `created_at` was just `NOW()` from the default; corrects them from the legacy table where the legacy value is older). - Explodes `usdc_purchases.splits` JSONB into one `sol_payments` row per element. Element shape is `{payout_wallet, amount, percentage, user_id, eth_wallet}` per `add_wallet_info_to_splits()` in the Python source. - Adds `sol_purchases_created_at_idx` so the route-side default sort by `created_at` doesn't degrade. **View** — `ddl/views/v_usdc_purchases.sql` - Exposes `sol_purchases` + `sol_payments` in the legacy column shape so step 2's route swap is mostly a one-token rename. - `seller_user_id` is derived from current content ownership (`tracks.owner_id` / `playlists.playlist_owner_id`). Note: this is current owner, not snapshotted at purchase time. Legacy was a snapshot — accepting this drift per design discussion. - `extra_amount` is derived as `amount - base_price` via a correlated subquery against `track_price_history` / `album_price_history` (block_timestamp <= purchase created_at, ORDER BY DESC LIMIT 1). - `splits` JSON is aggregated over `sol_payments` with user_id resolved via `COALESCE(users.spl_usdc_payout_wallet match, sol_claimable_accounts mint=USDC match)`. Network-cut payments (to the staking bridge wallet) emit `user_id: null`. - `vendor` is intentionally dropped from the view. - Filtered to `is_valid IS TRUE` to match the legacy table's semantics (Python only wrote validated purchases). **Trigger** — `ddl/functions/handle_usdc_purchase.sql` - Appends a `handle_sol_purchase` function and `on_sol_purchase AFTER INSERT ON sol_purchases` trigger. - Notification shape and `group_id` format match the legacy trigger byte-for-byte (verified against the existing function body), so during the backfill — where every inserted row fires the new trigger and tries to recreate notifications whose `group_id`s were created by the legacy trigger long ago — `ON CONFLICT DO NOTHING` makes them no-ops. - `vendor` and `extra_amount` are emitted as `null` in the new payload; downstream consumers must tolerate this. **Cleanup** — `sql/01_schema.sql` - Dropped a stale `block_timestamp` column from the `sol_purchases` table definition. No migration creates it and nothing in the repo references it; the dump had drifted from reality. ## What's NOT in this PR - **Reader changes.** All 14+ Go routes that join `usdc_purchases` (`v1_users_purchases`, `v1_users_sales`, `v1_users_purchasers`, `v1_explore_best_selling`, `v1_users_library_*`, `v1_fan_club_feed`, `comms_blasts`, `dbv1/access.go`, `comms/chat.go`, etc.) are unchanged. Until this PR's backfill is verified in production, swapping readers would risk old purchases disappearing. - **Python decommission.** `index_payment_router` keeps writing `usdc_purchases` (legacy trigger keeps firing on insert). The new trigger dedupes against it via `group_id`. - **Go indexer update for `created_at`.** A small follow-up: `solana/indexer/program/payment_router.go` should explicitly write `created_at` so new rows get the on-chain time rather than `NOW()` from the column default. Default is correct-enough until then. - **Vendor field.** Lost in the view; if anything frontend-side breaks on `vendor: null` notifications we'll need a workaround. ## Test plan After this lands, before opening the step-2 PR, verify on a prod replica: - [ ] Row-count parity: ```sql SELECT (SELECT count(*) FROM usdc_purchases) AS legacy, (SELECT count(*) FROM sol_purchases WHERE is_valid IS TRUE) AS new_valid; ``` - [ ] Splits parity for a 50-row sample: ```sql SELECT up.signature, jsonb_array_length(up.splits) AS legacy_splits, (SELECT count(*) FROM sol_payments WHERE signature = up.signature AND instruction_index = 0) AS new_splits FROM usdc_purchases up ORDER BY random() LIMIT 50; ``` - [ ] Spot-check `v_usdc_purchases` against `usdc_purchases` for a few signatures: same `buyer_user_id`, same `amount`, comparable `splits[*].user_id` and `payout_wallet`. - [ ] Confirm trigger dedupe: insert a `sol_purchases` row matching an existing `usdc_purchases` row in dev; assert no new notification row appears. - [ ] Cloud SQL logs during deploy: no `statement_timeout`, no `pg_type_typname_nsp_index`, no `deadlock detected`. - [ ] `go test ./api/...` green (no reader changes, no test changes expected). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Routes that joined
challenge_disbursements(Python-indexer-written) now read from a new compatibility viewv_challenge_disbursementsbacked by the Go indexer'ssol_reward_disbursementstable. Notifications continue to be created via a new trigger on the sol_* table. Python keeps dual-writing the legacy table for now.This is a deliberate wedge: only the challenges read path is moved. Other Solana-derived reads (transactions history, tips, purchases, withdrawals) are unchanged and stay on the legacy Python-written tables.
In scope (this PR)
Schema
ddl/migrations/0198_v_challenge_disbursements.sql:created_at TIMESTAMP DEFAULT NOW()tosol_reward_disbursements(the column did not exist; backfilled fromchallenge_disbursements.created_atfor matching signatures).v_challenge_disbursements (challenge_id, specifier, amount, signature, slot, created_at, user_id)oversol_reward_disbursements JOIN users ON users.wallet = recipient_eth_address.sql/01_schema.sqlupdated so sqlc parses the new column + view.Trigger
ddl/functions/handle_challenge_disbursements.sql: addshandle_sol_reward_disbursementtrigger onsol_reward_disbursementsthat mirrors the legacy notification-creation logic. Dedupes via the samegroup_id, so it dual-fires safely with the existing trigger during cutover. Also emitspg_notify('challenge_disbursed', ...)for future consumers (no subscriber wired yet).Go readers — now point at the view
api/v1_challenges_disbursements.goapi/v1_challenges_undisbursed.goapi/v1_users_challenges.goapi/v1_challenges_info.goapi/v1_coins_post_redeem.goapi/dbv1/queries/get_undisbursed_challenges.sql(+ regeneratedget_undisbursed_challenges.sql.goandmodels.goviasqlc generate)Test fixtures
database/seed.go— addedsol_reward_disbursementsbase row.api/v1_challenges_disbursements_test.go,api/v1_challenges_info_test.go— switched to seedingusers+sol_reward_disbursementswithrecipient_eth_addressmatchingusers.walletso the view resolvesuser_id.Not in scope / still to be done
These were considered during planning but explicitly deferred:
index_rewards_managercontinues to poll the Reward Manager program and writechallenge_disbursements,reward_manager_txs, andaudio_transactions_historyrewards rows. The legacy trigger onchallenge_disbursementsstays active; the new trigger's dedupe handles overlap. To be addressed when the discovery-provider service is dropped entirely./v1/challenges/{id}/attestport. Still served from Python. Anti-abuse oracle attestation signing needs to be ported to Go; not done in this PR per scope discussion.pg_notify('challenge_disbursed', ...)consumer. Trigger emits the event but nothing subscribes. Reserved for the future Python decommission (or a Go-side challenge-event-bus port).audio_transactions_historyrewards rows (USER_REWARD / TRENDING_REWARD). Reads of audio-transactions history still go through the legacy table; rewards typing in a future typed-history view is a separate workstream.user_balancesrefresh.index_rewards_managercallsenqueue_immediate_balance_refreshon disbursement. Untiluser_balancesreaders are audited and migrated tosol_user_balances, the Python refresh hook keeps running.user_tips,aggregate_user_tips), purchases (usdc_purchases), transactions history (audio + usdc), withdrawals, and Stripe/Coinbase/Coinflow top-up vendor detection are all untouched.Test plan
go build ./...clean (verified locally).go test ./api/ -run 'TestGetChallengeDisbursements|TestV1ChallengesInfo'against a running test postgres on:21300.sol_reward_disbursementsproduces anotificationrow withtype=challenge_rewardand the expectedgroup_id, and that a duplicate insert is a no-op.SELECT COUNT(*) FROM v_challenge_disbursementsis consistent withSELECT COUNT(*) FROM challenge_disbursementsmodulo rows whoserecipient_eth_addressdoesn't map to a current user./v1/challenges/disbursements,/v1/users/{id}/challenges,/v1/challenges/undisbursed,/v1/challenges/{id}/infoagainst a populated DB./v1/coins/{mint}/redeemidempotency check still rejects a second redemption for the same code+specifier.🤖 Generated with Claude Code