Skip to content

Backfill sol_purchases from usdc_purchases#815

Merged
rickyrombo merged 1 commit into
mainfrom
mp/purchases-backfill-sol-purchases
May 15, 2026
Merged

Backfill sol_purchases from usdc_purchases#815
rickyrombo merged 1 commit into
mainfrom
mp/purchases-backfill-sol-purchases

Conversation

@rickyrombo
Copy link
Copy Markdown
Contributor

Summary

Step 1 of the purchases-domain cutover. This PR populates the new Go-indexer tables with the historical data they're missing today, adds the compatibility view + parallel notification trigger, but leaves all readers on the legacy usdc_purchases table. The route swap is a separate PR (step 2) that lands after this one is verified on production.

The shape mirrors PR #809 (challenges cutover): bounded migration, view-based read translation, parallel trigger that dedupes via shared group_id.

What's in this PR

Schema / migrationddl/migrations/0199_backfill_sol_purchases.sql

  • Adds created_at TIMESTAMP DEFAULT NOW() to sol_purchases. Same gap we hit on sol_reward_disbursements in Switch challenge reads to sol_reward_disbursements #809.
  • Copies historical purchases from usdc_purchases into sol_purchases. from_account is resolved to the buyer's USDC user_bank via usdc_user_bank_accounts so the NOT NULL column has a real value.
  • Patches created_at on rows the Go indexer wrote before this migration (their created_at was just NOW() from the default; corrects them from the legacy table where the legacy value is older).
  • Explodes usdc_purchases.splits JSONB into one sol_payments row per element. Element shape is {payout_wallet, amount, percentage, user_id, eth_wallet} per add_wallet_info_to_splits() in the Python source.
  • Adds sol_purchases_created_at_idx so the route-side default sort by created_at doesn't degrade.

Viewddl/views/v_usdc_purchases.sql

  • Exposes sol_purchases + sol_payments in the legacy column shape so step 2's route swap is mostly a one-token rename.
  • seller_user_id is derived from current content ownership (tracks.owner_id / playlists.playlist_owner_id). Note: this is current owner, not snapshotted at purchase time. Legacy was a snapshot — accepting this drift per design discussion.
  • extra_amount is derived as amount - base_price via a correlated subquery against track_price_history / album_price_history (block_timestamp <= purchase created_at, ORDER BY DESC LIMIT 1).
  • splits JSON is aggregated over sol_payments with user_id resolved via COALESCE(users.spl_usdc_payout_wallet match, sol_claimable_accounts mint=USDC match). Network-cut payments (to the staking bridge wallet) emit user_id: null.
  • vendor is intentionally dropped from the view.
  • Filtered to is_valid IS TRUE to match the legacy table's semantics (Python only wrote validated purchases).

Triggerddl/functions/handle_usdc_purchase.sql

  • Appends a handle_sol_purchase function and on_sol_purchase AFTER INSERT ON sol_purchases trigger.
  • Notification shape and group_id format match the legacy trigger byte-for-byte (verified against the existing function body), so during the backfill — where every inserted row fires the new trigger and tries to recreate notifications whose group_ids were created by the legacy trigger long ago — ON CONFLICT DO NOTHING makes them no-ops.
  • vendor and extra_amount are emitted as null in the new payload; downstream consumers must tolerate this.

Cleanupsql/01_schema.sql

  • Dropped a stale block_timestamp column from the sol_purchases table definition. No migration creates it and nothing in the repo references it; the dump had drifted from reality.

What's NOT in this PR

  • Reader changes. All 14+ Go routes that join usdc_purchases (v1_users_purchases, v1_users_sales, v1_users_purchasers, v1_explore_best_selling, v1_users_library_*, v1_fan_club_feed, comms_blasts, dbv1/access.go, comms/chat.go, etc.) are unchanged. Until this PR's backfill is verified in production, swapping readers would risk old purchases disappearing.
  • Python decommission. index_payment_router keeps writing usdc_purchases (legacy trigger keeps firing on insert). The new trigger dedupes against it via group_id.
  • Go indexer update for created_at. A small follow-up: solana/indexer/program/payment_router.go should explicitly write created_at so new rows get the on-chain time rather than NOW() from the column default. Default is correct-enough until then.
  • Vendor field. Lost in the view; if anything frontend-side breaks on vendor: null notifications we'll need a workaround.

Test plan

After this lands, before opening the step-2 PR, verify on a prod replica:

  • Row-count parity:
    SELECT (SELECT count(*) FROM usdc_purchases) AS legacy,
           (SELECT count(*) FROM sol_purchases WHERE is_valid IS TRUE) AS new_valid;
  • Splits parity for a 50-row sample:
    SELECT up.signature,
           jsonb_array_length(up.splits) AS legacy_splits,
           (SELECT count(*) FROM sol_payments WHERE signature = up.signature AND instruction_index = 0) AS new_splits
      FROM usdc_purchases up ORDER BY random() LIMIT 50;
  • Spot-check v_usdc_purchases against usdc_purchases for a few signatures: same buyer_user_id, same amount, comparable splits[*].user_id and payout_wallet.
  • Confirm trigger dedupe: insert a sol_purchases row matching an existing usdc_purchases row in dev; assert no new notification row appears.
  • Cloud SQL logs during deploy: no statement_timeout, no pg_type_typname_nsp_index, no deadlock detected.
  • go test ./api/... green (no reader changes, no test changes expected).

🤖 Generated with Claude Code

Step 1 of the purchases-domain cutover from Python-indexer tables to the
Go indexer's sol_* tables. Adds a created_at column to sol_purchases
(parity with the legacy table), backfills historical purchases + their
splits from usdc_purchases into sol_purchases and sol_payments, adds the
v_usdc_purchases compatibility view, and mirrors the notification
trigger onto sol_purchases. Readers stay on the legacy table for now;
the route swap is a follow-up PR.

Also drops a stale block_timestamp column from the sol_purchases entry
in sql/01_schema.sql that no migration creates and nothing references.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Comment thread sql/01_schema.sql
city character varying,
region character varying,
country character varying,
block_timestamp timestamp with time zone
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting is this not really important for ordering?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't exist in production

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol

@rickyrombo rickyrombo merged commit 1cbd4a6 into main May 15, 2026
5 checks passed
@rickyrombo rickyrombo deleted the mp/purchases-backfill-sol-purchases branch May 15, 2026 23:01
rickyrombo added a commit that referenced this pull request May 15, 2026
Step 2 of the purchases-domain cutover. Now that #815 (step 1) has
backfilled sol_purchases and added the compatibility view, all ~17 Go
API routes that joined usdc_purchases swap over to v_usdc_purchases.

Code changes are minimal — table-name renames in route SQL. The view
absorbs the schema differences (sol_purchases + sol_payments + users +
tracks/playlists -> legacy column shape).

Test fixtures are rewritten: callers seed sol_purchases + sol_payments
instead of usdc_purchases. Drops fixture columns the view derives
(seller_user_id, extra_amount, splits). For tests that assert on
splits user_id, the seller's spl_usdc_payout_wallet is set so the
view's lookup resolves. For tests that assert on non-zero
extra_amount, a track_price_history row is seeded so the view's
amount - base_price computation produces the expected value.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants