feat(for-you): re-introduce /users/{id}/feed/for-you with lean 3-source pipeline#817
Open
dylanjeffers wants to merge 2 commits into
Open
feat(for-you): re-introduce /users/{id}/feed/for-you with lean 3-source pipeline#817dylanjeffers wants to merge 2 commits into
dylanjeffers wants to merge 2 commits into
Conversation
…uthors
The previous shadow-ban filter on the contest discovery list used
`aggregate_user.score < 0` (AAO output). Two problems:
1. `aggregate_user` has no index covering `score`, so the CTE forced a
full seq scan on every cold call. /v1/events/remix-contests?status=all
was hanging ~22s cold-cache (warm: ~100ms).
2. The AAO signal is a separate moderation lane from the community
karma-reports system that already governs comment visibility.
The two can drift.
Fix: align the contest filter with the comment-visibility filter. A host
is shadow-banned from contest discovery if they authored a comment that
crossed the same `high_karma_reporters` threshold (sum of reporters'
follower_count >= karmaCommentCountThreshold) that hides the comment
itself on v1_track_comments / v1_event_comments. The new CTE
`karma_reported_authors` lifts the comment-level signal up to user_id.
`muted_by_karma` is unchanged — still filters hosts muted by high-karma
users.
`comment_reports` is a small table indexed on `comment_id`, and the new
CTE only adds a hash-join on comments (PK lookup per hkr row), so the
cost is bounded by report volume rather than user-table size — no
sequential scan over millions of aggregate_user rows.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…urce pipeline Brings back the For You feed endpoint that was removed in #807. The killer in the original was the similar_artists CTE — a 1-hop saves-graph self-join that produced a 301M-row merge for power users. Dropped entirely; the lean pipeline keeps the other three sources: - in_network (followed-creator uploads, last 14 days, LIMIT 200) - trending (track_trending_scores week, LIMIT 100) - underground (week-trending, sub-1500 follower/following, LIMIT 50) Same ranking formula and same Go-side diversity pass. Perf guardrails from #805/#806 are retained (follow_set LIMIT 500, affinity sub-selects capped at 200/200/500), and my_artist_affinity additionally filters its tracks join to the last 90 days so old uploads can't pull the CTE wide. A partial index on track_trending_scores covering (type='TRACKS', version='pnagD', time_range='week', genre IS NULL) is added in 0198 — without it the trending/underground sources cost a fixed ~12s table scan per request regardless of caller. Auth middleware exemption for /feed/for-you (from #804) is re-added: the route's query user_id is a viewer hint for response decoration only; path :userId controls personalization. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings back
GET /v1/users/{id}/feed/for-you(removed in #807) with a leaner pipeline that drops the source that was causing the prod timeouts.The killer in the original was the
similar_artistsCTE — a 1-hop saves-graph self-join that produced a 301M-row merge for power users. Dropped entirely. The lean pipeline keeps the other three sources:in_networktrendingtrack_trending_scoresundergroundSame ranking formula as before:
Same Go-side diversity pass (per-artist
ROW_NUMBER()cap + 5-position lookahead to break consecutive-same-artist runs).Perf guardrails
Retained from #805 / #806:
follow_setcapped at 500 most-recently-followed.my_artist_affinitysub-selects capped (200 saves / 200 reposts / 500 plays, all by recency).Additional:
my_artist_affinityinnerJOIN tracksis now further restricted to tracks created in the last 90 days — old uploads can't pull the CTE wide.idx_track_trending_scores_for_youontrack_trending_scores (score DESC, track_id)covering theTRACKS / pnagD / week / null-genreslice. Without it, EXPLAIN showed a fixed ~12s scan oftrack_trending_scoresfor every request, regardless of caller.Files
api/v1_users_feed_for_you.goapi/v1_users_feed_for_you_test.goapi/server.goapi/auth_middleware.go/feed/for-youexemption (queryuser_idis advisory; path:userIdcontrols personalization)api/swagger/swagger-v1.yamlddl/migrations/0198_track_trending_scores_for_you_idx.sqlTest plan
go build ./api/...cleango vet ./api/...cleango test -c ./api/...compiles/v1/users/{id}/feed/for-you?user_id={id}&limit=5for a power-user account that previously timed out — should now return 200 in < 2s.🤖 Generated with Claude Code