feat(graphile-llm): wire billing metering into LLM plugins

[pyramation]

Summary

Wire billing metering infrastructure into the graphile-llm package with clean architectural separation between pure LLM plugins and opt-in billing.

Key design: meter slug = model name

Each LLM model gets its own billing meter. The meter slug IS the model name — no mapping, no guessing:

INSERT INTO meters (slug, display_name, meter_type, aggregation, credit_cost, category_meter)
VALUES
  ('inference',              'Inference',              'usage_pool', 'cumulative', 1, NULL),
  ('text-embedding-3-small', 'text-embedding-3-small', 'quota', 'cumulative', 1, 'inference'),
  ('gpt-4o-mini',           'gpt-4o-mini',            'quota', 'cumulative', 3, 'inference');

Three-level waterfall (handled by billing module's category_meter):

per-model meter → inference usage pool → universal credits

Architecture

Pure plugins (no billing dependency):

• LlmModulePlugin — resolves embedder + chat completer from config/env, exposes model names on build
• LlmTextSearchPlugin — adds text: String to VectorNearbyInput, embeds text server-side
• LlmTextMutationPlugin — adds {column}Text: String companion fields on mutation inputs

Opt-in metering (separate plugin):

• LlmMeteringPlugin — transparently wraps build.llmEmbedder with billing quota checks + usage recording via AsyncLocalStorage
• Reads model names from build (llmEmbeddingModel, llmChatModel) and uses them as default meter slugs
• Entity ID resolved via configurable resolveEntityId callback (default: jwt.claims.user_id)

Supporting utilities:

• config-cache.ts — LRU cache (5-min TTL, max 50) for billing_module metadata per database_id. Schema-existence guard checks information_schema.schemata before querying.
• metering.ts — meteredEmbed() / meteredChat() wrappers

Usage

// Without metering (pure LLM only)
GraphileLlmPreset({
  defaultEmbedder: { provider: 'openai', model: 'text-embedding-3-small' },
})

// With metering (meter slug = 'text-embedding-3-small')
GraphileLlmPreset({
  defaultEmbedder: { provider: 'openai', model: 'text-embedding-3-small' },
  metering: true,
})

// Bill per-database (platform tier)
GraphileLlmPreset({
  defaultEmbedder: { provider: 'openai', model: 'text-embedding-3-small' },
  metering: { resolveEntityId: (pg) => pg['jwt.claims.database_id'] },
})

Two-tier billing model

Documented in docs/spec/llm-metering.md:

• Platform tier: entity_id = database_id, Constructive bills the DB owner
• Tenant tier: entity_id = actor_id, DB owner tracks per-user usage (optional, only if billing_module provisioned)

Review & Testing Checklist for Human

• Verify the AsyncLocalStorage approach in metering-plugin.ts handles concurrent requests correctly
• Confirm information_schema.schemata guard in config-cache.ts works for databases without metaschema_modules_public
• Review resolveEntityId default (jwt.claims.user_id) — for platform-tier billing, you'd want jwt.claims.database_id instead
• Test with metering disabled (default) — should work identically to before
• Review docs/spec/llm-metering.md for accuracy of the two-tier billing model and waterfall design
• Verify per-model meters are seeded in the billing_module with category_meter = 'inference'

Recommended test plan: Set up a database with billing_module, seed model-name meters with category_meter = 'inference', configure metering: true, and run embedding queries. Verify record_usage entries appear in the ledger under the model name slug.

Notes

• The spec at docs/spec/llm-metering.md is the canonical reference for the metering architecture. It covers the two-tier model, meter slug convention, waterfall, and graceful degradation.
• RAG plugin metering is not yet wired (Phase 2).
• Two-tier double-write (platform + tenant) is designed but not yet implemented — current code writes to one tier only.

Link to Devin session: https://app.devin.ai/sessions/2b5a29d83d3f478e8d3d972653b4879c
Requested by: @pyramation

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring