Skip to content

feat(graphile-llm): add inference usage logging to metering plugin#1196

Merged
pyramation merged 2 commits into
mainfrom
feat/inference-usage-logging
May 21, 2026
Merged

feat(graphile-llm): add inference usage logging to metering plugin#1196
pyramation merged 2 commits into
mainfrom
feat/inference-usage-logging

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

@pyramation pyramation commented May 19, 2026

Summary

Adds inference usage logging to the LLM metering plugin. Every metered LLM call (embed or chat) now INSERTs into the usage_log_inference table with full usage metadata.

Changes:

  • metering.ts: adds logInferenceUsage() helper that INSERTs into the inference log table; adds databaseId, actorId, inferenceLog to MeteringContext; both meteredEmbed and meteredChat now log usage on success and quota_exceeded
  • metering-plugin.ts: extracts databaseId, actorId from pgSettings; queries inferenceLog config from getLlmBillingConfig()
  • config-cache.ts: adds inferenceLog to the ModuleConfigCache — queries inference_log_module config alongside billing config
  • Token counts use placeholderAmountTokens (~4 chars/token estimation) — clearly marked as placeholders pending team approval of generateWithUsage() on agentic-kit

Depends on: #1192 (billing metering + cache standardization, now merged)

Review & Testing Checklist for Human

  • Verify usage_log_inference rows appear after an LLM embed/chat call with correct model, provider, request_type, token counts, latency_ms, and status
  • Verify quota_exceeded events are logged (not just successes)
  • Confirm placeholderAmountTokens variables are clearly placeholder — search for the comment "replace with actual provider token counts once generateWithUsage() is approved"

Notes

Token estimation is intentionally a placeholder (Math.ceil(text.length / 4)). The swap-out point is clearly marked in the code. Once generateWithUsage() is approved on agentic-kit, replace placeholderAmountTokens with actual provider counts.

Link to Devin session: https://app.devin.ai/sessions/2b5a29d83d3f478e8d3d972653b4879c
Requested by: @pyramation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration Bot force-pushed the feat/inference-usage-logging branch from d2463b7 to df79e9e Compare May 19, 2026 20:54
@devin-ai-integration devin-ai-integration Bot force-pushed the feat/llm-billing-metering branch 2 times, most recently from b5d6099 to 9c0265c Compare May 20, 2026 22:31
Base automatically changed from feat/llm-billing-metering to main May 21, 2026 01:54
@devin-ai-integration devin-ai-integration Bot force-pushed the feat/inference-usage-logging branch from df79e9e to 6b9fab7 Compare May 21, 2026 01:57
Adds inline INSERT into usage_log_inference table after billing
record_usage calls in both meteredEmbed and meteredChat functions.

Changes:
- config-cache.ts: Add InferenceLogConfig type and resolution from
  inference_log_module metaschema table (cached alongside billing config)
- metering.ts: Add InferenceLogEntry type, logInferenceUsage helper,
  and calls after billing in meteredEmbed/meteredChat (including
  quota_exceeded events). Add databaseId, actorId, inferenceLog to
  MeteringContext. Add embeddingModel, chatModel, provider to
  MeteringOptions.
- metering-plugin.ts: Wire databaseId, actorId, inferenceLog into
  MeteringContext. Pass model names to MeteringOptions.
- index.ts: Export new types (InferenceLogEntry, InferenceLogConfig)
  and logInferenceUsage function.

Gracefully skips if inference_log_module is not provisioned.
TODO: dual-write to child (generated) database for platform aggregation.
@devin-ai-integration devin-ai-integration Bot force-pushed the feat/inference-usage-logging branch from 6b9fab7 to fb532f3 Compare May 21, 2026 03:02
@pyramation pyramation merged commit 84a72b0 into main May 21, 2026
37 checks passed
@pyramation pyramation deleted the feat/inference-usage-logging branch May 21, 2026 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant