Skip to content

feat: support conversation history directly in AI Provider model runners#1371

Open
jsonbailey wants to merge 1 commit into
mainfrom
jb/message-history-in-providers
Open

feat: support conversation history directly in AI Provider model runners#1371
jsonbailey wants to merge 1 commit into
mainfrom
jb/message-history-in-providers

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented May 7, 2026

Summary

Port the multi-turn conversation history pattern from the Python AI SDK provider runners to the js-core AI SDK provider runners. Mirrors the behaviour shipped in python-server-sdk-ai #166.

Each provider model runner now keeps an internal conversation history that is seeded from the AI config's messages on construction and grows with each successful call. On every run() invocation:

  1. The user prompt is appended to the existing history before being sent to the model.
  2. If the call succeeds and produces non-empty content, the user prompt and the assistant's reply are persisted to the history.
  3. If the call fails or returns empty content, the history is left unchanged so the next call can retry cleanly.

The public run(input) interface is unchanged and still returns the same RunnerResult. The history is purely internal state on the runner instance.

Per-provider approach

  • OpenAI (OpenAIModelRunner.ts) — private LDMessage[] history, mirroring the Python OpenAI runner. The OpenAI Chat Completions API has no native conversation state.
  • LangChain (LangChainModelRunner.ts) — uses InMemoryChatMessageHistory from @langchain/core/chat_history, the JS equivalent of langchain_core.chat_history. This mirrors the Python LangChain runner.
  • Vercel (VercelModelRunner.ts) — private ModelMessage[] history (Vercel AI SDK native v5 type). No analogue exists in the Python SDK; chose ModelMessage[] to avoid re-converting on every call.

Behaviour change for multi-turn callers (medium risk)

Callers that invoke run() multiple times on the same runner instance will now accumulate conversation history across calls. Token usage will grow with each turn since the full history is sent on every call. Callers that want stateless behaviour should construct a new runner per call.

Test plan

  • Multi-turn accumulation across two successful run() calls (per provider)
  • No accumulation when run() throws (per provider)
  • No accumulation when run() returns empty content (per provider)
  • Config / system messages still prepended ahead of accumulated history on every call (per provider)
  • Existing tests still pass (43 OpenAI / 42 LangChain / 24 Vercel)
  • yarn workspaces foreach -pR --topological-dev --from '@launchdarkly/server-sdk-ai' run build succeeds
  • Lint passes for each touched package

Note

Medium Risk
Changes run() semantics for repeated calls by accumulating prior prompts/responses, which can increase token usage and alter model outputs; failures/empty content are explicitly excluded from history to reduce runaway state.

Overview
Adds stateful multi-turn support to the LangChainModelRunner, OpenAIModelRunner, and VercelModelRunner by keeping an internal conversation history seeded from config.messages and appending it to every run() call.

History is only persisted after successful calls with non-empty content; exceptions or empty/multimodal responses leave history unchanged to allow clean retries. New unit tests in each provider verify accumulation behavior and that config/system messages stay prepended ahead of the growing history.

Reviewed by Cursor Bugbot for commit b74c851. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

@launchdarkly/js-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 26281 bytes
Compressed size limit: 29000
Uncompressed size: 128971 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

@launchdarkly/js-client-sdk-common size report
This is the brotli compressed size of the ESM build.
Compressed size: 38487 bytes
Compressed size limit: 39000
Uncompressed size: 211236 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

@launchdarkly/js-client-sdk size report
This is the brotli compressed size of the ESM build.
Compressed size: 31906 bytes
Compressed size limit: 34000
Uncompressed size: 113658 bytes

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

@launchdarkly/browser size report
This is the brotli compressed size of the ESM build.
Compressed size: 179498 bytes
Compressed size limit: 200000
Uncompressed size: 830837 bytes

Each provider model runner now keeps an internal conversation history that is
seeded from the AI config's messages and grows with each successful call. The
user prompt and the assistant's reply are appended to history only when the
call succeeds and produces non-empty content, so failed calls leave history
unchanged for retries. Mirrors python-server-sdk-ai #166.

- OpenAI: private LDMessage[] history, mirrored on the Python OpenAI runner.
- LangChain: uses InMemoryChatMessageHistory from @langchain/core/chat_history
  to mirror the Python LangChain runner.
- Vercel: private ModelMessage[] history (Vercel-native types).
@jsonbailey jsonbailey force-pushed the jb/message-history-in-providers branch from 017efa0 to b74c851 Compare May 11, 2026 14:20
@jsonbailey jsonbailey marked this pull request as ready for review May 12, 2026 13:38
@jsonbailey jsonbailey requested a review from a team as a code owner May 12, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant