Skip to content

fix: handle stream interruption for OpenAI-compatible providers#21727

Open
jwcrystal wants to merge 2 commits intoanomalyco:devfrom
jwcrystal:fix/openai-compat-stream-interrupt
Open

fix: handle stream interruption for OpenAI-compatible providers#21727
jwcrystal wants to merge 2 commits intoanomalyco:devfrom
jwcrystal:fix/openai-compat-stream-interrupt

Conversation

@jwcrystal
Copy link
Copy Markdown

@jwcrystal jwcrystal commented Apr 9, 2026

Issue for this PR

Closes #20466
Related #15393, #17680, #17574, #17307

Type of change

  • Bug fix

What does this PR do?

OpenAI-compatible providers (Ollama, cloud models) can interrupt SSE streams mid-response. Previously, opencode silently accepted the truncated output as a complete response — no retry, no error, just a cut-off answer.

Three fixes:

  1. retry.ts — Added SSE timeout, connection reset, abort, and stream truncation patterns to retryable() matching so interrupted streams are automatically retried.

  2. message-v2.ts — Classified SSE read timed out errors as APIError(isRetryable: true) instead of Unknown, so the retry mechanism recognizes and retries chunk timeouts.

  3. openai-compatible-chat-language-model.ts — When flush() is called without a finish_reason from the provider while output is still active (text/reasoning/tool-call), emit an error event. This triggers the error handling path which retries, rather than silently accepting a truncated response.

How did you verify your code works?

  • 5 new retryable() test cases: SSE timeout, connection reset, aborted, stream ended unexpectedly, existing rate-limit still passes
  • 1 new fromError() test case: SSE read timed outAPIError(isRetryable: true, code: SSE_TIMEOUT)
  • All 51 tests across retry.test.ts and message-v2.test.ts pass

Screenshots / recordings

N/A — backend fix, no UI changes.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

@github-actions github-actions bot added needs:issue needs:compliance This means the issue will auto-close after 2 hours. labels Apr 9, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

Thanks for your contribution!

This PR doesn't have a linked issue. All PRs must reference an existing issue.

Please:

  1. Open an issue describing the bug/feature (if one doesn't exist)
  2. Add Fixes #<number> or Closes #<number> to this PR description

See CONTRIBUTING.md for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

The following comment was made by an LLM, it may be inaccurate:

Based on my search, I found one potentially related PR:

PR #19116 - fix(opencode): reconnect on network disruptions (VPN switch, SSE timeout, connection reset)

This PR addresses similar network and SSE timeout issues, though it appears to focus on reconnection rather than the specific retry mechanism for incomplete responses. However, it may have overlapping concerns around SSE timeout handling and connection reset patterns.

All other results were either the current PR (21727) itself or unrelated retry/error handling fixes.

No other significant duplicate PRs found.

@github-actions github-actions bot removed needs:compliance This means the issue will auto-close after 2 hours. needs:issue labels Apr 9, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 9, 2026

Thanks for updating your PR! It now meets our contributing guidelines. 👍

Three fixes for Ollama/cloud models using OpenAI-compatible format
where responses can be interrupted mid-stream:

1. retry.ts: Add SSE timeout, connection reset, abort, and stream
   truncation patterns to retryable error matching so interrupted
   streams are automatically retried.

2. message-v2.ts: Classify 'SSE read timed out' errors as
   APIError(isRetryable: true) instead of Unknown, so the retry
   mechanism recognizes and retries chunk timeouts.

3. openai-compatible-chat-language-model.ts: When the TransformStream
   flush() is called without a finish_reason from the provider while
   output is still active (text/reasoning/tool-call), emit an error
   event. This triggers the error handling path which retries rather
   than silently accepting a truncated response.

Tests: 5 new retryable() cases + 1 fromError() SSE timeout case,
all passing.
@jwcrystal jwcrystal force-pushed the fix/openai-compat-stream-interrupt branch from f495243 to 8a0fea2 Compare April 9, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SSE timeout errors are not retried when chunkTimeout is exceeded

1 participant