Skip to content

fix(google_genai): Redact binary data in inline_data and fix multi-part message extraction#5977

Draft
ericapisani wants to merge 1 commit intomasterfrom
ep/py-2287-missing-images-in-response-5aw
Draft

fix(google_genai): Redact binary data in inline_data and fix multi-part message extraction#5977
ericapisani wants to merge 1 commit intomasterfrom
ep/py-2287-missing-images-in-response-5aw

Conversation

@ericapisani
Copy link
Copy Markdown
Member

Redact binary/byte data that appears in inline_data, which fixes a failing checkBinaryRedaction assertion in the AI testing framework.

Also includes changes for the following:

  • Properly handling lists of part-like items (merging into single multi-part user message)
  • Handling bare inline_data dicts that aren't wrapped in Part objects
  • Always substituting blob data (both bytes and base64 strings)
  • Moving PIL import to module level with availability flag to reduce all the dynamic imports of the module within the code

Fixes PY-2287 and #5965

…rt message extraction

Redact binary/byte data that appears in `inline_data`, which fixes a failing `checkBinaryRedaction` assertion in the AI testing framework.

Also includes changes for the following:
- Properly handling lists of part-like items (merging into single multi-part user message)
- Handling bare inline_data dicts that aren't wrapped in Part objects
- Always substituting blob data (both bytes and base64 strings)
- Moving PIL import to module level with availability flag to reduce all the dynamic imports of the module within the code

Fixes PY-2287 and #5965

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@linear-code
Copy link
Copy Markdown

linear-code bot commented Apr 10, 2026

@github-actions
Copy link
Copy Markdown
Contributor

Semver Impact of This PR

🟢 Patch (bug fixes)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


New Features ✨

  • (ai) Redact base64 data URLs in image_url content blocks by ericapisani in #5953
  • (integrations) Instrument pyreqwest tracing by servusdei2018 in #5682

Bug Fixes 🐛

Anthropic

  • Capture exceptions for stream() calls by alexander-alderman-webb in #5950
  • Stop setting transaction status when child span fails by alexander-alderman-webb in #5717
  • Only finish relevant spans in .create() patches by alexander-alderman-webb in #5716

Other

  • (google_genai) Redact binary data in inline_data and fix multi-part message extraction by ericapisani in #5977
  • (pydantic-ai) Use first-class hooks when available by alexander-alderman-webb in #5947
  • (wsgi) Respect HTTP_X_FORWARDED_PROTO in request.url construction by sl0thentr0py in #5963

Internal Changes 🔧

  • (ai) Remove gen_ai.tool.type span attribute by ericapisani in #5964
  • (anthropic) Separate sync and async .create() patches by alexander-alderman-webb in #5715
  • (openai) Split token counting by API for easier deprecation by ericapisani in #5930
  • (opentelemetry) Ignore mypy error by alexander-alderman-webb in #5927
  • Fix license metadata in setup.py by sl0thentr0py in #5934
  • Update validate-pr workflow by stephanie-anderson in #5931

Other

  • Handle None span context in the span processor and pin tokenizers version for anthropic tests on Python 3.8 by alexander-alderman-webb in #5967

🤖 This preview updates automatically when you update the PR.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 10, 2026

Codecov Results 📊

142 passed | Total: 142 | Pass Rate: 100% | Execution Time: 21.04s

All tests are passing successfully.

❌ Patch coverage is 59.02%. Project has 14068 uncovered lines.

Files with missing lines (1)
File Patch % Lines
utils.py 80.70% ⚠️ 99 Missing and 56 partials

Generated by Codecov Action

@ericapisani
Copy link
Copy Markdown
Member Author

bugbot run

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d197318. Configure here.

"data": BLOB_DATA_SUBSTITUTE,
}
}
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inline data message missing role and content keys

Medium Severity

The elif "inline_data" in contents: branch produces a message dict {"inline_data": {...}} that has no "role" key and uses "inline_data" instead of "content" as the data key. Every other branch in extract_contents_messages returns messages with both "role" and "content" keys. This malformed structure causes downstream functions like redact_blob_message_parts (which checks message.get("content")) and _truncate_single_message_content_if_present (which checks "content" not in message) to silently skip over these messages. This path is reachable when a list like [Content(...), {"inline_data": {...}}] is processed recursively because not all items are part-like.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d197318. Configure here.

messages.append(
{
"role": role or "user",
"role": role,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Role fallback removed for falsy role values

Low Severity

The "role": role or "user" was changed to "role": role in the elif "text" in contents: branch, removing the fallback for explicitly falsy role values (e.g., None or ""). Since role comes from contents.get("role", "user"), a dict with "role": None would produce a message with role=None. The sibling if parts: branch at line 226 still uses role or "user", creating an inconsistency.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit d197318. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant