Commit 4e4cd18
feat: llma / error tracking integration (#376)
* feat: llma / error tracking integration
* capture all metadata in llm event
* instrument with contexts
* bump version
* indentation
* linting
* tests
* raise
* test: add exception capture integration tests for langchain
Add 6 tests covering the new LLMA + error tracking integration:
- capture_exception called on span/generation errors
- $exception_event_id added to AI events
- No capture when autocapture disabled
- AI properties passed to exception event
- Handles None return from capture_exception
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: pass context tags to capture() for test compatibility
- Export get_tags() from posthog module
- Explicitly pass context tags to capture() in AI utils
- Fix $ai_model fallback to extract from response.model
- Fix ruff formatting in langchain test_callbacks.py
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: disable auto-capture exceptions in LLM context
The new_context() defaults to capture_exceptions=True which would
auto-capture any exception regardless of enable_exception_autocapture
setting. This was inconsistent with LangChain callbacks which
explicitly check the setting.
Pass capture_exceptions=False to let exception handling be controlled
explicitly by the enable_exception_autocapture setting.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: isolate LLM context with fresh=True to avoid tag inheritance
Use fresh=True to start with a clean context for each LLM call.
This avoids inheriting $ai_* tags from parent contexts which could
cause mismatched AI metadata due to the tag merge order bug in
contexts.py (parent tags incorrectly override child tags).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: correct tag merge order so child tags take precedence
The collect_tags() method had a bug where parent tags would overwrite
child tags, despite the comment saying the opposite. This fix ensures
child context tags properly override parent tags.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: remove fresh=True now that tag merge order is fixed
With the collect_tags() bug fixed, child tags properly override parent
tags. LLM events can now inherit useful parent context tags (request_id,
user info, etc.) while still having their $ai_* tags take precedence.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test: add test for child tags overriding parent tags
Verifies that in non-fresh contexts, child tags properly override
parent tags with the same key while still inheriting other parent tags.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: add TODO for OpenAI/Anthropic/Gemini exception capture
Document that exception capture needs to be added for the direct SDK
wrappers, similar to how it's implemented in LangChain callbacks.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: David Newell <david@Mac.communityfibre.co.uk>
Co-authored-by: Andrew Maguire <andrewm4894@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>1 parent c1548c4 commit 4e4cd18
9 files changed
Lines changed: 534 additions & 210 deletions
File tree
- posthog
- ai
- langchain
- test
- ai
- langchain
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
1 | 6 | | |
2 | 7 | | |
3 | 8 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
32 | 35 | | |
33 | 36 | | |
34 | 37 | | |
| |||
190 | 193 | | |
191 | 194 | | |
192 | 195 | | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
193 | 209 | | |
194 | 210 | | |
195 | 211 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
26 | 25 | | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | | - | |
39 | 38 | | |
| 39 | + | |
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
46 | 45 | | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| |||
506 | 506 | | |
507 | 507 | | |
508 | 508 | | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
509 | 517 | | |
510 | 518 | | |
511 | 519 | | |
| |||
576 | 584 | | |
577 | 585 | | |
578 | 586 | | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
579 | 593 | | |
580 | 594 | | |
581 | 595 | | |
582 | 596 | | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
583 | 605 | | |
584 | 606 | | |
585 | 607 | | |
| |||
607 | 629 | | |
608 | 630 | | |
609 | 631 | | |
610 | | - | |
611 | | - | |
612 | | - | |
613 | | - | |
614 | | - | |
615 | | - | |
616 | 632 | | |
617 | 633 | | |
618 | 634 | | |
| |||
863 | 879 | | |
864 | 880 | | |
865 | 881 | | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
866 | 903 | | |
867 | 904 | | |
868 | 905 | | |
| |||
0 commit comments