Commit b061e7d
debug(deltanet): TQ_DELTA_RESET_EVERY ablation proves state-driven drift
Adds env-gated ablation hook in deltanet_forward to periodically zero
the recurrent state (delta_state + conv_state) across all layers. Default
off; thread-local counter fires once per layer-0 call.
Ablation on Qwen3.6-35B IQ4_XS "Once upon a time in a faraway land":
unset: 117 tokens → "It could do math!" loop (baseline)
RESET_EVERY=50: content diverges early, no 'math' loop but degrades
RESET_EVERY=120: identical 0-117, post-117 loop REPLACED with
different incoherent text ("0 Comments | Views: 4,986")
Causal conclusion: the specific 117-token repetition loop IS driven by
DeltaNet state accumulation. Reset changes post-drift content entirely,
so KV / MoE / attention aren't the mechanism. This is NOT a fix (reset
throws useful context), but gives us a diagnostic lever for future
per-head/per-layer norm probing to localize the exploding subtensor.
Regression 15/15 PASS (env-gated, default behavior unchanged).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 2a1d40d commit b061e7d
2 files changed
Lines changed: 52 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
6 | 35 | | |
7 | 36 | | |
8 | 37 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
623 | 623 | | |
624 | 624 | | |
625 | 625 | | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
626 | 649 | | |
627 | 650 | | |
628 | 651 | | |
| |||
0 commit comments