Commit d1c6057
debug(deltanet): TQ_DELTA_PROBE — locates L0 as the outlier layer
Adds per-layer state L2 norm dump env (comma-separated list of layer-0
call counts). Thread-local, zero-cost when unset.
Applied on Qwen3.6-35B IQ4_XS "Once upon a time in a faraway land":
call=50 L0=127.0 L1=41.3 L2=17.9 L8=31.1 others 7-24
call=100 L0=150.7 L1=42.4 L2=16.7 L8=31.5 others 7-24
call=115 L0=154.9 L1=40.8 L2=16.6 L8=30.6 others 7-24 ← 117-tok loop start
call=120 L0=154.5 L1=40.8 L2=16.7 L8=30.5 others 7-24
L0's DeltaNet recurrent state sits at 3-10× every other layer's norm.
Grew 127→155 over 100 tokens (+22%) while others stayed ±10%.
R16 proved 117-tok repetition loop IS state-driven. R17 localizes the
suspect to L0's recurrent state specifically — either a_log decay param
is scaled differently at L0, or our implementation has an L0-specific
bug. Next round: inspect a_log weight + compare our decay math to
refs/llama.cpp.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent b061e7d commit d1c6057
2 files changed
Lines changed: 58 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
6 | 38 | | |
7 | 39 | | |
8 | 40 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
633 | 633 | | |
634 | 634 | | |
635 | 635 | | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
636 | 662 | | |
637 | 663 | | |
638 | 664 | | |
| |||
0 commit comments