Commit a05d4e4
debug(deltanet): R18 null — a_log formula IS correct for Unsloth GGUF
Tested removing our -expf(delta_a_log) on hypothesis the GGUF stored
pre-transformed -exp(A_log) like llama.cpp kimi-linear.cpp:142 suggests.
Result: "Paris" factual probe collapses to "T" (immediate garbage) on
Qwen3.6-35B IQ4_XS. Rolled back.
Conclusion: this Unsloth UD-IQ4_XS GGUF stores RAW A_log (Unsloth's
conversion differs from kimi-linear's convert_hf_to_gguf.py convention).
Our -expf(delta_a_log) is correct for THIS gguf.
L0's outlier DeltaNet state norm (~155) is by design — heads with large
negative a_log have weak decay intentionally. Not a kernel bug.
The 117-tok loop cause is elsewhere in the DeltaNet path. TQ_DELTA_PROBE
and TQ_DELTA_RESET_EVERY envs remain available for future ablations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent d1c6057 commit a05d4e4
1 file changed
Lines changed: 30 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
6 | 36 | | |
7 | 37 | | |
8 | 38 | | |
| |||
0 commit comments