paper(working-memory-cliff): v0.4 — ReDeEP-Cliff hypothesis tested + new failure mode named

Phase 2B Karpathy loop. Tested whether the cliff failure mode shares
mechanism with RAG silent hallucination as described by ReDeEP (Sun et
al., ICLR 2025): hallucination has low External Context Score and high
Parametric Knowledge Score. We don't have direct access to attention
heads and FFN residuals from the quant.cpp forward pass, so we used
surface-level proxies on the existing 198 NIAH trial responses:

  copy_score(response, haystack) = 4-gram overlap, proxy for ECS
  novel_score = 1 - copy_score,                proxy for PKS

Results across 4 Karpathy rounds on the existing data, no new compute:

R1 — pooled across all 115 cliff failures, FAIL has HIGHER copy_score
     than PASS (0.532 vs 0.440). This is the OPPOSITE of ReDeEP's RAG
     hallucination signature. Hypothesis rejected at top level.

R2 — subtype classification of all 115 failures:
       CONTINUE  97  84%   literal wikitext continuation
       OTHER      9   8%   other forms of continuation
       HEADER     8   7%   wikitext markup echo (= = =, etc)
       SYNTH      1  <1%   needle + subject fusion (the v0.3 example)

     The "Boulter was hired as CFO" synthesised hallucination that
     v0.3 cited as the dominant failure mode is one trial out of 115.
     The dominant mode is literal continuation, and it has the
     OPPOSITE ReDeEP signature from RAG hallucination.

R3 — position of longest matching substring across the haystack:
       Q1 (0-25%)   87  81%
       Q2 (25-50%)  11  10%
       Q3 (50-75%)   6   6%
       Q4 (75-100%)  3   3%

     81% of cliff continuations resume from the FIRST quartile of the
     haystack, not from the end of the prompt. The model is jumping
     back to the beginning of the document, not autocompleting where
     the assistant turn would have started.

R4 — decile precision: 70% of continuations resume specifically from
     the 10-20% sub-region of the haystack — the start of the article
     BODY in wikitext, just after the title and lead paragraph. The
     model recognises "this is a Wikipedia article" from the title
     markup and emits the body content from its canonical body-start
     position.

We name this new failure mode:

  PRIMACY-BIASED DOCUMENT CONTINUATION OVERFLOW

It is mechanistically distinct from RAG silent hallucination (opposite
signature), from "Lost in the Middle" (which is about retrieval
position, not generation source), and from attention sink collapse
(BOS sink is being overruled, not lost). It is also distinct from
parametric hallucination — the model is not inventing from internal
memory, it is literally copying from the loaded context.

Implications for mitigation:
- ReDeEP's AARF (Add Attention, Reduce FFN) is designed for the
  parametric-takeover regime. For our cliff failure it would either
  be ineffective or counterproductive — the cliff has the opposite
  imbalance.
- The correct mitigation direction is anchor strengthening: increase
  the chat-template anchor's effective attention weight to outcompete
  the document-continuation prior. Phase 2C candidates outlined in §8:
  PQRI (periodic question re-injection), conversational chunking,
  QASI (SinkTrack-style instruction injection into BOS sink).

Self-correction of v0.3:
- v0.3 §4.6 cited the Boulter+CFO synthesised hallucination example
  as the most consequential cliff failure mode and equated it with
  RAG silent hallucination. This was based on a single visually
  striking example. Subtype analysis on all 115 failures shows it's
  one trial out of 115 (<1%), not the dominant mode.
- v0.4 §4.6 is rewritten with the corrected taxonomy, the explicit
  ReDeEP signature comparison, the position-quartile analysis, and a
  clear "honest correction" note for v0.3 readers.
- v0.4 §1 TL;DR and §6 Discussion are updated for the corrected
  mechanism understanding.
- v0.4 §8 Future Work is rewritten with the concrete next-step
  mitigation experiments suggested by the new mechanism.

Files added (Phase 2B Karpathy loop):
- bench/results/niah/redeep_proxy.{py,json}
    R1 — pooled copy/novel proxy on all 198 trials
- bench/results/niah/redeep_subtype.{py,json}
    R2 — failure subtype classification + per-subtype ReDeEP comparison
- bench/results/niah/continuation_origin.{py,json}
    R3+R4 — quartile/decile position of longest haystack match

The result: a stronger, more publishable finding than the original
hypothesis would have been. We discovered a *new failure mode at edge
scale* rather than confirming an existing mechanism.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: unamedkr

.gitignore

@@ -67,3 +67,5 @@ build_metal/
 
 .env
 
+
+blog/

bench/results/niah/continuation_origin.json

@@ -0,0 +1,12 @@
+{
+  "n_failures": 115,
+  "n_with_match": 107,
+  "buckets": {
+    "Q1 (0-25%)": 87,
+    "Q2 (25-50%)": 11,
+    "Q3 (50-75%)": 6,
+    "Q4 (75-100%)": 3
+  },
+  "mean_position": 0.23035950770630845,
+  "mean_offset_end": -0.7696404922936915
+}

bench/results/niah/continuation_origin.py

@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+"""R3: where does the continuation start?
+
+R1+R2 found that 84% of cliff failures are literal continuation of the
+haystack ("Doctors, followed by a role in..."). This script checks
+WHERE in the haystack the continuation matches, to test the hypothesis
+that the model is *resuming the wikitext from a specific position*
+rather than generating from a random place.
+
+Hypotheses to distinguish:
+
+  H1 — RESUME-AT-END: continuation starts from where the assistant turn
+       was supposed to begin. The chat template anchor was too weak; the
+       model defaulted to "auto-complete from end of input".
+
+  H2 — RESUME-FROM-NEEDLE: continuation starts from somewhere near the
+       inserted needle. The model partially noticed the needle but then
+       resumed copying haystack from there.
+
+  H3 — JUMP-TO-EARLY: continuation starts from somewhere early in the
+       haystack (model uses recency bias to grab a fluent passage).
+
+We measure: for each CONTINUE failure, where is the longest matching
+substring in the haystack? Then bucket by haystack position quartile.
+"""
+import csv
+import json
+import re
+import sys
+from collections import Counter
+from pathlib import Path
+
+REPO = Path(__file__).resolve().parent.parent.parent.parent
+PROXY = REPO / "bench/results/niah/redeep_proxy.json"
+SUBTYPE = REPO / "bench/results/niah/redeep_subtype.json"
+WIKITEXT = REPO / "bench/data/wikitext2_test.txt"
+
+NEEDLES = [
+    "The chief financial officer of Northwind Logistics is Sarah Chen, hired in 2023.",
+    "The launch date for Project Aurora is November 14th in San Francisco.",
+    "The reactor cooling tank at the Helios facility holds exactly eight thousand liters of distilled water.",
+]
+
+
+def normalize(s: str) -> str:
+    return re.sub(r"[^a-z0-9 ]+", " ", s.lower()).strip()
+
+
+def reconstruct_haystack(ctx_tokens: int, needle_idx: int) -> tuple:
+    """Returns (haystack_with_needle, needle_position_chars)."""
+    raw = WIKITEXT.read_text(encoding="utf-8", errors="replace")
+    target_chars = int(ctx_tokens * 3.6)
+    hay = raw[:target_chars]
+    end = hay.rfind(". ")
+    if end > 0:
+        hay = hay[:end + 1]
+    needle = NEEDLES[needle_idx]
+    desired = len(hay) // 2
+    sb = hay.rfind(". ", 0, max(desired, 2))
+    sb = 0 if sb < 0 else sb + 2
+    return hay[:sb] + needle + " " + hay[sb:], sb
+
+
+def longest_match_position(response: str, haystack: str, min_len: int = 30) -> int:
+    """Return the haystack character position where the longest substring
+    of `response` (length >= min_len) appears, or -1 if none found."""
+    rn = normalize(response)
+    hn = normalize(haystack)
+    best_pos = -1
+    best_len = min_len - 1
+    # Walk windows of decreasing length
+    for L in range(min(len(rn), 80), min_len - 1, -5):
+        for i in range(0, max(1, len(rn) - L)):
+            sub = rn[i:i+L]
+            if not sub.strip():
+                continue
+            pos = hn.find(sub)
+            if pos >= 0 and L > best_len:
+                best_pos = pos
+                best_len = L
+                break  # found longest at this L
+        if best_pos >= 0:
+            break
+    return best_pos
+
+
+def main():
+    with open(PROXY) as f:
+        proxy = json.load(f)
+
+    # Load all FAIL trials with their failed responses
+    failures = [r for r in proxy["trials"] if r["pass"] == 0]
+
+    # Reconstruct haystacks once per (ctx, needle)
+    haystack_cache = {}
+    def get_hay(ctx, needle):
+        key = (ctx, needle)
+        if key not in haystack_cache:
+            haystack_cache[key] = reconstruct_haystack(ctx, needle)
+        return haystack_cache[key]
+
+    # For each failure, find the longest match position
+    results = []
+    for r in failures:
+        haystack, needle_pos = get_hay(r["context"], r["needle"])
+        pos = longest_match_position(r["response"], haystack)
+        haylen = len(normalize(haystack))
+        if pos < 0 or haylen == 0:
+            position_pct = None
+            relative_to_needle = None
+            relative_to_end = None
+        else:
+            position_pct = pos / haylen
+            # Approx needle position in normalized chars
+            needle_pct = needle_pos / max(len(haystack), 1)
+            relative_to_needle = position_pct - needle_pct
+            relative_to_end = position_pct - 1.0  # negative = before end
+        results.append({
+            **r,
+            "match_pos_pct": position_pct,
+            "rel_to_needle": relative_to_needle,
+            "rel_to_end":    relative_to_end,
+        })
+
+    # Distribution of match positions across all failures
+    valid = [r for r in results if r["match_pos_pct"] is not None]
+    print(f"Failures with detectable haystack match: {len(valid)} / {len(failures)}")
+    print(f"  (those without a match — model invented enough that no 30-char substring matches)\n")
+
+    # Bucket by quartile
+    buckets = Counter()
+    for r in valid:
+        p = r["match_pos_pct"]
+        if   p < 0.25: buckets["Q1 (0-25%)"]   += 1
+        elif p < 0.50: buckets["Q2 (25-50%)"]  += 1
+        elif p < 0.75: buckets["Q3 (50-75%)"]  += 1
+        else:          buckets["Q4 (75-100%)"] += 1
+
+    print("Continuation start position (longest matching substring) — quartile of haystack:")
+    for q in ["Q1 (0-25%)", "Q2 (25-50%)", "Q3 (50-75%)", "Q4 (75-100%)"]:
+        n = buckets[q]
+        bar = "█" * (n * 30 // max(len(valid), 1))
+        print(f"  {q:<14} {n:>3}  {bar}")
+    print()
+
+    # Mean position relative to needle and end
+    if valid:
+        mean_pos = sum(r["match_pos_pct"] for r in valid) / len(valid)
+        mean_to_end = sum(r["rel_to_end"] for r in valid) / len(valid)
+        mean_to_needle = sum(r["rel_to_needle"] for r in valid) / len(valid)
+        print(f"Mean match position (% of haystack):  {mean_pos:.2f}")
+        print(f"Mean offset from end of haystack:      {mean_to_end:+.2f}  (negative = before end)")
+        print(f"Mean offset from needle position:      {mean_to_needle:+.2f}  (positive = after needle)")
+        print()
+
+    # Hypothesis verdicts
+    print("--- Hypothesis verdicts ---\n")
+    if valid:
+        q4_frac = buckets["Q4 (75-100%)"] / len(valid)
+        q1_frac = buckets["Q1 (0-25%)"]  / len(valid)
+        if q4_frac > 0.5:
+            print(f"H1 (RESUME-AT-END) — SUPPORTED: {q4_frac:.0%} of continuations match")
+            print( "                                 the last quartile of the haystack.")
+            print( "                                 → chat template anchor failed; model")
+            print( "                                   defaulted to autocomplete from prompt end.")
+        elif q1_frac > 0.5:
+            print(f"H3 (JUMP-TO-EARLY) — SUPPORTED: {q1_frac:.0%} of continuations match")
+            print( "                                 the first quartile of the haystack.")
+        else:
+            # Distribution is broader — check needle proximity
+            near_needle = sum(1 for r in valid if abs(r["rel_to_needle"]) < 0.1)
+            if near_needle / len(valid) > 0.5:
+                print(f"H2 (RESUME-FROM-NEEDLE) — SUPPORTED: {near_needle/len(valid):.0%}")
+                print( "                            of continuations match within ±10% of needle position.")
+            else:
+                print("MIXED — no single position dominates. Continuations are spread across the haystack.")
+
+    # Save
+    out = REPO / "bench/results/niah/continuation_origin.json"
+    with open(out, "w") as f:
+        json.dump({
+            "n_failures":      len(failures),
+            "n_with_match":    len(valid),
+            "buckets":         dict(buckets),
+            "mean_position":   mean_pos if valid else None,
+            "mean_offset_end": mean_to_end if valid else None,
+        }, f, indent=2)
+    print(f"\nSaved to {out}")
+
+
+if __name__ == "__main__":
+    main()