Commit 116d28c
committed
Avoid whole-loading index sidecars during lookups
Refactor index query paths to operate on sidecar handles instead of
materializing payload arrays in memory.
- stream full exact lookups from sidecars for both compact and run-bounded paths
- stream ordered full reuse with on-the-fly merge of base and incremental runs
- remove light and medium exact-query fallbacks that whole-loaded payload sidecars
- cache in-memory sidecars as compressed handles instead of NumPy arrays
- add handle-based summary/boundary readers for query planning and navigation
- fix in-memory cache cleanup so stale sidecar handles do not survive array finalization
- keep light lossy chunk pruning conservative to avoid dropping boundary matches
- add regression tests guarding against _load_array_sidecar / _load_full_arrays
in exact and ordered query execution1 parent 99be334 commit 116d28c
2 files changed
Lines changed: 642 additions & 247 deletions
0 commit comments