Add multi-value GET-parameter chunker for waterdata OGC API

[thodson-usgs]

Summary

Adds a multi-value GET-parameter chunker (dataretrieval/waterdata/chunking.py) that splits cartesian combinations of multi-value list params across sub-requests so each URL fits the server's ~8 KB byte limit. Sits OUTSIDE @filters.chunked on _fetch_once, so list-chunking is the outer loop and CQL-filter-chunking is the inner loop. The two chunkers coordinate via a filter-aware probe.

Depends on #273. That PR fixes the pre-existing silent-truncation bug in _walk_pages whose blast radius this PR materially expands. Merge order: #273 → #233 → this PR. See "Merge ordering" below.

Why

PR #233 routes multi-value waterdata queries through GET with comma-separated values (e.g. monitoring_location_id=USGS-A,USGS-B,…). For real-world workloads (~300+ sites or pcodes), the URL blows past the server's 8 KB nginx buffer and the API returns HTTP 414. This PR makes long list params transparently fan out.

Design

@chunking.multi_value_chunked(build_request=_construct_api_requests)   # outer
@filters.chunked(build_request=_construct_api_requests)                # inner
def _fetch_once(args: dict) -> tuple[DataFrame, Response]: ...

• Planner (_plan_chunks): greedy halving of the largest chunk across all dims until the worst-case sub-request URL fits the byte limit. Cartesian product of per-dim partitions becomes the sub-request set. O(log N) per dim.
• Coordination with filters.chunked: when planning, the URL probe substitutes the filter with its longest top-level OR-clause via _filter_aware_probe_args. This models the per-sub-request URL that filters.chunked will actually emit — the filter chunker can shrink the filter down to one clause per sub-request but no smaller (it bails if any clause exceeds the budget). Without this coordination, a long OR-filter combined with multi-value lists triggered premature RequestTooLarge exceptions even when the combined chunkers would have made things fit.
• Hard cap (max_chunks, default 1000): the cartesian product won't exceed this. Note: this caps the count of chunks, not the count of HTTP requests, because each chunk may paginate (each pagination is a separate HTTP request that counts against the hourly quota).
• Quota-aware abort (quota_safety_floor, default 50): after every sub-request the wrapper reads x-ratelimit-remaining. If it drops below the floor, the wrapper raises QuotaExhausted carrying the combined partial frame, the chunk offset, and the last-observed remaining — letting callers either salvage what they have or resume from the next chunk after the hourly window resets. This is critical because of the pagination silent-truncation bug fixed in Fix silent / misleading failures in paginated waterdata pagination #273; without quota awareness the chunker could otherwise drive the API into 429 territory mid-call and (pre-Fix silent / misleading failures in paginated waterdata pagination #273) silently lose pagination pages.

Failure modes (post-fix)

Scenario	Result
URL fits without chunking	Decorator passes through; one request, no overhead beyond a single planning probe
Lists need splitting	Greedy halve → cartesian product of N sub-requests
Filter chunking also needed	Outer mv-loop × inner filter-loop; identical-row dedup at combine
Filter has clauses bigger than achievable budget	RequestTooLarge with explicit message
Cartesian product > max_chunks	RequestTooLarge with the actual count
x-ratelimit-remaining drops below floor mid-call	QuotaExhausted with .partial_frame, .completed_chunks, .total_chunks

Coordination empirically verified

Validated against api.waterdata.usgs.gov (worktree contains the three live test scripts):

Test	Setup	Result
3-dim equivalence	4 sites × 3 pcodes × 3 stats, url_limit=220	16 sub-requests, every axis confirmed split, chunked frame exactly equal to unchunked reference
Stress (6 edge cases)	Singletons, lopsided sizes, monitoring-locations POST path, RequestTooLarge floor, etc.	All pass
Filter/mv composition (3 regimes)	mv-only / filter-only / both with previously-failing url_limit=220	All pass — frames match references; the "both" case would have raised RequestTooLarge without the filter-aware probe

Test plan

• Unit tests for _filter_aware_probe_args (8 cases)
• Unit tests for _plan_chunks greedy halving, RequestTooLarge floor, coordination with filter chunker, max_chunks cap
• Unit tests for multi_value_chunked pass-through, cartesian product shape, lazy URL-limit reads, max_chunks override
• Unit tests for QuotaExhausted machinery (8 cases: header parsing, default floor, mid-call abort with partial frame, last-chunk-no-abort, zero-floor disable, message contents)
• Defensive: filter and filter_lang excluded from _NEVER_CHUNK so a caller passing filter as a list can't produce malformed CQL
• Live: 3-dim equivalence, 6-case stress matrix, 3-regime filter/mv composition all green against production API
• No regression: existing _construct_api_requests_* and _check_* tests still pass

30 new + existing tests; runs offline (uses a deterministic fake build_request). Live scripts in /tmp of the development worktree, not in the test suite.

Merge ordering

• Fix silent / misleading failures in paginated waterdata pagination #273 first (silent-truncation fix in _walk_pages). This chunker amplifies the frequency of the latent silent-truncation bug — without Fix silent / misleading failures in paginated waterdata pagination #273, every paginated sub-request inside the cartesian product becomes a chance for silent data loss when the hourly quota is exhausted. The chunker's QuotaExhausted guard mitigates this somewhat by aborting before the 429, but it's belt-and-suspenders, not a substitute.
• Use GET with comma-separated values for multi-value waterdata queries #233 next (GET-with-commas routing). This chunker layers on top of Use GET with comma-separated values for multi-value waterdata queries #233's GET routing.
• This PR last.

If the team prefers to land in a different order, this PR can ship without #273 — but the chunker should be considered "production-ready" only once #273 is merged.

Files changed

• dataretrieval/waterdata/chunking.py (new) — decorator, planner, RequestTooLarge, QuotaExhausted
• dataretrieval/waterdata/utils.py — wire @chunking.multi_value_chunked outside @filters.chunked on _fetch_once; updated docstring
• tests/waterdata_test.py — 30 new tests covering planner, coordination, cap, quota machinery

Public API additions

• dataretrieval.waterdata.chunking.RequestTooLarge — raised when planning can't make the URL fit
• dataretrieval.waterdata.chunking.QuotaExhausted — raised mid-call when the quota safety floor is breached; carries .partial_frame, .partial_response, .completed_chunks, .total_chunks, .remaining
• dataretrieval.waterdata.chunking.multi_value_chunked — decorator, in case downstream wraps a custom _fetch_once
• Module constants _DEFAULT_MAX_CHUNKS = 1000, _DEFAULT_QUOTA_SAFETY_FLOOR = 50 (underscored — not part of the stable public API; available for monkey-patching by users with non-default quotas)

Known limitations (deferred)

• The cap counts chunks, not HTTP requests. Paginated sub-requests each make multiple HTTP calls; a chunked call with 1000 chunks where each paginates 5× = 5000 HTTP requests. Document workaround: lower max_chunks proportionally for paginating workloads, or rely on the QuotaExhausted abort to catch overruns at runtime.
• No auto-retry / no sleep-until-reset on QuotaExhausted. Caller decides. Could add on_quota_exhausted="sleep" mode in a follow-up if desired.
• Sequential sub-requests. Parallelism would reduce wall time but complicates rate-limit handling.

🤖 Generated with Claude Code