fix(content-drive): global search hides matches without content-type filter (#37211)

[ihoffmann-dot]

Summary

PR 2 (implementation) for the issue-resolution spec approved and merged in #37525.

Fixes #37211 — Content Drive's unfiltered global search silently drops matches that the same term finds once a content-type filter is applied, because the DB-order chunked scan (BrowserAPIImpl.getContentByChunks) terminated on a hard row-count cutoff (BROWSER_DB_MAX_SCAN_ROWS) instead of match coverage, while the legacy Search All portlet (ESContentResourcePortlet, querying ES directly) does not have this gap.

Fix: stays DB-first per accepted ADR-0018 (DB drives candidate order/pagination; index only narrows) — the ES-narrowed scan now bounds cost by elapsed time (BROWSER_DB_MAX_SCAN_TIME_MILLIS, new) instead of row count, so it keeps scanning while still affordable instead of giving up at an arbitrary row count. The permission-only scan path is unchanged (row-count cutoff, no completeness gap to fix).

Reconciled with #37395 (already merged to main), which touches the same getContentByChunks method (chunk-size clamping + exit-order fix) — this branch has been merged with main to integrate both.

/speckit-converge reports converged against the current code (0 findings) — see research.md/plan.md in specs/37418-content-drive-global/ for the full ADR-driven rationale.

Test plan

• TDD gate: tests written, developer-approved, confirmed failing (Red) before implementation (Principle V)
• Integration tests in BrowserAPITest covering: unfiltered global search finding a match past the (lowered, for test purposes) scan limit (AC-001); content-type filter only narrowing, never adding, matches (AC-002); scan-cost regression guard (bounded elapsed time)
• Verified no regression against the two pre-existing scan-limit tests (scanLimitStopsLoop, scanLimitAlignedWithChunkBoundary_doesNotSkipLeftoverItems from fix(content-drive): resolve single-pass-eligible field filters in one scan (#37184) #37395) — all 5 tests green together
• /speckit-converge run against final code — converged, 0 findings

⚠️ Pending post-merge (not blocking, tracked)

• Manual verification against a large/large-like dataset (~718,174 contentlets): AC-003 (Content Drive vs. legacy Search All equivalence) and AC-004 at real scale — see quickstart.md. This is the one item /speckit-converge explicitly carves out as a sanctioned manual-only exception (no automated harness exists for Search All ↔ Content Drive comparison, and seeding real-world scale in an integration test is impractical) — tracked as task T019 in specs/37418-content-drive-global/tasks.md. Will follow up once a suitable large dataset/environment is available.

🤖 Generated with Claude Code

[claude]

Claude finished @ihoffmann-dot's task in 2m 47s —— View job

---

Code Review — #37211 content-drive global search scan bound

• Read the diff (BrowserAPIImpl.java, BrowserAPITest.java, spec.md)
• Trace the scan loop / budget logic for introduced bugs
• Verify Config.getLongProperty, TimeUnit import, config defaults
• Review new integration tests for correctness & flakiness

---

I reviewed the change against origin/main. The core fix — bounding the ES-narrowed scan by elapsed time (BROWSER_DB_MAX_SCAN_TIME_MILLIS) plus a co-bound row hard cap (BROWSER_DB_MAX_SCAN_ROWS_ES_HARD_CAP), while leaving the permission-only path on the original row-count cutoff — is implemented correctly and reads cleanly.

Things I specifically verified:

• Config.getLongProperty exists (Config.java:582) and java.util.concurrent.TimeUnit is imported (BrowserAPIImpl.java:93). ✅
• The new bounds are correctly gated behind applyESFilter; on the permission-only path scanTimeBudgetMillis/esRowHardCap are -1 but the applyESFilter && guards on lines 394–396 prevent them from ever being read as a live cutoff. ✅
• Exit-order precedence is preserved: satisfied-page (:362) and natural DB exhaustion (:376) still win over the budget check (:401), so the Content Drive: field filter re-scans the folder candidate query up to 4x per request #37184 chunk-boundary/cursor-skip fixes aren't regressed. ✅
• Loop always terminates on the ES path — DB exhaustion, time budget, or the 1M-row hard cap. No unbounded-scan path introduced. ✅
• Traced the four new tests by hand (chunk-size 5, scan/cap 15, filler 20) — cursor and hasMoreContent expectations line up with the loop's actual behavior. ✅

New Issues

• 🟡 Medium: dotcms-integration/.../BrowserAPITest.java:288 — test_getPaginatedContents_timeBudgetExhausted_resumesFromCursor uses a 1ms time budget to force the cutoff after the first chunk. Assumption: the budget is only checked after a full chunk is fetched + ES-filtered + permission-filtered (:394), and any real DB+ES round trip comfortably exceeds 1ms, so firstPage.hasMoreContent == true holds reliably. What to verify: if a future change ever short-circuits the ES call (e.g. an in-memory/cached chunk path), a sub-millisecond first chunk could let the whole fixture scan complete and flip this assertion, silently. Low risk today; noting it because the assertion's stability rests on wall-clock timing rather than a deterministic condition. Not blocking.

Everything else is sound. The best-effort/load-dependent completeness trade-off is explicitly documented in the Javadoc (:270) and comments, which is the right call given ADR-0018's DB-first constraint. The row hard cap default (1,000,000) comfortably covers the ~718k-contentlet target scale, so it doesn't reintroduce the original completeness gap at that scale.

The one carve-out remains the manual large-dataset verification (AC-003/AC-004, tracked as T019) — correctly flagged as out of automated scope in the PR body.

No blocking issues. 👍

· branch issue-37211-content-drive-global-search-filter