Skip to content

feat(onboard): mine reviews across the whole window in busy repositories - #685

Merged
theCodeDrift merged 2 commits into
mainfrom
feat/onboard-review-mining
Nov 12, 2026
Merged

theCodeDrift merged 2 commits into
mainfrom
feat/onboard-review-mining

Conversation

@theCodeDrift

Copy link
Copy Markdown
Member

Onboarding's review-comment source now covers the whole 90-day window in busy repositories. Took the recipe-guidance route from the issue rather than a new mine-reviews command (that could be a follow-up).

  • prompts/recipes.ts: names the 1000-result cap on gh pr list/gh search; counts merged PRs via the search API's total_count; lists merged PRs in date chunks; fetches every inline review comment for the window with one gh api --paginate "repos/{owner}/{repo}/pulls/comments?since=START"; samples every Nth PR across the window (counts from a sample are a floor); no hand-rolled pagination loops. GitLab gets the same via glab api --paginate.
  • agent/onboard.md step 7 (topic v11 → v12): record counted PRs/comments per candidate while mining, and recount if route narrows a candidate.
  • openspec/specs/cli-onboard/spec.md: requirement text and two scenarios, edited in place following the onboard recipe: end the final message with the report link #555 precedent.

Tests: 5 new in host-tools.test.ts, 1 new in onboard.test.ts. pnpm lint, pnpm typecheck, pnpm test (3168 passed), openspec validate --specs --strict all pass. The gh commands were run against taskless/cli; the glab commands are untested.

Note: #675's PR also bumps the onboard topic to v12; whichever merges second goes to v13.

Commits are unsigned: the signing key couldn't be unlocked remotely, and the owner chose to ship unsigned.

Closes #674

@theCodeDrift

Copy link
Copy Markdown
Member Author

@claude /review

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @theCodeDrift's task in 2m 51s —— View job


Review complete

Review mode: incremental — read 0 prior review thread(s) before reviewing. .prior-review.json contains no resolved/unresolved review threads and no prior review bodies — only the trigger comment itself (@claude /review). This is therefore the first substantive review of this PR; nothing to carry forward as addressed or still-open.

  • Read prior review data
  • Fetch PR diff and metadata
  • Read changed files in full context (recipes.ts around reviewCommentSource/indentBlock/variable wiring, onboard.md step 7, cli-onboard/spec.md)
  • Analyze correctness, security, performance, tests, clarity
  • Post inline comments for findings
  • Post final summary

Summary

I found no bugs in the GitHub-side logic. Traced the whole chain carefully since none of it can be exercised by the test suite (it's prose the agent follows, not executable code):

  • The search/issues?q=...is:merged...&per_page=1 --jq .total_count count is correct — the Search API's total_count reflects the true match count even past the 1000-result pagination ceiling, so it's a valid way to decide whether chunking is needed.
  • gh pr list --state merged --search "merged:START..END" --limit 1000 is valid gh search syntax for a date-bounded merged-PR listing.
  • repos/{owner}/{repo}/pulls/comments?since=START is the real "list review comments in a repository" endpoint, it does support since, and it correctly returns comments across all PRs (open and closed) — the comment's caveat to cross-reference against the merged-PR list before counting is necessary and present.
  • The {owner}/{repo} placeholder explanation (filled in only inside the endpoint argument, not in -f) is accurate and explains why the query rides in the URL instead of as a separate field.
  • indentBlock(reviewCommentSource(...), " ") lines up with how SOURCE_PR_REVIEW is substituted into the numbered step in onboard.md (packages/cli/src/prompts/recipes.ts:1111-1114), and the nested bullets/code fences inside GITHUB_REVIEW_WINDOW/GITLAB_REVIEW_WINDOW indent far enough to stay inside the list item — matches the dedicated indentation test.
  • onboard.md step 7's new "count while you mine" paragraph (packages/cli/src/agent/onboard.md:153-157) doesn't conflict with the existing "note how many PRs/comments raised each candidate" instruction a few lines above it — it's a refinement of when to record, not a duplicate rule.

[New] One real gap, self-disclosed but worth restating plainly: the PR body says "the glab commands are untested," and the GitLab path rests on an unverified assumption. GITLAB_REVIEW_WINDOW (packages/cli/src/prompts/recipes.ts:850-855) filters merged MRs with updated_after=START and then keeps only those whose merged_at falls in the window, because GitLab's merge-requests list endpoint has no merged_after/merged_before parameter. That's the right workaround if merging an MR always bumps updated_at to at or after the merge time — which is true in the normal case but I can't rule out GitLab instance/edge-case behavior (e.g., imported/migrated MRs where updated_at wasn't touched by the merge event) without being able to run glab against a real GitLab repo myself. Given the author already flagged this as untested, I'd suggest a smoke test against a real GitLab repo before relying on this in production, same as was done for the gh side.

No other correctness, security, performance, or clarity issues found. Test coverage (5 new cases in host-tools.test.ts, 1 in onboard.test.ts) matches the new behavior described in the diff.
(branch)

@theCodeDrift

Copy link
Copy Markdown
Member Author

Re: @github-actions — "Claude finished @theCodeDrift's task in 2m 51s … One real gap: the GitLab path rests on an unverified updated_after assumption"
#685 (comment)

GitLab's merge-request list endpoint does take merged_after, so 9fb73c4 filters on it directly and drops the reliance on a merge bumping updated_at. The merged_at check stays as a guard for an older instance that ignores the parameter, and host-tools.test.ts now pins the filter. A live glab smoke test against a real GitLab repo is still outstanding.

— AI Coding Agent

The onboarding recipe now names the 1000-result cap on gh pr list and gh
search, gives date-chunked listing and one paginated call for every inline
review comment in the window, and samples every Nth PR across the window
rather than the newest N. Step 7 records counted evidence per candidate so
`surfaced` can be recounted when route narrows a candidate.

Closes #674
…fter

GitLab's merge-request list takes merged_after, so the recipe no longer
leans on a merge always bumping updated_at. The merged_at check stays as
a guard for an older instance that ignores the parameter.
@theCodeDrift
theCodeDrift force-pushed the feat/onboard-review-mining branch from 9fb73c4 to 2bbb91c Compare November 12, 2026 04:56
@theCodeDrift
theCodeDrift merged commit 55b15e5 into main Nov 12, 2026
4 checks passed
@theCodeDrift
theCodeDrift deleted the feat/onboard-review-mining branch November 12, 2026 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

onboard: review mining for busy repositories (1000-result cap, pagination, sampling)

1 participant