Enhance Copilot Review Analyst skill: v3 newsletter report + adoption-signal & fairness methodology, Fixes AB#3696013 - #449
Conversation
…-signal & fairness methodology, Fixes AB#3696013 Report / rendering: - v3 newsletter Outlook report: 1200px hybrid-fluid shell, edge-to-edge newsletter masthead (inset hero + 1120px VML), big-number KPI hero grid, single-row meta-chips, Methodology section - Headline now leads with adoption % (Helpful & adopted), not precision - Document Outlook rendering learnings in report-formatting.md: single-row chip table (not inline-block), mso-line-height-rule:exactly for prose, edge-to-edge frame (whiten bgcolor AND inline CSS), inset masthead + VML width, Graph-draft delivery with Outlook COM fallback Classification / methodology: - Declined verdict + Copilot precision (Helpful / (Helpful + Incorrect)); Declined is neutral - Force-push confound validation before scoring precision - Engineer-intention nuance: correct-but-declined feedback is not a Copilot error - New adoption signals: reaction + apply-suggestion detection, merged-only PR filter, Phase 0 preflight coverage audit, review-coverage split, self-consistency gate, PR triage table Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4fbe65f-9d71-4531-b63e-b5a12b21fba4
github-actions
Bot
changed the title
|
✅ Work item link check complete. Description contains link AB#3696013 to an Azure Boards work item. |
1 similar comment
|
✅ Work item link check complete. Description contains link AB#3696013 to an Azure Boards work item. |
…-signal & fairness methodology, Fixes AB#3696013 Report / rendering: - v3 newsletter Outlook report: 1200px hybrid-fluid shell, edge-to-edge newsletter masthead (inset hero + 1120px VML), big-number KPI hero grid, single-row meta-chips, Methodology section - Headline now leads with adoption % (Helpful & adopted), not precision - Document Outlook rendering learnings in report-formatting.md: single-row chip table (not inline-block), mso-line-height-rule:exactly for prose, edge-to-edge frame (whiten bgcolor AND inline CSS), inset masthead + VML width, Graph-draft delivery with Outlook COM fallback Classification / methodology: - Declined verdict + Copilot precision (Helpful / (Helpful + Incorrect)); Declined is neutral - Force-push confound validation before scoring precision - Engineer-intention nuance: correct-but-declined feedback is not a Copilot error - New adoption signals: reaction + apply-suggestion detection, merged-only PR filter, Phase 0 preflight coverage audit, review-coverage split, self-consistency gate, PR triage table Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4fbe65f-9d71-4531-b63e-b5a12b21fba4
…-microsoft-copilot-review-analysis
…-signal & fairness methodology, Fixes AB#3696013 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4fbe65f-9d71-4531-b63e-b5a12b21fba4
…-microsoft-copilot-review-analysis
Removes the AuthClientAndroidPipelines and telemetry-platform ignore entries that were inadvertently carried over from earlier branch work; they are not part of the copilot-review-analyst skill changes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4fbe65f-9d71-4531-b63e-b5a12b21fba4
|
PR Body mentioned a .gitignore addition, but i'm not seeing it in the files, is it stale? |
append-history.ps1 computed $unknownCount but never wrote it to the
snapshot, so whenever any comment stayed 'unknown' the additive buckets
(helpful + incorrect + declined + unresolved) summed to less than total.
That made history.json inconsistent with final-classification.ps1, which
treats 'unknown' as a canonical bucket in its sum-to-total gate.
- Add unknown = { count, pct } to the emitted snapshot (after unresolved)
- Compute $unknownPct alongside the other bucket percentages
- Print an Unknown line in the console summary for parity
- Add a self-consistency assertion that warns if the five canonical
additive buckets do not reconcile to total
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d4fbe65f-9d71-4531-b63e-b5a12b21fba4
|
❌ Work item link check failed. Description does not contain AB#{ID}. Click here to Learn more. |
|
@fadidurah you're right, it was stale. The |
| @@ -54,102 +56,109 @@ if (Test-Path $precisePath) { | |||
| $precise = Get-Content $precisePath -Raw | ConvertFrom-Json | |||
There was a problem hiding this comment.
$precise / precise.json is now dead code.
This PR removed the $preciseMap re-derivation , but precise.json is still loaded into $precise here and never referenced again anywhere in the file. The .SYNOPSIS and .PARAMETER InputDir also still say the script "Reads final_classification.json and precise.json."
Suggest dropping the load and the $precisePath line, and updating the doc comments, so the script no longer implies a dependency it doesn't have.
There was a problem hiding this comment.
You're right — dead code. Removed in 60f214c: dropped both $precisePath and the $precise load block, and updated .DESCRIPTION + .PARAMETER InputDir to reference only final_classification.json.
I kept the one rationale comment noting we deliberately don't re-derive silent-helpful from precise.json — that documents the absence of the dependency rather than implying one, so it shouldn't be misleading. Verified the script runs end-to-end with no precise.json present and buckets still reconcile to total.
An earlier change dropped the $preciseMap re-derivation, but precise.json was still loaded into $precise (and $precisePath computed) and never referenced again. The .SYNOPSIS/.PARAMETER InputDir doc comments also still claimed the script reads precise.json, implying a dependency it no longer has. - Drop $precisePath and the $precise load block - Update .DESCRIPTION and .PARAMETER InputDir to mention only final_classification.json The design-rationale comment noting we deliberately do NOT re-derive silent-helpful from precise.json is kept — it documents the absence of the dependency rather than implying one. Verified the script runs end-to-end with no precise.json present and buckets still reconcile to total. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: d4fbe65f-9d71-4531-b63e-b5a12b21fba4
3 participants
Summary
Bakes this cycle's learnings into the copilot-review-analyst skill (
.github/skills/copilot-review-analyst). Two themes: a redesigned v3 newsletter report + hardened Outlook rendering, and a stronger, fairer classification methodology.Fixes AB#3696013
Report / rendering
assets/…-Outlook.html): 1200px hybrid-fluid shell, edge-to-edge newsletter masthead (inset hero + 1120px VML width), big-number KPI hero grid, single-row meta-chips, and a Methodology section. Committed as the anonymized sample, single-quoted (escaping-free).references/report-formatting.md§ v3 Newsletter Rendering:inline-blocktables stack vertically in classic Outlook.line-heightplusmso-line-height-rule:exactlyor the Word engine collapses leading.bgcolorand inline CSS (Word engine readsbgcolor).Classification / methodology
Helpful / (Helpful + Incorrect); Declined is neutral and never counted against Copilot.Notes
.md/.htmlassets remain structural references (illustrative prior-run data — computed fresh each run).