Skip to content

feat(autoresearch): improve setup and run observability#3355

Merged
fercgomes merged 3 commits into
mainfrom
posthog-code/autoresearch-dashboard-ux
Jul 10, 2026
Merged

feat(autoresearch): improve setup and run observability#3355
fercgomes merged 3 commits into
mainfrom
posthog-code/autoresearch-dashboard-ux

Conversation

@fercgomes

@fercgomes fercgomes commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • redesign the Autoresearch setup input with clearer metric direction, examples, help content, feedback contact, and accept edits defaults
  • surface streamed research findings, hypotheses, iteration plans, approach labels, live activity, context usage, time breakdowns, and completed run summaries
  • record completed metric reports while the agent is still working, including composer runs whose initial prompt is already active
  • prioritize iteration metrics and results in the dashboard hierarchy
  • add dedicated and configurable Storybook coverage for the full dashboard and individual states
Screenshot 2026-07-10 at 16 44 11 Screenshot 2026-07-10 at 16 44 17 Screenshot 2026-07-10 at 16 32 43 Screenshot 2026-07-10 at 16 32 25 Screenshot 2026-07-10 at 16 02 03

Validation

  • 155 core Autoresearch tests
  • 25 UI Autoresearch tests
  • core and UI typechecks
  • static Storybook build
  • Biome and diff checks

Created with Autoresearch.


Created with PostHog Code

Generated-By: PostHog Code
Task-Id: 0ccd57ce-3e49-42eb-8de2-5b03465d559d
@trunk-io

trunk-io Bot commented Jul 10, 2026

Copy link
Copy Markdown

😎 Merged manually by @fercgomes - details.

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown

React Doctor found no issues in the changed files. 🎉

Reviewed by React Doctor for commit 9339e76.

@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "feat(autoresearch): improve setup and ru..." | Re-trigger Greptile

@fercgomes fercgomes requested a review from pauldambra July 10, 2026 19:45

if (session.isPromptPending) {
if (promptCount === 0) continue;
this.ingestCompletedReports(run, session);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Streaming Reports Become Iterations

When a turn is still streaming, this call parses any complete autoresearch block already present and recordMetricReport() immediately appends an iteration and can complete the run. If the agent later corrects the metric or emits the final report in the same turn, the dashboard records an interim value as a real iteration, burns the iteration budget, and can stop the run before the final answer arrives.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 69f144d. Streaming now treats the tail report as provisional. A report is recorded mid-turn only after the transcript explicitly starts the next iteration, and the final unresolved report is committed only when the turn ends. Target and iteration-budget completion decisions also run only at turn completion. Added regression coverage for corrected tail reports and multi-iteration streaming.

endedAt: number | null,
now: number,
): AutoresearchActivitySnapshot {
const relevant = events.filter((event) => event.ts >= startedAt);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Historical Runs Include Later Activity

When a completed run is selected, AutoresearchPanel still passes the current task session events into this analyzer. Since the filter only checks event.ts >= startedAt, tool calls and agent chunks from a newer run or later manual chat are included in the old run's activity list, time breakdown, and parsed plan.

Suggested change
const relevant = events.filter((event) => event.ts >= startedAt);
const relevant = events.filter(
(event) => event.ts >= startedAt && (endedAt === null || event.ts <= endedAt),
);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 69f144d. Historical activity is now bounded to events between the run start and end timestamps. Added a regression test proving later manual activity is excluded from the historical run timeline and time breakdown.

@fercgomes fercgomes requested a review from a team July 10, 2026 19:45
@fercgomes

Copy link
Copy Markdown
Contributor Author

/trunk merge

@fercgomes

Copy link
Copy Markdown
Contributor Author

/trunk cancel

Generated-By: PostHog Code
Task-Id: 0ccd57ce-3e49-42eb-8de2-5b03465d559d
@fercgomes

Copy link
Copy Markdown
Contributor Author

/trunk merge

@fercgomes

Copy link
Copy Markdown
Contributor Author

/trunk cancel

Generated-By: PostHog Code
Task-Id: 0ccd57ce-3e49-42eb-8de2-5b03465d559d
@fercgomes fercgomes enabled auto-merge (squash) July 10, 2026 20:46
@fercgomes fercgomes merged commit 88efc3f into main Jul 10, 2026
26 checks passed
@fercgomes fercgomes deleted the posthog-code/autoresearch-dashboard-ux branch July 10, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants