diff --git a/README.md b/README.md index 251655c..4802c86 100644 --- a/README.md +++ b/README.md @@ -43,9 +43,9 @@ npx trace-engine-cli /path/to/trace.json.gz ### How Agents Discover Capabilities 1. **Default overview + built-in drill-down hints (`trace-engine-cli `)**: - Calling `trace-engine-cli ` with no subcommand runs `overview` and appends a `# Drill-down commands:` reference footer to the output. That first call gives the agent the trace bounds, Core Web Vitals, main-thread category breakdown, and the exact syntax for every follow-up subcommand. + Calling `trace-engine-cli ` with no subcommand runs `overview` and appends a `# Drill-down commands:` reference footer to the output. That first call gives the agent the trace bounds, Core Web Vitals, main-thread category breakdown, and syntax for follow-up commands. 2. **Top-level command discovery (`trace-engine-cli` or `trace-engine-cli --help`)**: - Running `trace-engine-cli` with no arguments or passing `--help` prints all 13 subcommands (`overview`, `insights`, `tasks`, `find`, `profile`, `diff`, `scheduling`, `network`, `interactions`, `shifts`, `events`, `navigations`, `scan`) and global options. + Running `trace-engine-cli` with no arguments or passing `--help` prints the available commands and global options. 3. **Subcommand flag discovery (`trace-engine-cli --help`)**: Running `--help` on any subcommand (for example, `trace-engine-cli insights --help` or `trace-engine-cli events --help`) prints the flags, choices, and defaults for that command. @@ -57,7 +57,8 @@ Add this snippet to your project instruction file (`AGENTS.md`, `CLAUDE.md`, or ## Performance Trace Analysis (`.json`, `.json.gz`) - Never parse or search trace files manually with `jq`, `python`, `node -e`, `zcat`, or shell loops. Use `trace-engine-cli`, which handles gzip decompression, DevTools Trace Engine parsing, and token-compact TOON formatting. -- Start any single-trace investigation by running `trace-engine-cli `. Read the overview and the `# Drill-down commands:` footer appended to the output to choose follow-up commands. +- Start an open-ended trace investigation with `trace-engine-cli `. Use its drill-down suggestions to choose follow-up commands. +- When you already know which analyses you need for one trace, run `trace-engine-cli report --sections ` to parse it once. - To search across multiple trace files or directories, run `trace-engine-cli scan [options]`. - Run `trace-engine-cli --help` or `trace-engine-cli --help` to inspect available subcommands and flags. ``` @@ -105,6 +106,15 @@ High-level trace metadata, duration, Core Web Vitals (LCP, INP, CLS), and main-t trace-engine-cli trace.json.gz ``` +### `trace-engine-cli report ` +When you know which analyses a question needs, select them with `--sections` +so they share one trace parse. Run `trace-engine-cli report --help` to see the +available sections. + +```bash +trace-engine-cli report trace.json.gz --sections overview,shifts,network +``` + ### `trace-engine-cli insights ` Diagnostic insight cards for Core Web Vitals and performance bottlenecks. ```bash @@ -227,6 +237,10 @@ trace-engine-cli scan fixtures/traces/ --cat devtools.timeline --phase X 3. Inspect relevant subsystem: `trace-engine-cli shifts --events`, `trace-engine-cli network `, or `trace-engine-cli insights --category ` 4. Inspect raw event payloads: `trace-engine-cli events --name --fields ` +### Recipe D: Analyzing CPU Activity in a Subframe +1. Run `trace-engine-cli threads ` and find the renderer thread by frame URL. +2. Pass its `PID:TID` as `--thread` to a CPU command, for example `trace-engine-cli tasks --thread 123:1`. Frames that share a renderer thread share its CPU results. + --- ## 5. Capture Caveats @@ -285,6 +299,10 @@ All time-range inputs (`--time-range`, `--window-a`, `--window-b`, `--bucket`) s ### 1. `overview` (default) High-level trace metadata, Core Web Vitals (FCP, LCP, CLS, INP), non-overlapping main-thread category breakdown (Scripting, Layout, Rendering, GC, System, Idle), and diagnostic warnings. +The breakdown uses the same instrumented timeline events with and without CPU +samples. With `--thread PID:TID`, the breakdown describes that +renderer thread and Web Vitals still describe the page. + ```bash # Default invocation (includes the drill-down commands footer) trace-engine-cli trace.json.gz @@ -325,13 +343,15 @@ mainThreadBreakdown: warnings[1]: 1 render-blocking resource(s) detected. # Drill-down commands: +# - trace-engine-cli report [--sections ] # - trace-engine-cli insights [--category ] [--navigation ] -# - trace-engine-cli tasks [--min-duration ] [--tree ] [--top ] -# - trace-engine-cli find --function [--time-range ] +# - trace-engine-cli tasks [--min-duration ] [--tree ] [--url-only] [--top ] +# - trace-engine-cli find [--function ] [--url ] [--time-range ] +# - trace-engine-cli threads # - trace-engine-cli profile [--time-range ] [--bucket ] # - trace-engine-cli diff --window-a --window-b # - trace-engine-cli scheduling [--time-range ] -# - trace-engine-cli network [--render-blocking] [--slowest ] [--initiator-chain ] +# - trace-engine-cli network [--render-blocking] [--slowest ] [--initiator-chain ] [--strip-query] # - trace-engine-cli interactions # - trace-engine-cli shifts [--events] # - trace-engine-cli events [--name ] [--cat ] [--fields ] @@ -346,6 +366,10 @@ warnings[1]: 1 render-blocking resource(s) detected. ### 2. `insights` Extracts actionable diagnostic insight cards generated by Chrome DevTools Insight models. +The default is a main-frame navigation with a measured LCP when available. +Use `--navigation` to select a trace navigation ID or InsightSet ID. +With `--time-range`, insights select from navigation windows that overlap the range. + ```bash # All insights trace-engine-cli insights trace.json.gz @@ -365,7 +389,9 @@ trace-engine-cli insights trace.json.gz --navigation 9E74B39B7C378297071F6E3618C --- ### 3. `tasks` -Main thread task execution, long tasks, and aggregated call trees. +Long tasks and aggregated call trees for a renderer thread. Aggregated rows +show their first event timestamp. Top-down rows include the call path, and +`--url-only` limits output to rows with source URLs. ```bash # List long tasks above 50ms (default) @@ -374,8 +400,8 @@ trace-engine-cli tasks trace.json.gz # Custom duration threshold and limit within relative window trace-engine-cli tasks trace.json.gz --min-duration 20 --time-range "1s,4s" --top 10 -# Bottom-up call tree (aggregated and sorted by self-time) -trace-engine-cli tasks trace.json.gz --tree bottom-up --top 20 +# Bottom-up call tree with source URLs, sorted by self-time +trace-engine-cli tasks trace.json.gz --tree bottom-up --url-only --top 20 # Top-down call tree (aggregated and sorted by total duration) trace-engine-cli tasks trace.json.gz --tree top-down --top 20 @@ -384,14 +410,15 @@ trace-engine-cli tasks trace.json.gz --tree top-down --top 20 --- ### 4. `find` -Search for executions of a specific function name or pattern to verify call frequency, duration, or confirm starvation (0 calls in window). +Search executions by function name or script URL substring to inspect call +frequency and duration. Pass both `--function` and `--url` to require both matches. ```bash # Search for invocations of a function across the whole trace trace-engine-cli find trace.json.gz --function "drawEventTiles" -# Query within a specific relative time window -trace-engine-cli find trace.json.gz --function "drawEventTiles" --time-range "3.5s,4.2s" +# Search calls in one script within a relative time window +trace-engine-cli find trace.json.gz --url "/app.js" --time-range "3.5s,4.2s" ``` --- @@ -434,6 +461,9 @@ trace-engine-cli scheduling trace.json.gz --time-range "3.5s,4.1s" ### 8. `network` Network request waterfall, priority, timings (Queueing, DNS, SSL, TTFB, Download), and request initiator chains. +Use `--strip-query` to remove query strings from request and initiator URLs in +output. Initiator-chain matching still uses the original URLs. + ```bash # Top 20 slowest requests trace-engine-cli network trace.json.gz --slowest 20 @@ -459,6 +489,10 @@ trace-engine-cli interactions trace.json.gz ### 10. `shifts` Cumulative Layout Shift (CLS) session window clusters, cumulative scores, and culprit DOM elements causing visual instability. Pass `--events` to include individual shift event payloads, navigation IDs, and impacted DOM nodes. +For shifted nodes, the CLI reports recorded backend node IDs, renderer process and +frame, and bounding rectangles. It also shows any DOM names and selectors +present in the recording. + ```bash # Cluster summary trace-engine-cli shifts trace.json.gz @@ -486,7 +520,7 @@ trace-engine-cli events trace.json.gz --cat devtools.timeline --phase X --top 20 --- ### 12. `navigations` (alias: `navs`) -Inspect all navigations in the trace, their primary main frame status, and their mapped Chrome DevTools Trace Engine `InsightSet` IDs: +Inspect navigations, their primary main frame status, and their mapped Chrome DevTools Trace Engine `InsightSet` IDs. Pass `--time-range` to list starts within a window. ```bash trace-engine-cli navigations trace.json.gz @@ -513,6 +547,37 @@ trace-engine-cli scan fixtures/ --name TracingStartedInBrowser --fields args.dat --- +### 14. `threads` + +List renderer threads, their frame URLs, and CPU sample availability. Use +`--thread PID:TID` with `overview`, `tasks`, `profile`, `find`, or `diff` to +select a renderer, including one for an out-of-process iframe. + +```bash +trace-engine-cli threads trace.json.gz +``` + +--- + +### 15. `report` + +Run selected analyses from one trace parse. Each section uses its standalone +command's defaults. + +| Option | Description | +| --- | --- | +| `--sections ` | Comma-separated section names. Defaults to `overview,tasks,shifts,network`. Use `report --help` to list the available sections. | +| `--time-range ` | Shared time window. Each section interprets it as its standalone command does. | +| `--thread ` | Renderer thread for `overview`, `tasks`, and `profile`. | +| `--strip-query` | Remove query strings from URLs in the `network` section. | + +```bash +trace-engine-cli report trace.json.gz +trace-engine-cli report trace.json.gz --sections overview,insights,tasks --format md +``` + +--- + ## Output Formats ### TOON (Default) diff --git a/src/cli.ts b/src/cli.ts index f088144..c9238da 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -23,11 +23,19 @@ import { extractNavigations } from './engine/extractors/navigations.js'; import { extractNetwork } from './engine/extractors/network.js'; import { extractOverview } from './engine/extractors/overview.js'; import { extractProfile } from './engine/extractors/profile.js'; +import { + DEFAULT_REPORT_SECTIONS, + extractReport, + REPORT_SECTIONS, + type ReportSection, +} from './engine/extractors/report.js'; import { scanTraces } from './engine/extractors/scan.js'; import { extractScheduling } from './engine/extractors/scheduling.js'; import { extractShifts } from './engine/extractors/shifts.js'; import { extractTasks } from './engine/extractors/tasks.js'; +import { listRendererThreads } from './engine/helpers.js'; import { loadTraceFromFile } from './engine/loader.js'; +import type { ThreadsResult } from './engine/types.js'; import { type FormatOptions, formatResult } from './formatters/index.js'; import { parseTimeRange, parseTimeValue, resolveTimeRange } from './utils/filter.js'; @@ -47,6 +55,34 @@ function parseCommaList(val?: unknown): string[] | undefined { return items.length > 0 ? items : undefined; } +const THREAD_OPTION = { + type: 'string', + describe: 'Renderer thread as PID:TID. Use threads to look up a frame URL.', +} satisfies import('yargs').Options; + +function positiveInteger(value: number): number { + if (!Number.isSafeInteger(value) || value < 1) { + throw new Error('Result limits must be positive integers.'); + } + return value; +} + +function parseReportSections(value?: string): ReportSection[] { + if (value === undefined) return DEFAULT_REPORT_SECTIONS; + + const names = parseCommaList(value); + if (!names) throw new Error('--sections must include at least one report section.'); + return [...new Set(names)].map((name) => { + const section = REPORT_SECTIONS.find((supported) => supported === name); + if (!section) { + throw new Error( + `Unknown report section "${name}". Choose from ${REPORT_SECTIONS.join(', ')}.`, + ); + } + return section; + }); +} + export async function runCli(argv: string[]): Promise { const cli = yargs(hideBin(argv)) .scriptName('trace-engine-cli') @@ -77,7 +113,7 @@ export async function runCli(argv: string[]): Promise { ['overview ', '$0 '], 'High-level trace metadata, Core Web Vitals, and main thread category breakdown (default command)', (y) => - y.positional('file', { + y.option('thread', THREAD_OPTION).positional('file', { describe: 'Path to trace file (.json or .json.gz)', type: 'string', demandOption: true, @@ -93,7 +129,7 @@ export async function runCli(argv: string[]): Promise { loaded.parsedTrace.data.Meta.traceBounds, args.absoluteTimestamps as boolean, ); - const result = extractOverview(loaded, { timeRange }); + const result = extractOverview(loaded, { timeRange, threadId: args.thread }); const isDefaultCmd = !args._.includes('overview'); console.log( formatResult(result, { @@ -104,6 +140,63 @@ export async function runCli(argv: string[]): Promise { ); }, ) + .command( + 'report ', + 'Run several analyses with a single parse', + (y) => + y + .positional('file', { + describe: 'Path to trace file (.json or .json.gz)', + type: 'string', + demandOption: true, + }) + .option('sections', { + type: 'string', + describe: `Comma-separated analyses: ${REPORT_SECTIONS.join(', ')} (default: ${DEFAULT_REPORT_SECTIONS.join(',')})`, + }) + .option('thread', THREAD_OPTION) + .option('strip-query', { + type: 'boolean', + describe: 'Remove URL query strings from the network section', + }), + async (args) => { + const sections = parseReportSections(args.sections); + const loaded = await loadTraceFromFile(args.file, args.traceEngine); + const timeRange = resolveTimeRange( + parseTimeRange(args.timeRange), + loaded.parsedTrace.data.Meta.traceBounds, + args.absoluteTimestamps, + ); + const results = extractReport(loaded, { + sections, + timeRange, + threadId: args.thread, + stripQuery: args.stripQuery, + }); + const output = + args.format === 'md' + ? results + .map(([type, result]) => formatResult(result, { format: 'md', type })) + .join('\n\n---\n\n') + : formatResult(Object.fromEntries(results), { format: args.format }); + console.log(output); + }, + ) + .command( + 'threads ', + 'List renderer threads, their frame URLs, and CPU sample availability', + (y) => + y.positional('file', { + describe: 'Path to trace file (.json or .json.gz)', + type: 'string', + demandOption: true, + }), + async (args) => { + const loaded = await loadTraceFromFile(args.file, args.traceEngine); + const result: ThreadsResult = { threads: listRendererThreads(loaded.parsedTrace.data) }; + console.log(formatResult(result, { format: args.format, type: 'threads' })); + }, + ) .command( 'insights ', 'Diagnostic insight cards (LCP, INP, CLS, render-blocking, third-parties)', @@ -129,7 +222,13 @@ export async function runCli(argv: string[]): Promise { args.file as string, args.traceEngine as string | undefined, ); + const timeRange = resolveTimeRange( + parseTimeRange(args.timeRange), + loaded.parsedTrace.data.Meta.traceBounds, + args.absoluteTimestamps, + ); const result = extractInsights(loaded, { + timeRange, category: args.category as | 'lcp' | 'inp' @@ -152,6 +251,7 @@ export async function runCli(argv: string[]): Promise { 'Long tasks, bottom-up and top-down TraceTree call aggregations', (y) => y + .option('thread', THREAD_OPTION) .positional('file', { describe: 'Path to trace file (.json or .json.gz)', type: 'string', @@ -168,9 +268,14 @@ export async function runCli(argv: string[]): Promise { describe: "Generate aggregated call tree ('bottom-up' sorted by self-time, 'top-down' sorted by total duration)", }) + .option('url-only', { + type: 'boolean', + describe: 'Only show rows with a source URL', + }) .option('top', { alias: ['limit'], type: 'number', + coerce: positiveInteger, default: 20, describe: 'Limit output to top N tasks (default: 20)', }), @@ -186,10 +291,12 @@ export async function runCli(argv: string[]): Promise { args.absoluteTimestamps as boolean, ); const result = extractTasks(loaded, { + urlOnly: args.urlOnly, minDurationMs: args.minDuration as number, tree: args.tree as 'bottom-up' | 'top-down' | undefined, top: args.top as number, timeRange, + threadId: args.thread, }); console.log( formatResult(result, { @@ -204,6 +311,7 @@ export async function runCli(argv: string[]): Promise { 'Search for function execution instances, call frequency, and starvation intervals', (y) => y + .option('thread', THREAD_OPTION) .positional('file', { describe: 'Path to trace file (.json or .json.gz)', type: 'string', @@ -212,12 +320,16 @@ export async function runCli(argv: string[]): Promise { .option('function', { alias: ['fn', 'name'], type: 'string', - demandOption: true, - describe: 'Function name or pattern to search for', + describe: 'Function name substring to search for', + }) + .option('url', { + type: 'string', + describe: 'Script URL substring to search for, optionally combined with --function', }) .option('top', { alias: ['limit'], type: 'number', + coerce: positiveInteger, default: 50, describe: 'Maximum number of invocation entries to list (default: 50)', }), @@ -233,8 +345,10 @@ export async function runCli(argv: string[]): Promise { args.absoluteTimestamps as boolean, ); const result = extractFind(loaded, { - functionName: args.function as string, + functionName: args.function, + url: args.url, timeRange, + threadId: args.thread, limit: args.top as number, }); console.log( @@ -250,6 +364,7 @@ export async function runCli(argv: string[]): Promise { 'Time-bucketed CPU profiling progression across discrete intervals', (y) => y + .option('thread', THREAD_OPTION) .positional('file', { describe: 'Path to trace file (.json or .json.gz)', type: 'string', @@ -263,6 +378,7 @@ export async function runCli(argv: string[]): Promise { .option('top', { alias: ['limit'], type: 'number', + coerce: positiveInteger, default: 3, describe: 'Top N functions to include per time bucket (default: 3)', }), @@ -277,9 +393,13 @@ export async function runCli(argv: string[]): Promise { loaded.parsedTrace.data.Meta.traceBounds, args.absoluteTimestamps as boolean, ); - const bucketMs = args.bucket ? parseTimeValue(String(args.bucket)) : undefined; + const bucketMs = args.bucket === undefined ? undefined : parseTimeValue(args.bucket); + if (bucketMs !== undefined && (!Number.isFinite(bucketMs) || bucketMs < 0.01)) { + throw new Error('--bucket must be at least 0.01ms.'); + } const result = extractProfile(loaded, { timeRange, + threadId: args.thread, bucketMs, top: args.top as number, }); @@ -296,6 +416,7 @@ export async function runCli(argv: string[]): Promise { 'Differential CPU profiling comparing two time windows in the trace', (y) => y + .option('thread', THREAD_OPTION) .positional('file', { describe: 'Path to trace file (.json or .json.gz)', type: 'string', @@ -316,6 +437,7 @@ export async function runCli(argv: string[]): Promise { .option('top', { alias: ['limit'], type: 'number', + coerce: positiveInteger, default: 20, describe: 'Maximum differential entries to return (default: 20)', }), @@ -338,6 +460,7 @@ export async function runCli(argv: string[]): Promise { const result = extractDiff(loaded, { windowA, + threadId: args.thread, windowB, top: args.top as number, }); @@ -388,6 +511,10 @@ export async function runCli(argv: string[]): Promise { type: 'string', demandOption: true, }) + .option('strip-query', { + type: 'boolean', + describe: 'Remove query strings from request and initiator URLs in the output', + }) .option('render-blocking', { type: 'boolean', default: false, @@ -396,6 +523,7 @@ export async function runCli(argv: string[]): Promise { .option('slowest', { alias: ['top', 'limit'], type: 'number', + coerce: positiveInteger, default: 20, describe: 'Limit output to N slowest requests (default: 20)', }) @@ -416,6 +544,7 @@ export async function runCli(argv: string[]): Promise { args.absoluteTimestamps as boolean, ); const result = extractNetwork(loaded, { + stripQuery: args.stripQuery, renderBlocking: args.renderBlocking as boolean, slowest: args.slowest as number, initiatorChain: args.initiatorChain as string | undefined, @@ -530,6 +659,7 @@ export async function runCli(argv: string[]): Promise { .option('top', { alias: ['limit'], type: 'number', + coerce: positiveInteger, default: 50, describe: 'Maximum number of matching events to return (default: 50)', }), @@ -576,7 +706,12 @@ export async function runCli(argv: string[]): Promise { args.file as string, args.traceEngine as string | undefined, ); - const result = extractNavigations(loaded); + const timeRange = resolveTimeRange( + parseTimeRange(args.timeRange), + loaded.parsedTrace.data.Meta.traceBounds, + args.absoluteTimestamps, + ); + const result = extractNavigations(loaded, timeRange); console.log( formatResult(result, { format: args.format, @@ -624,6 +759,7 @@ export async function runCli(argv: string[]): Promise { .option('top', { alias: ['limit'], type: 'number', + coerce: positiveInteger, default: 50, describe: 'Maximum number of matched files to return (default: 50)', }), diff --git a/src/engine/extractors/diff.ts b/src/engine/extractors/diff.ts index 02fa0c0..c98bb8d 100644 --- a/src/engine/extractors/diff.ts +++ b/src/engine/extractors/diff.ts @@ -14,11 +14,12 @@ import type { TimeRange } from '../../utils/filter.js'; import { round } from '../../utils/units.js'; -import { getMainRendererThread } from '../helpers.js'; +import { getRendererThread, rendererCpuFilter } from '../helpers.js'; import type { LoadedTrace } from '../loader.js'; import type { DiffEntryItem, DiffResult } from '../types.js'; export interface ExtractDiffOptions { + threadId?: string; windowA: TimeRange; windowB: TimeRange; top?: number; @@ -39,6 +40,7 @@ export function extractDiff(loadedTrace: LoadedTrace, options: ExtractDiffOption const topCount = options.top ?? 20; const result: DiffResult = { + threadId: options.threadId, windowA: { startMs: windowA.startMs, endMs: windowA.endMs, @@ -52,7 +54,7 @@ export function extractDiff(loadedTrace: LoadedTrace, options: ExtractDiffOption entries: [], }; - const mainThread = getMainRendererThread(parsedTrace.data); + const mainThread = getRendererThread(parsedTrace.data, options?.threadId); if (!mainThread?.entries?.length) { return result; } @@ -64,7 +66,7 @@ export function extractDiff(loadedTrace: LoadedTrace, options: ExtractDiffOption } { const bottomUp = new engine.Extras.TraceTree.BottomUpRootNode(mainThreadEntries, { textFilter: { accept: () => true }, - filters: [], + filters: [rendererCpuFilter(loadedTrace)], startTime: engine.Types.Timing.Milli(range.startMs), endTime: engine.Types.Timing.Milli(range.endMs), }); diff --git a/src/engine/extractors/find.ts b/src/engine/extractors/find.ts index 6ce688f..a2aaad9 100644 --- a/src/engine/extractors/find.ts +++ b/src/engine/extractors/find.ts @@ -14,12 +14,14 @@ import { isEventInBounds, type TimeRange } from '../../utils/filter.js'; import { microToMilli, round } from '../../utils/units.js'; -import { getMainRendererThread } from '../helpers.js'; +import { getRendererThread } from '../helpers.js'; import type { LoadedTrace } from '../loader.js'; import type { FindCallInstance, FindIntervalItem, FindResult } from '../types.js'; export interface ExtractFindOptions { - functionName: string; + threadId?: string; + functionName?: string; + url?: string; timeRange?: TimeRange; limit?: number; } @@ -29,12 +31,18 @@ export interface ExtractFindOptions { */ export function extractFind(loadedTrace: LoadedTrace, options: ExtractFindOptions): FindResult { const { engine, parsedTrace } = loadedTrace; - const functionQuery = options.functionName.trim(); + const functionQuery = options.functionName?.trim(); + const urlQuery = options.url?.trim(); + if (!functionQuery && !urlQuery) throw new Error('Specify --function or --url to search for.'); + const functionNeedle = functionQuery?.toLowerCase(); + const urlNeedle = urlQuery?.toLowerCase(); const timeRange = options.timeRange; const limit = options.limit ?? 50; const result: FindResult = { + threadId: options.threadId, functionQuery, + urlQuery, timeRange, totalCalls: 0, totalDurationMs: 0, @@ -42,12 +50,11 @@ export function extractFind(loadedTrace: LoadedTrace, options: ExtractFindOption calls: [], }; - const mainThread = getMainRendererThread(parsedTrace.data); - if (!mainThread || !functionQuery) { + const mainThread = getRendererThread(parsedTrace.data, options?.threadId); + if (!mainThread) { return result; } - const queryLower = functionQuery.toLowerCase(); const matchedCalls: FindCallInstance[] = []; for (const entry of mainThread.entries) { @@ -61,8 +68,12 @@ export function extractFind(loadedTrace: LoadedTrace, options: ExtractFindOption const frame = engine.Extras.TraceTree.eventStackFrame(entry); const fnName = frame?.functionName || entry.name || ''; - if (fnName.toLowerCase().includes(queryLower)) { + if ( + (!functionNeedle || fnName.toLowerCase().includes(functionNeedle)) && + (!urlNeedle || frame?.url.toLowerCase().includes(urlNeedle)) + ) { matchedCalls.push({ + functionName: fnName, timestampMs: round(startMs), durationMs: round(durMs), url: frame?.url, diff --git a/src/engine/extractors/insights.ts b/src/engine/extractors/insights.ts index cfdc1f6..f1ccbbe 100644 --- a/src/engine/extractors/insights.ts +++ b/src/engine/extractors/insights.ts @@ -12,17 +12,18 @@ // See the License for the specific language governing permissions and // limitations under the License. +import type { TimeRange } from '../../utils/filter.js'; import { microToMilli, round } from '../../utils/units.js'; -import { getPrimaryInsightSet } from '../helpers.js'; +import { findInsightEntry } from '../helpers.js'; import type { LoadedTrace } from '../loader.js'; import type { - ClsCulpritInsight, InpBreakdownInsight, InsightsResult, LcpBreakdownInsight, RenderBlockingInsight, ThirdPartyEntityInsight, } from '../types.js'; +import { extractShifts, layoutShiftNodeLabel } from './shifts.js'; /** * Configuration options for diagnostic insights extraction. @@ -30,8 +31,9 @@ import type { export interface ExtractInsightsOptions { /** Filter insights by diagnostic category. Network requests are handled by extractNetwork. */ category?: 'lcp' | 'inp' | 'cls' | 'render-blocking' | 'third-parties'; - /** Target navigation ID or InsightSet ID. Defaults to the first InsightSet in the trace. */ + /** Target navigation ID or InsightSet ID. Defaults to the main-frame load with LCP. */ navigationId?: string; + timeRange?: TimeRange; } /** @@ -41,45 +43,61 @@ export function extractInsights( loadedTrace: LoadedTrace, options?: ExtractInsightsOptions, ): InsightsResult { - const { parsedTrace } = loadedTrace; + const { parsedTrace, engine } = loadedTrace; + const entry = findInsightEntry(parsedTrace, options?.navigationId, options?.timeRange); + const insightSet = entry?.model; const result: InsightsResult = { - navigationId: options?.navigationId, + navigationId: entry?.navigationId ?? options?.navigationId, + insightSetId: entry?.insightSetId, }; const warnings: string[] = []; - const insightSet = getPrimaryInsightSet(parsedTrace, options?.navigationId); - if (options?.navigationId && !insightSet) { - warnings.push(`No insights found for navigation ID: ${options.navigationId}`); + if (options?.navigationId && !entry) { + const rangeNote = options.timeRange ? ' in the selected time range' : ''; + return { + ...result, + warnings: [`No insights found for navigation ID: ${options.navigationId}${rangeNote}`], + }; + } + + if (options?.timeRange && !entry) { + warnings.push('No navigation insights overlap the selected time range.'); } const category = options?.category; const shouldInclude = (cat: string) => !category || category === cat; - const allowGlobalFallback = !options?.navigationId; + const scopeToNavigation = Boolean(entry?.navigation || options?.navigationId); + const allowGlobalFallback = !scopeToNavigation && !options?.timeRange; // 1. LCP Breakdown - if (shouldInclude('lcp') && insightSet?.LCPBreakdown) { - const l = insightSet.LCPBreakdown; - const subparts = l.subparts; - // Despite the 'Ts' suffix, the trace engine emits lcpTs in milliseconds. - const lcpInsight: LcpBreakdownInsight = { - totalLcpMs: round(l.lcpMs ?? l.lcpTs ?? 0), - ttfbMs: subparts?.ttfb?.range ? round(microToMilli(subparts.ttfb.range)) : 0, - loadDelayMs: subparts?.loadDelay?.range ? round(microToMilli(subparts.loadDelay.range)) : 0, - loadDurationMs: subparts?.loadDuration?.range - ? round(microToMilli(subparts.loadDuration.range)) - : 0, - renderDelayMs: subparts?.renderDelay?.range - ? round(microToMilli(subparts.renderDelay.range)) - : 0, - lcpElement: l.lcpEvent?.args?.data?.nodeName, - lcpUrl: l.lcpRequest?.args?.data?.url, - }; - result.lcpBreakdown = lcpInsight; + if (shouldInclude('lcp') && entry) { + const lcp = insightSet?.LCPBreakdown; + const subparts = lcp?.subparts; + if (lcp?.lcpMs !== undefined) { + const toMs = (range?: number) => (range === undefined ? null : round(microToMilli(range))); + const lcpInsight: LcpBreakdownInsight = { + totalLcpMs: round(lcp.lcpMs), + ttfbMs: toMs(subparts?.ttfb?.range), + loadDelayMs: toMs(subparts?.loadDelay?.range), + loadDurationMs: toMs(subparts?.loadDuration?.range), + renderDelayMs: toMs(subparts?.renderDelay?.range), + lcpElement: lcp.lcpEvent?.args?.data?.nodeName, + lcpUrl: lcp.lcpRequest?.args?.data?.url, + }; + result.lcpBreakdown = lcpInsight; - if (lcpInsight.renderDelayMs > 1000) { - warnings.push(`High LCP element render delay: ${lcpInsight.renderDelayMs}ms.`); - } - if (lcpInsight.loadDelayMs > 1000) { - warnings.push(`High LCP resource load delay: ${lcpInsight.loadDelayMs}ms.`); + if (!subparts) { + warnings.push( + 'LCP was measured, but the trace has no timing breakdown for this navigation.', + ); + } + if ((lcpInsight.renderDelayMs ?? 0) > 1000) { + warnings.push(`High LCP element render delay: ${lcpInsight.renderDelayMs}ms.`); + } + if ((lcpInsight.loadDelayMs ?? 0) > 1000) { + warnings.push(`High LCP resource load delay: ${lcpInsight.loadDelayMs}ms.`); + } + } else if (category === 'lcp') { + warnings.push('No LCP was measured for this navigation.'); } } @@ -123,40 +141,20 @@ export function extractInsights( // 3. CLS Culprits if (shouldInclude('cls')) { - const clsList: ClsCulpritInsight[] = []; - const clsModel = insightSet?.CLSCulprits; - - if (clsModel?.clusters && clsModel.clusters.length > 0) { - for (const cluster of clsModel.clusters) { - const score = cluster.clusterCumulativeScore ?? 0; - const tsMs = round(microToMilli(cluster.clusterWindow?.min ?? cluster.ts ?? 0)); - const culpritNodes: string[] = []; - if (cluster.events) { - for (const ev of cluster.events) { - const nodes = ev.args?.data?.impacted_nodes || []; - for (const n of nodes) { - if (n.node_id !== undefined) culpritNodes.push(`Node #${n.node_id}`); - } - } - } - clsList.push({ - score: round(score, 4), - timestampMs: tsMs, - culpritNodes, - }); - } - } else if (allowGlobalFallback && parsedTrace.data.LayoutShifts?.clusters) { - for (const cluster of parsedTrace.data.LayoutShifts.clusters) { - clsList.push({ - score: round(cluster.clusterCumulativeScore ?? 0, 4), - timestampMs: round(microToMilli(cluster.clusterWindow?.min ?? 0)), - culpritNodes: [], - }); - } - } - - if (clsList.length > 0) { - result.clsCulprits = clsList; + const layoutShifts = parsedTrace.data.LayoutShifts?.clusters ?? []; + const navigationId = entry?.navigationId ?? engine.Types.Events.NO_NAVIGATION; + const selected = scopeToNavigation + ? (insightSet?.CLSCulprits?.clusters ?? + layoutShifts.filter((cluster) => cluster.navigationId === navigationId)) + : layoutShifts; + const shifts = extractShifts(loadedTrace, { timeRange: options?.timeRange }, selected); + if (shifts.warnings) warnings.push(...shifts.warnings); + if (shifts.clusters.length > 0) { + result.clsCulprits = shifts.clusters.map((cluster) => ({ + score: cluster.cumulativeScore, + timestampMs: cluster.timestampMs, + culpritNodes: cluster.culprits.map(layoutShiftNodeLabel), + })); } } @@ -180,8 +178,9 @@ export function extractInsights( // 5. Third Parties if (shouldInclude('third-parties') && insightSet?.ThirdParties) { const thirdParties: ThirdPartyEntityInsight[] = []; - const summaries = insightSet.ThirdParties.entitySummaries || []; + const { entitySummaries: summaries, firstPartyEntity } = insightSet.ThirdParties; for (const s of summaries) { + if (s.entity === firstPartyEntity) continue; thirdParties.push({ entityName: s.entity?.name || 'Unknown', transferSizeBytes: s.transferSize ?? 0, diff --git a/src/engine/extractors/navigations.ts b/src/engine/extractors/navigations.ts index 2c2005e..8e7e011 100644 --- a/src/engine/extractors/navigations.ts +++ b/src/engine/extractors/navigations.ts @@ -12,6 +12,7 @@ // See the License for the specific language governing permissions and // limitations under the License. +import { isTimestampInBounds, type TimeRange } from '../../utils/filter.js'; import { microToMilli, round } from '../../utils/units.js'; import { findInsightEntry } from '../helpers.js'; import type { LoadedTrace } from '../loader.js'; @@ -55,7 +56,10 @@ function buildNavigationItem( * maps each navigation to its corresponding InsightSet ID (e.g. 'NAVIGATION_0'), * and determines whether the navigation belongs to the primary main frame. */ -export function extractNavigations(loadedTrace: LoadedTrace): NavigationsResult { +export function extractNavigations( + loadedTrace: LoadedTrace, + timeRange?: TimeRange, +): NavigationsResult { const { parsedTrace } = loadedTrace; const meta = parsedTrace.data.Meta; const navigationsMap = meta?.navigationsByNavigationId; @@ -64,7 +68,6 @@ export function extractNavigations(loadedTrace: LoadedTrace): NavigationsResult const mainFrameNavs = meta?.mainFrameNavigations; const insightsMap = parsedTrace.insights; - const insightSetIds: string[] = insightsMap ? Array.from(insightsMap.keys()) : []; const navigations: NavigationItem[] = []; const seenIds = new Set(); @@ -99,11 +102,23 @@ export function extractNavigations(loadedTrace: LoadedTrace): NavigationsResult } } - navigations.sort((a, b) => a.timestampMs - b.timestampMs); + const matched = navigations.filter((navigation) => + isTimestampInBounds(navigation.timestampMs, timeRange), + ); + matched.sort((a, b) => a.timestampMs - b.timestampMs); + const insightSetIds = timeRange + ? [ + ...new Set( + matched.flatMap((navigation) => + navigation.insightSetId ? [navigation.insightSetId] : [], + ), + ), + ] + : [...(insightsMap?.keys() ?? [])]; return { - total: navigations.length, - navigations, + total: matched.length, + navigations: matched, insightSetIds, }; } diff --git a/src/engine/extractors/network.ts b/src/engine/extractors/network.ts index 2498bd9..de6c0c6 100644 --- a/src/engine/extractors/network.ts +++ b/src/engine/extractors/network.ts @@ -23,11 +23,19 @@ export type SyntheticNetworkRequest = NonNullable< export interface ExtractNetworkOptions { renderBlocking?: boolean; + stripQuery?: boolean; slowest?: number; initiatorChain?: string; timeRange?: TimeRange; } +function stripUrlQuery(url: string): string { + const query = url.indexOf('?'); + const fragment = url.indexOf('#'); + if (query === -1 || (fragment !== -1 && fragment < query)) return url; + return url.slice(0, query) + (fragment === -1 ? '' : url.slice(fragment)); +} + export function extractNetwork( loadedTrace: LoadedTrace, options?: ExtractNetworkOptions, @@ -37,6 +45,7 @@ export function extractNetwork( const slowest = options?.slowest ?? 20; const initiatorTarget = options?.initiatorChain; const timeRange = options?.timeRange; + const displayUrl = (url: string) => (options?.stripQuery ? stripUrlQuery(url) : url); const rawRequests = parsedTrace.data.NetworkRequests?.byTime || []; const result: NetworkResult = { @@ -63,8 +72,8 @@ export function extractNetwork( const initiatorUrl = initiator?.args.data.url || initiator?.name; chain.push({ - url, - initiatorUrl, + url: displayUrl(url), + initiatorUrl: initiatorUrl === undefined ? undefined : displayUrl(initiatorUrl), type: current.args.data.mimeType || current.name, }); @@ -120,7 +129,7 @@ export function extractNetwork( processed.push({ requestId: req.args.data.requestId, - url, + url: displayUrl(url), mimeType: req.args.data.mimeType || '', priority: req.args.data.priority || 'Normal', timing: { diff --git a/src/engine/extractors/overview.ts b/src/engine/extractors/overview.ts index 6928192..55b322c 100644 --- a/src/engine/extractors/overview.ts +++ b/src/engine/extractors/overview.ts @@ -15,8 +15,8 @@ import type { TimeRange } from '../../utils/filter.js'; import { microToMilli, round } from '../../utils/units.js'; import { - getMainRendererThread, - getPrimaryInsightSet, + findInsightEntry, + getRendererThread, MISSING_CPU_SAMPLES, PAGE_TRACE_CAPTURE_HINT, shouldWarnAboutLayoutShiftCapture, @@ -50,6 +50,7 @@ function rateInp(inpMs: number): CwvRating { } export interface ExtractOverviewOptions { + threadId?: string; timeRange?: TimeRange; } @@ -82,6 +83,8 @@ export function extractOverview( const navigationStartMs = firstNav ? round(microToMilli(firstNav.ts)) : null; // 3. Core Web Vitals + const entry = findInsightEntry(parsedTrace); + const insightSet = entry?.model; const pageMetrics = data.PageLoadMetrics?.metricScoresByFrameId?.get(mainFrameId); let fcp: OverviewResult['coreWebVitals']['fcp'] = null; @@ -89,30 +92,31 @@ export function extractOverview( if (pageMetrics) { const MetricName = engine.Handlers.ModelHandlers.PageLoadMetrics.MetricName; - for (const navMetrics of pageMetrics.values()) { - const fcpMetric = navMetrics.get(MetricName.FCP); - if (fcpMetric && fcpMetric.timing !== undefined && !fcp) { - const valMs = round(microToMilli(fcpMetric.timing)); - fcp = { - valueMs: valMs, - rating: rateFcp(valMs), - }; + let metrics = entry?.navigation ? pageMetrics.get(entry.navigation) : undefined; + if (!entry?.navigation) { + for (const navMetrics of pageMetrics.values()) { + if (navMetrics.has(MetricName.LCP)) { + metrics = navMetrics; + break; + } + metrics ??= navMetrics; } + } - const lcpMetric = navMetrics.get(MetricName.LCP); - if (lcpMetric && lcpMetric.timing !== undefined && !lcp) { - const valMs = round(microToMilli(lcpMetric.timing)); - lcp = { - valueMs: valMs, - rating: rateLcp(valMs), - }; - } + const fcpTiming = metrics?.get(MetricName.FCP)?.timing; + if (fcpTiming !== undefined) { + const valueMs = round(microToMilli(fcpTiming)); + fcp = { valueMs, rating: rateFcp(valueMs) }; + } + + const lcpTiming = metrics?.get(MetricName.LCP)?.timing; + if (lcpTiming !== undefined) { + const valueMs = round(microToMilli(lcpTiming)); + lcp = { valueMs, rating: rateLcp(valueMs) }; } } // Check insights for LCP URL / Element if available - const insightSet = getPrimaryInsightSet(parsedTrace); - if (insightSet?.LCPBreakdown) { const lcpInsight = insightSet.LCPBreakdown; if (lcpInsight.lcpMs !== undefined && !lcp) { @@ -178,12 +182,20 @@ export function extractOverview( totalMs: 0, }; - const mainThread = getMainRendererThread(data); + const mainThread = getRendererThread(data, options?.threadId); if (mainThread?.entries && mainThread.entries.length > 0) { const bottomUp = new engine.Extras.TraceTree.BottomUpRootNode(mainThread.entries, { textFilter: { accept: () => true }, - filters: [], + // Renderer.entryToNode is built with CPU samples, which can displace overlapping timeline + // events. Keep timeline attribution independent of whether samples were captured. + filters: [ + { + accept: (event) => + !engine.Types.Events.isProfileCall(event) && + !engine.Types.Events.isPhaseAsync(event.ph), + }, + ], startTime: engine.Types.Timing.Milli(minMs), endTime: engine.Types.Timing.Milli(maxMs), }); @@ -197,13 +209,14 @@ export function extractOverview( name === 'MajorGC' || name === 'MinorGC' || name === 'GCEvent' || - name.startsWith('v8.gc') + name.toLowerCase().startsWith('v8.gc') ) { breakdown.gcMs += node.selfTime; } else if (name === 'Layout' || name === 'UpdateLayoutTree' || name === 'InvalidateLayout') { breakdown.layoutMs += node.selfTime; } else if ( name === 'UpdateLayerTree' || + name === 'Layerize' || name === 'Paint' || name === 'CompositeLayers' || name === 'RasterTask' || @@ -211,10 +224,11 @@ export function extractOverview( ) { breakdown.renderingMs += node.selfTime; } else if ( - name.startsWith('f:') || name === 'EvaluateScript' || name === 'FunctionCall' || - name === 'v8.compile' || + name.startsWith('v8.compile') || + name === 'v8.callFunction' || + name === 'RunMicrotasks' || name === 'v8.run' || name === 'TimerFire' || name === 'FireAnimationFrame' || @@ -266,6 +280,7 @@ export function extractOverview( } return { + threadId: options?.threadId, bounds: { minMs, maxMs, durationMs }, mainUrl, navigationStartMs, diff --git a/src/engine/extractors/profile.ts b/src/engine/extractors/profile.ts index 03180a8..0f11b8c 100644 --- a/src/engine/extractors/profile.ts +++ b/src/engine/extractors/profile.ts @@ -14,11 +14,12 @@ import type { TimeRange } from '../../utils/filter.js'; import { microToMilli, round } from '../../utils/units.js'; -import { getMainRendererThread } from '../helpers.js'; +import { getRendererThread, MISSING_CPU_SAMPLES, rendererCpuFilter } from '../helpers.js'; import type { LoadedTrace } from '../loader.js'; import type { ProfileResult } from '../types.js'; export interface ExtractProfileOptions { + threadId?: string; timeRange?: TimeRange; bucketMs?: number; top?: number; @@ -43,16 +44,20 @@ export function extractProfile( const topCount = options?.top ?? 3; const result: ProfileResult = { + threadId: options?.threadId, timeRange: { startMs, endMs }, bucketSizeMs, buckets: [], }; - const mainThread = getMainRendererThread(parsedTrace.data); + const mainThread = getRendererThread(parsedTrace.data, options?.threadId); if (!mainThread?.entries?.length) { return result; } + if (!mainThread.profileCalls.length) result.warnings = [MISSING_CPU_SAMPLES]; + + const cpuFilter = rendererCpuFilter(loadedTrace); let currentStart = startMs; while (currentStart < endMs) { const currentEnd = Math.min(endMs, round(currentStart + bucketSizeMs)); @@ -60,12 +65,12 @@ export function extractProfile( const bottomUp = new engine.Extras.TraceTree.BottomUpRootNode(mainThread.entries, { textFilter: { accept: () => true }, - filters: [], + filters: [cpuFilter], startTime: engine.Types.Timing.Milli(currentStart), endTime: engine.Types.Timing.Milli(currentEnd), }); - const funcList: Array<{ name: string; selfTimeMs: number }> = []; + const funcList: Array<{ name: string; url?: string; selfTimeMs: number }> = []; let totalActiveMs = 0; const children = bottomUp.children(); @@ -80,6 +85,7 @@ export function extractProfile( totalActiveMs += node.selfTime; funcList.push({ name: fnName, + url: frame?.url || undefined, selfTimeMs: round(node.selfTime), }); } @@ -88,6 +94,7 @@ export function extractProfile( const topFunctions = funcList.slice(0, topCount).map((f) => ({ name: f.name, + url: f.url, selfTimeMs: f.selfTimeMs, selfTimePct: totalActiveMs > 0 ? round((f.selfTimeMs / totalActiveMs) * 100, 1) : 0, })); diff --git a/src/engine/extractors/report.ts b/src/engine/extractors/report.ts new file mode 100644 index 0000000..8adaea7 --- /dev/null +++ b/src/engine/extractors/report.ts @@ -0,0 +1,68 @@ +// Copyright 2026 Google LLC +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +import type { TimeRange } from '../../utils/filter.js'; +import type { LoadedTrace } from '../loader.js'; +import { extractInsights } from './insights.js'; +import { extractInteractions } from './interactions.js'; +import { extractNavigations } from './navigations.js'; +import { extractNetwork } from './network.js'; +import { extractOverview } from './overview.js'; +import { extractProfile } from './profile.js'; +import { extractScheduling } from './scheduling.js'; +import { extractShifts } from './shifts.js'; +import { extractTasks } from './tasks.js'; + +export const REPORT_SECTIONS = [ + 'overview', + 'insights', + 'tasks', + 'profile', + 'scheduling', + 'network', + 'interactions', + 'shifts', + 'navigations', +] as const; + +export type ReportSection = (typeof REPORT_SECTIONS)[number]; + +export const DEFAULT_REPORT_SECTIONS: ReportSection[] = ['overview', 'tasks', 'shifts', 'network']; + +interface ExtractReportOptions { + sections: readonly ReportSection[]; + timeRange?: TimeRange; + threadId?: string; + stripQuery?: boolean; +} + +export function extractReport( + loaded: LoadedTrace, + options: ExtractReportOptions, +): Array<[ReportSection, unknown]> { + const { timeRange, threadId, stripQuery } = options; + const extract: Record unknown> = { + overview: () => extractOverview(loaded, { timeRange, threadId }), + insights: () => extractInsights(loaded, { timeRange }), + tasks: () => extractTasks(loaded, { timeRange, threadId }), + profile: () => extractProfile(loaded, { timeRange, threadId }), + scheduling: () => extractScheduling(loaded, { timeRange }), + network: () => extractNetwork(loaded, { timeRange, stripQuery }), + interactions: () => extractInteractions(loaded, { timeRange }), + shifts: () => extractShifts(loaded, { timeRange }), + navigations: () => extractNavigations(loaded, timeRange), + }; + + return options.sections.map((section) => [section, extract[section]()]); +} diff --git a/src/engine/extractors/shifts.ts b/src/engine/extractors/shifts.ts index 8a851d3..9c96677 100644 --- a/src/engine/extractors/shifts.ts +++ b/src/engine/extractors/shifts.ts @@ -14,11 +14,7 @@ import { isEventInBounds, type TimeRange } from '../../utils/filter.js'; import { microToMilli, round } from '../../utils/units.js'; -import { - getPrimaryInsightSet, - shouldWarnAboutLayoutShiftCapture, - UNCONFIRMED_LAYOUT_SHIFT_CAPTURE, -} from '../helpers.js'; +import { shouldWarnAboutLayoutShiftCapture, UNCONFIRMED_LAYOUT_SHIFT_CAPTURE } from '../helpers.js'; import type { LoadedTrace } from '../loader.js'; import type { ShiftClusterItem, @@ -40,11 +36,27 @@ export type TraceImpactedNode = NonNullable< NonNullable['impacted_nodes'] >[number]; -/** Extends TraceImpactedNode with optional tag_name and selector fields used in synthetic trace payloads. */ -export type TraceImpactedNodeWithSelector = TraceImpactedNode & { - tag_name?: string; - selector?: string; -}; +export function layoutShiftNodeLabel(node: ShiftCulprit): string { + return ( + node.selector || + node.debugName || + node.tagName || + (node.nodeId !== undefined ? `Node #${node.nodeId}` : 'Unknown node') + ); +} + +function getNodeDetails(node: TraceImpactedNode, shift: SyntheticLayoutShift) { + return { + nodeId: node.node_id ?? undefined, + processId: shift.pid, + frameId: shift.args.frame || undefined, + tagName: 'tag_name' in node && typeof node.tag_name === 'string' ? node.tag_name : undefined, + selector: 'selector' in node && typeof node.selector === 'string' ? node.selector : undefined, + debugName: node.debug_name, + oldRect: node.old_rect, + newRect: node.new_rect, + }; +} /** * Extracts layout shift clusters, scores, and culprit elements. @@ -52,14 +64,14 @@ export type TraceImpactedNodeWithSelector = TraceImpactedNode & { export function extractShifts( loadedTrace: LoadedTrace, options?: ExtractShiftsOptions, + rawClusters?: readonly SyntheticLayoutShiftCluster[], ): ShiftsResult { - const { parsedTrace } = loadedTrace; const timeRange = options?.timeRange; - const rawClusters = parsedTrace.data.LayoutShifts?.clusters || []; + const sourceClusters = rawClusters ?? loadedTrace.parsedTrace.data.LayoutShifts?.clusters ?? []; const clusters: ShiftClusterItem[] = []; - for (const cluster of rawClusters) { + for (const cluster of sourceClusters) { const startMs = microToMilli(cluster.clusterWindow?.min ?? cluster.ts ?? 0); const durMs = microToMilli(cluster.clusterWindow?.range ?? cluster.dur ?? 0); const score = cluster.clusterCumulativeScore ?? 0; @@ -68,101 +80,74 @@ export function extractShifts( continue; } - const culpritMap = new Map(); + const culpritMap = new Map(); const individualEvents: ShiftIndividualEvent[] = []; - if (cluster.events) { - for (const ev of cluster.events) { - const scoreDelta = ev.args.data?.weighted_score_delta; - const nodes = (ev.args.data?.impacted_nodes || []) as TraceImpactedNodeWithSelector[]; - for (const n of nodes) { - const key = - n.selector || - (n.node_id ? `node:${n.node_id}` : n.debug_name || n.tag_name || 'unknown'); - const existing = culpritMap.get(key); - if (existing) { - if (scoreDelta !== undefined) { - existing.scoreDelta = round((existing.scoreDelta ?? 0) + scoreDelta, 4); - } - } else { - culpritMap.set(key, { - nodeId: n.node_id, - tagName: n.tag_name, - selector: n.selector || (n.node_id ? `Node #${n.node_id}` : undefined), - debugName: n.debug_name, - scoreDelta: scoreDelta !== undefined ? round(scoreDelta, 4) : undefined, - }); + for (const ev of cluster.events) { + const data = ev.args.data; + const scoreDelta = data?.weighted_score_delta; + const impactedNodes = (data?.impacted_nodes ?? []).map((node) => getNodeDetails(node, ev)); + for (const node of impactedNodes) { + const { processId, frameId, nodeId, selector, debugName, tagName } = node; + // Backend node IDs can repeat across renderers and frames. + const identity = nodeId ?? (selector || debugName || tagName || undefined); + const key = + identity === undefined ? Symbol() : JSON.stringify([processId, frameId, identity]); + const existing = culpritMap.get(key); + if (existing) { + if (scoreDelta !== undefined) { + existing.scoreDelta = (existing.scoreDelta ?? 0) + scoreDelta; } + existing.newRect = node.newRect; + } else { + culpritMap.set(key, { ...node, scoreDelta }); } + } - if (options?.includeEvents) { - const evTsMs = microToMilli(ev.ts); - const evScore = ev.args.data?.score ?? ev.args.data?.weighted_score_delta ?? 0; - const evWeighted = ev.args.data?.weighted_score_delta; - const evNavId = ev.args.data?.navigationId; - const hadRecentInput = ev.args.data?.had_recent_input; - const impactedNodes = nodes.map((node) => ({ - nodeId: node.node_id, - tagName: node.tag_name, - selector: node.selector, - debugName: node.debug_name, - oldRect: node.old_rect, - newRect: node.new_rect, - })); - - individualEvents.push({ - timestampMs: round(evTsMs), - score: round(evScore, 4), - weightedScoreDelta: evWeighted !== undefined ? round(evWeighted, 4) : undefined, - navigationId: evNavId, - hadRecentInput, - impactedNodes: impactedNodes.length > 0 ? impactedNodes : undefined, - }); - } + if (options?.includeEvents) { + individualEvents.push({ + timestampMs: round(microToMilli(ev.ts)), + score: round(data?.score ?? scoreDelta ?? 0, 4), + weightedScoreDelta: scoreDelta !== undefined ? round(scoreDelta, 4) : undefined, + navigationId: data?.navigationId, + hadRecentInput: data?.had_recent_input, + impactedNodes: impactedNodes.length > 0 ? impactedNodes : undefined, + }); } } - const clusterNavId = - cluster.navigationId || - (cluster.events ? cluster.events[0]?.args.data?.navigationId : undefined); + const clusterNavId = cluster.navigationId || cluster.events[0]?.args.data?.navigationId; clusters.push({ timestampMs: round(startMs), durationMs: round(durMs), cumulativeScore: round(score, 4), navigationId: clusterNavId, - culprits: Array.from(culpritMap.values()), + culprits: [...culpritMap.values()].map((node) => ({ + ...node, + scoreDelta: node.scoreDelta !== undefined ? round(node.scoreDelta, 4) : undefined, + })), ...(options?.includeEvents ? { events: individualEvents } : {}), }); } - // If no clusters in handler, check insights - if (clusters.length === 0) { - const insightSet = getPrimaryInsightSet(parsedTrace); - const clsModel = insightSet?.CLSCulprits; - if (clsModel?.clusters && clsModel.clusters.length > 0) { - for (const c of clsModel.clusters) { - const startMs = microToMilli(c.clusterWindow?.min ?? 0); - const durMs = microToMilli(c.clusterWindow?.range ?? 0); - const score = c.clusterCumulativeScore ?? 0; - if (isEventInBounds(startMs, durMs, timeRange)) { - clusters.push({ - timestampMs: round(startMs), - durationMs: round(durMs), - cumulativeScore: round(score, 4), - culprits: [], - }); - } - } - } - } - // Sort by score descending clusters.sort((a, b) => b.cumulativeScore - a.cumulativeScore); - const result: ShiftsResult = { clusters }; + const warnings: string[] = []; if (shouldWarnAboutLayoutShiftCapture(loadedTrace)) { - result.warnings = [UNCONFIRMED_LAYOUT_SHIFT_CAPTURE]; + warnings.push(UNCONFIRMED_LAYOUT_SHIFT_CAPTURE); } + const missingNodeMetadata = clusters.some((cluster) => + cluster.culprits.some((node) => !node.tagName && !node.selector && !node.debugName), + ); + if (missingNodeMetadata) { + warnings.push( + 'DOM names and selectors were unavailable for some shifted nodes. Backend node IDs and rectangles are shown when recorded.', + ); + } + + const result: ShiftsResult = { clusters }; + if (warnings.length > 0) result.warnings = warnings; return result; } diff --git a/src/engine/extractors/tasks.ts b/src/engine/extractors/tasks.ts index f6b05cb..cc0f603 100644 --- a/src/engine/extractors/tasks.ts +++ b/src/engine/extractors/tasks.ts @@ -14,164 +14,175 @@ import { isEventInBounds, type TimeRange } from '../../utils/filter.js'; import { microToMilli, round } from '../../utils/units.js'; -import { getMainRendererThread } from '../helpers.js'; +import { getRendererThread, MISSING_CPU_SAMPLES, rendererCpuFilter } from '../helpers.js'; import type { LoadedTrace } from '../loader.js'; import type { TaskItem, TasksResult } from '../types.js'; export interface ExtractTasksOptions { + threadId?: string; minDurationMs?: number; tree?: 'bottom-up' | 'top-down'; top?: number; + urlOnly?: boolean; timeRange?: TimeRange; } +function rankAndLimit(candidates: T[], limit: number): void { + candidates.sort((a, b) => b.score - a.score); + if (candidates.length > limit) candidates.length = limit; +} + /** * Extracts long main thread tasks and aggregated top-down / bottom-up call trees. */ export function extractTasks(loadedTrace: LoadedTrace, options?: ExtractTasksOptions): TasksResult { const { engine, parsedTrace } = loadedTrace; const minDurationMs = options?.minDurationMs ?? 50; - const top = options?.top ?? 20; + const top = Math.trunc(options?.top ?? 20); + const urlOnly = options?.urlOnly; const treeMode = options?.tree; const timeRange = options?.timeRange; - const mainThread = getMainRendererThread(parsedTrace.data); + const mainThread = getRendererThread(parsedTrace.data, options?.threadId); + + const result: TasksResult = { mode: treeMode ?? 'list', threadId: options?.threadId, tasks: [] }; + if (!mainThread) return result; + if (!mainThread.profileCalls.length) result.warnings = [MISSING_CPU_SAMPLES]; + if (!(top > 0)) return result; - if (!mainThread) { + const traceBounds = parsedTrace.data.Meta.traceBounds; + const treeOptions = { + startTime: engine.Types.Timing.Milli(timeRange?.startMs ?? microToMilli(traceBounds.min)), + endTime: engine.Types.Timing.Milli(timeRange?.endMs ?? microToMilli(traceBounds.max)), + }; + const traceTree = engine.Extras.TraceTree; + const cpuFilter = rendererCpuFilter(loadedTrace); + type TreeNode = InstanceType; + + function treeItem(node: TreeNode): TaskItem { + const frame = traceTree.eventStackFrame(node.event); + // Bottom-up nodes are created in event end order. A recursive call can end before its caller. + const firstTs = node.events.reduce( + (min, event) => Math.min(min, event.ts), + node.event.ts, + ); return { - mode: treeMode || 'list', - tasks: [], + startTimeMs: round(microToMilli(firstTs)), + durationMs: round(node.totalTime), + selfTimeMs: round(node.selfTime), + functionName: frame?.functionName || node.event.name, + url: frame?.url || undefined, + lineNumber: frame?.lineNumber, + columnNumber: frame?.columnNumber, }; } - const traceBounds = parsedTrace.data.Meta.traceBounds; - const startMs = timeRange ? timeRange.startMs : microToMilli(traceBounds.min); - const endMs = timeRange ? timeRange.endMs : microToMilli(traceBounds.max); - if (treeMode === 'bottom-up') { - const bottomUp = new engine.Extras.TraceTree.BottomUpRootNode(mainThread.entries, { + const bottomUp = new traceTree.BottomUpRootNode(mainThread.entries, { + ...treeOptions, textFilter: { accept: () => true }, - filters: [], - startTime: engine.Types.Timing.Milli(startMs), - endTime: engine.Types.Timing.Milli(endMs), + filters: [cpuFilter], }); - const tasks: TaskItem[] = []; - const children = bottomUp.children(); - for (const node of children.values()) { - if (node.selfTime <= 0) continue; - const frame = node.event ? engine.Extras.TraceTree.eventStackFrame(node.event) : null; - tasks.push({ - startTimeMs: round(startMs), - durationMs: round(node.totalTime), - selfTimeMs: round(node.selfTime), - functionName: frame?.functionName || node.event?.name || String(node.id) || 'Unknown', - url: frame?.url || undefined, - lineNumber: frame?.lineNumber, - columnNumber: frame?.columnNumber, - }); + const candidates: Array<{ node: TreeNode; score: number }> = []; + for (const node of bottomUp.children().values()) { + if (node.selfTime > 0 && (!urlOnly || traceTree.eventStackFrame(node.event)?.url)) { + candidates.push({ node, score: round(node.selfTime) }); + // Trim in batches to bound memory and sort work for a small --top. + if (candidates.length === top * 2) rankAndLimit(candidates, top); + } } + rankAndLimit(candidates, top); - tasks.sort((a, b) => b.selfTimeMs - a.selfTimeMs); - return { - mode: 'bottom-up', - tasks: tasks.slice(0, top), - }; + for (const { node } of candidates) result.tasks.push(treeItem(node)); + return result; } if (treeMode === 'top-down') { - const topDown = new engine.Extras.TraceTree.TopDownRootNode(mainThread.entries, { - filters: [], - startTime: engine.Types.Timing.Milli(startMs), - endTime: engine.Types.Timing.Milli(endMs), + const topDown = new traceTree.TopDownRootNode(mainThread.entries, { + ...treeOptions, + filters: options?.urlOnly + ? [cpuFilter, { accept: (event) => Boolean(traceTree.eventStackFrame(event)?.url) }] + : [cpuFilter], }); - const tasks: TaskItem[] = []; - const children = topDown.children(); - for (const node of children.values()) { - const frame = node.event ? engine.Extras.TraceTree.eventStackFrame(node.event) : null; - tasks.push({ - startTimeMs: round(startMs), - durationMs: round(node.totalTime), - selfTimeMs: round(node.selfTime), - functionName: frame?.functionName || node.event?.name || String(node.id) || 'Unknown', - url: frame?.url || undefined, - lineNumber: frame?.lineNumber, - columnNumber: frame?.columnNumber, - }); + // TraceTree computes descendants on demand. Expand only the most expensive paths. + const pending: Array<{ node: TreeNode; parents: string[] }> = [ + ...topDown.children().values(), + ].map((node) => ({ node, parents: [] })); + while (pending.length && result.tasks.length < top) { + pending.sort((a, b) => b.node.totalTime - a.node.totalTime); + const next = pending.shift(); + if (!next) break; + const item = treeItem(next.node); + const callPath = [...next.parents, item.functionName]; + result.tasks.push({ ...item, callPath }); + if (result.tasks.length === top || !next.node.hasChildren()) continue; + for (const node of next.node.children().values()) pending.push({ node, parents: callPath }); } - - tasks.sort((a, b) => b.durationMs - a.durationMs); - return { - mode: 'top-down', - tasks: tasks.slice(0, top), - }; + return result; } - // Default List Mode - const tasks: TaskItem[] = []; - const entryToNode = parsedTrace.data.Renderer?.entryToNode; + const candidates: Array<{ entry: (typeof mainThread.entries)[number]; score: number }> = []; + // URL eligibility is only known after inspecting a task's descendants. + const candidateLimit = urlOnly ? Infinity : top; for (const entry of mainThread.entries) { - if (!entry.dur) continue; - const durMs = microToMilli(entry.dur); - const startEntryMs = microToMilli(entry.ts); - - if (durMs < minDurationMs) continue; - if (!isEventInBounds(startEntryMs, durMs, timeRange)) continue; - - const isTop = entry.name === 'RunTask' || engine.Helpers.Trace.isTopLevelEvent(entry); - - if (isTop) { - let functionName = entry.name; - let url: string | undefined; - let lineNumber: number | undefined; - let columnNumber: number | undefined; - let selfTimeMs = durMs; - - const frame = engine.Extras.TraceTree.eventStackFrame(entry); - if (frame?.functionName) { - functionName = frame.functionName; - url = frame.url; - lineNumber = frame.lineNumber; - columnNumber = frame.columnNumber; - } + if (!entry.dur || (entry.name !== 'RunTask' && !engine.Helpers.Trace.isTopLevelEvent(entry))) { + continue; + } + const durationMs = microToMilli(entry.dur); + const startTimeMs = microToMilli(entry.ts); + if (durationMs < minDurationMs || !isEventInBounds(startTimeMs, durationMs, timeRange)) { + continue; + } + + candidates.push({ entry, score: round(durationMs) }); + if (candidates.length === candidateLimit * 2) rankAndLimit(candidates, candidateLimit); + } + rankAndLimit(candidates, candidateLimit); - if (entryToNode) { - const node = entryToNode.get(entry); - if (node?.children && node.children.length > 0) { - let childDurationSum = 0; - for (const c of node.children) { - childDurationSum += microToMilli(c.entry.dur ?? 0); - } - selfTimeMs = Math.max(0, durMs - childDurationSum); - - if (!frame?.functionName) { - const mainChild = node.children[0].entry; - const childFrame = engine.Extras.TraceTree.eventStackFrame(mainChild); - functionName = `${entry.name} > ${childFrame?.functionName || mainChild.name}`; - url = childFrame?.url; - lineNumber = childFrame?.lineNumber; - columnNumber = childFrame?.columnNumber; - } + const entryToNode = parsedTrace.data.Renderer?.entryToNode; + for (const { entry, score: durationMs } of candidates) { + if (result.tasks.length === top) break; + + const node = entryToNode?.get(entry); + let source = entry; + let frame = traceTree.eventStackFrame(entry); + if (node && !frame?.url) { + let bestTime = -1; + const pending = [...node.children]; + while (pending.length > 0) { + const child = pending.pop(); + if (!child) break; + const childFrame = traceTree.eventStackFrame(child.entry); + const time = child.selfTime ?? child.entry.dur ?? 0; + if (childFrame?.url && childFrame.functionName && time > bestTime) { + source = child.entry; + frame = childFrame; + bestTime = time; } + for (const descendant of child.children) pending.push(descendant); } - tasks.push({ - startTimeMs: round(startEntryMs), - durationMs: round(durMs), - selfTimeMs: round(selfTimeMs), - functionName, - url, - lineNumber, - columnNumber, - }); + if (source === entry && !frame?.functionName && node.children[0]) { + source = node.children[0].entry; + frame = traceTree.eventStackFrame(source); + } } + + if (urlOnly && !frame?.url) continue; + const name = frame?.functionName || source.name; + result.tasks.push({ + startTimeMs: round(microToMilli(entry.ts)), + durationMs, + selfTimeMs: round(microToMilli(node?.selfTime ?? entry.dur ?? 0)), + functionName: source === entry ? name : `${entry.name} > ${name}`, + url: frame?.url || undefined, + lineNumber: frame?.lineNumber, + columnNumber: frame?.columnNumber, + }); } - tasks.sort((a, b) => b.durationMs - a.durationMs); - return { - mode: 'list', - tasks: tasks.slice(0, top), - }; + return result; } diff --git a/src/engine/helpers.ts b/src/engine/helpers.ts index 2c0b810..b6efe41 100644 --- a/src/engine/helpers.ts +++ b/src/engine/helpers.ts @@ -12,7 +12,10 @@ // See the License for the specific language governing permissions and // limitations under the License. +import { isEventInBounds, type TimeRange } from '../utils/filter.js'; +import { microToMilli } from '../utils/units.js'; import type { LoadedTrace, ParsedTrace } from './loader.js'; +import type { ThreadItem } from './types.js'; type MapValue = T extends Map ? V : never; type RendererProcess = MapValue['processes']>; @@ -53,52 +56,98 @@ export function getMainRendererThread(data: ParsedTrace['data']): RendererThread return null; } +export function getRendererThread( + data: ParsedTrace['data'], + threadId?: string, +): RendererThread | null { + if (threadId === undefined) return getMainRendererThread(data); + + for (const [pid, process] of data.Renderer?.processes ?? []) { + for (const [tid, thread] of process.threads) { + if (threadId === `${pid}:${tid}` && thread.entries.length > 0) return thread; + } + } + throw new Error( + `Unknown renderer thread "${threadId}". Use trace-engine-cli threads to list PID:TID values.`, + ); +} + +export function listRendererThreads(data: ParsedTrace['data']): ThreadItem[] { + const threads: ThreadItem[] = []; + const defaultThread = getMainRendererThread(data); + for (const [pid, process] of data.Renderer?.processes ?? []) { + const frames = [...(data.Meta.frameByProcessId.get(pid)?.values() ?? [])]; + const frameUrls = [...new Set(frames.map((frame) => frame.url).filter(Boolean))]; + for (const [tid, thread] of process.threads) { + if (!thread.entries.length) continue; + threads.push({ + id: `${pid}:${tid}`, + name: thread.name, + frameUrls, + isDefault: thread === defaultThread, + hasCpuSamples: thread.profileCalls.length > 0, + }); + } + } + return threads; +} + +export function rendererCpuFilter({ engine, parsedTrace }: LoadedTrace) { + const entryToNode = parsedTrace.data.Renderer?.entryToNode; + return { + // The renderer tree discards overlapping entries that cannot represent nested CPU work. + accept: (event: RendererThread['entries'][number]) => + entryToNode?.has(event) === true && !engine.Types.Events.isPhaseAsync(event.ph), + }; +} + +export function getNavigationId(navigation: InsightSet['navigation']): string | undefined { + const args = navigation?.args; + if (args && 'data' in args) return args.data?.navigationId; + return undefined; +} + export interface InsightEntryResult { insightSetId: string; + navigation: InsightSet['navigation']; + navigationId?: string; model: InsightModels; } /** - * Finds an insight set entry matching a navigation ID or key. + * Finds a navigation by trace ID or InsightSet ID. When no navigation is given, + * prefer the main-frame load with an LCP measurement over pre-navigation activity. */ export function findInsightEntry( parsedTrace: ParsedTrace, navigationId?: string, -): InsightEntryResult | null { - if (!parsedTrace.insights || parsedTrace.insights.size === 0) { - return null; - } - - if (navigationId) { - const directMatch = parsedTrace.insights.get(navigationId); - if (directMatch) { - return { insightSetId: navigationId, model: directMatch.model }; + timeRange?: TimeRange, +): InsightEntryResult | undefined { + let first: InsightEntryResult | undefined; + let firstNavigation: InsightEntryResult | undefined; + let firstWithLcp: InsightEntryResult | undefined; + for (const [key, set] of parsedTrace.insights ?? []) { + if (!isEventInBounds(microToMilli(set.bounds.min), microToMilli(set.bounds.range), timeRange)) { + continue; } - for (const [key, insightSet] of parsedTrace.insights.entries()) { - const navArgs = insightSet.navigation?.args; - const navEventId = - navArgs && 'data' in navArgs && navArgs.data ? navArgs.data.navigationId : undefined; - if (key === navigationId || insightSet.id === navigationId || navEventId === navigationId) { - return { insightSetId: key || insightSet.id || navigationId, model: insightSet.model }; - } + const traceId = getNavigationId(set.navigation); + const entry = { + insightSetId: key, + navigation: set.navigation, + navigationId: traceId, + model: set.model, + }; + if (navigationId) { + if (key === navigationId || set.id === navigationId || traceId === navigationId) return entry; + continue; + } + first ??= entry; + if (set.navigation && set.frameId === parsedTrace.data.Meta.mainFrameId) { + firstNavigation ??= entry; + if (set.model.LCPBreakdown?.lcpMs !== undefined) firstWithLcp ??= entry; } - return null; - } - - for (const [key, insightSet] of parsedTrace.insights.entries()) { - return { insightSetId: key || insightSet.id || '', model: insightSet.model }; } - return null; -} - -/** - * Retrieves the insight model set from the parsed trace, optionally filtered by navigation ID. - */ -export function getPrimaryInsightSet( - parsedTrace: ParsedTrace, - navigationId?: string, -): InsightModels | null { - return findInsightEntry(parsedTrace, navigationId)?.model ?? null; + return firstWithLcp ?? firstNavigation ?? first; } export const PAGE_TRACE_CAPTURE_HINT = diff --git a/src/engine/types.ts b/src/engine/types.ts index 7525266..5d36706 100644 --- a/src/engine/types.ts +++ b/src/engine/types.ts @@ -22,6 +22,7 @@ export type CwvRating = 'good' | 'needs-improvement' | 'poor'; * All duration and timestamp properties are normalized to milliseconds (ms). */ export interface OverviewResult { + threadId?: string; bounds: { minMs: number; maxMs: number; @@ -65,10 +66,10 @@ export interface OverviewResult { */ export interface LcpBreakdownInsight { totalLcpMs: number; - ttfbMs: number; - loadDelayMs: number; - loadDurationMs: number; - renderDelayMs: number; + ttfbMs: number | null; + loadDelayMs: number | null; + loadDurationMs: number | null; + renderDelayMs: number | null; lcpElement?: string; lcpUrl?: string; } @@ -117,6 +118,7 @@ export interface ThirdPartyEntityInsight { */ export interface InsightsResult { navigationId?: string; + insightSetId?: string; lcpBreakdown?: LcpBreakdownInsight | null; inpBreakdown?: InpBreakdownInsight | null; clsCulprits?: ClsCulpritInsight[]; @@ -130,6 +132,7 @@ export interface InsightsResult { */ export interface TaskItem { startTimeMs: number; + callPath?: string[]; durationMs: number; selfTimeMs: number; functionName: string; @@ -142,10 +145,24 @@ export interface TaskItem { * Aggregated main thread task execution details. */ export interface TasksResult { + threadId?: string; + warnings?: string[]; mode: 'list' | 'top-down' | 'bottom-up'; tasks: TaskItem[]; } +export interface ThreadItem { + id: string; + name: string | null; + frameUrls: string[]; + isDefault: boolean; + hasCpuSamples: boolean; +} + +export interface ThreadsResult { + threads: ThreadItem[]; +} + /** * Individual network request details and timing waterfall. */ @@ -207,11 +224,7 @@ export interface InteractionsResult { /** * DOM node identified as a layout shift culprit. */ -export interface ShiftCulprit { - nodeId?: number; - tagName?: string; - selector?: string; - debugName?: string; +export interface ShiftCulprit extends ShiftImpactedNodeItem { scoreDelta?: number; } @@ -220,6 +233,8 @@ export interface ShiftCulprit { */ export interface ShiftImpactedNodeItem { nodeId?: number; + processId?: number; + frameId?: string; tagName?: string; selector?: string; debugName?: string; @@ -263,6 +278,7 @@ export interface ShiftsResult { * Individual invocation instance of a searched function. */ export interface FindCallInstance { + functionName?: string; timestampMs: number; durationMs: number; url?: string; @@ -284,7 +300,9 @@ export interface FindIntervalItem { * Result of searching for a specific function's executions across a trace. */ export interface FindResult { - functionQuery: string; + threadId?: string; + functionQuery?: string; + urlQuery?: string; timeRange?: { startMs: number; endMs: number }; totalCalls: number; totalDurationMs: number; @@ -304,6 +322,7 @@ export interface ProfileBucketItem { totalActiveMs: number; topFunctions: Array<{ name: string; + url?: string; selfTimeMs: number; selfTimePct: number; }>; @@ -313,6 +332,8 @@ export interface ProfileBucketItem { * Time-bucketed CPU profiling result. */ export interface ProfileResult { + threadId?: string; + warnings?: string[]; timeRange: { startMs: number; endMs: number }; bucketSizeMs: number; buckets: ProfileBucketItem[]; @@ -335,6 +356,7 @@ export interface DiffEntryItem { * Differential profile result comparing two trace intervals. */ export interface DiffResult { + threadId?: string; windowA: { startMs: number; endMs: number; durationMs: number }; windowB: { startMs: number; endMs: number; durationMs: number }; entries: DiffEntryItem[]; diff --git a/src/formatters/index.ts b/src/formatters/index.ts index ed739f8..8b7f874 100644 --- a/src/formatters/index.ts +++ b/src/formatters/index.ts @@ -21,6 +21,7 @@ export interface FormatOptions { | 'overview' | 'insights' | 'tasks' + | 'threads' | 'network' | 'interactions' | 'shifts' @@ -36,13 +37,15 @@ export interface FormatOptions { const NEXT_STEPS_HINT_TOON = ` # Drill-down commands: +# - trace-engine-cli report [--sections ] # - trace-engine-cli insights [--category ] [--navigation ] -# - trace-engine-cli tasks [--min-duration ] [--tree ] [--top ] -# - trace-engine-cli find --function [--time-range ] +# - trace-engine-cli tasks [--min-duration ] [--tree ] [--url-only] [--top ] +# - trace-engine-cli find [--function ] [--url ] [--time-range ] +# - trace-engine-cli threads # - trace-engine-cli profile [--time-range ] [--bucket ] # - trace-engine-cli diff --window-a --window-b # - trace-engine-cli scheduling [--time-range ] -# - trace-engine-cli network [--render-blocking] [--slowest ] [--initiator-chain ] +# - trace-engine-cli network [--render-blocking] [--slowest ] [--initiator-chain ] [--strip-query] # - trace-engine-cli interactions # - trace-engine-cli shifts [--events] # - trace-engine-cli events [--name ] [--cat ] [--fields ] @@ -55,13 +58,15 @@ const NEXT_STEPS_HINT_TOON = ` const NEXT_STEPS_HINT_MD = ` --- ### Drill-down Commands +- \`trace-engine-cli report [--sections ]\` - \`trace-engine-cli insights [--category ] [--navigation ]\` -- \`trace-engine-cli tasks [--min-duration ] [--tree ] [--top ]\` -- \`trace-engine-cli find --function [--time-range ]\` +- \`trace-engine-cli tasks [--min-duration ] [--tree ] [--url-only] [--top ]\` +- \`trace-engine-cli find [--function ] [--url ] [--time-range ]\` +- \`trace-engine-cli threads \` - \`trace-engine-cli profile [--time-range ] [--bucket ]\` - \`trace-engine-cli diff --window-a --window-b \` - \`trace-engine-cli scheduling [--time-range ]\` -- \`trace-engine-cli network [--render-blocking] [--slowest ] [--initiator-chain ]\` +- \`trace-engine-cli network [--render-blocking] [--slowest ] [--initiator-chain ] [--strip-query]\` - \`trace-engine-cli interactions \` - \`trace-engine-cli shifts [--events]\` - \`trace-engine-cli events [--name ] [--cat ] [--fields ]\` diff --git a/src/formatters/markdown.ts b/src/formatters/markdown.ts index b9af784..3c73edd 100644 --- a/src/formatters/markdown.ts +++ b/src/formatters/markdown.ts @@ -24,8 +24,10 @@ import type { ProfileResult, ScanResult, SchedulingResult, + ShiftImpactedNodeItem, ShiftsResult, TasksResult, + ThreadsResult, } from '../engine/types.js'; import { formatBytes, formatMs, formatScore } from '../utils/units.js'; @@ -60,7 +62,11 @@ export function formatOverviewMarkdown(overview: OverviewResult): string { ); lines.push(''); - lines.push('## Main Thread Time Breakdown'); + lines.push( + overview.threadId + ? `## Thread ${overview.threadId} Time Breakdown` + : '## Main Thread Time Breakdown', + ); lines.push(''); lines.push('| Category | Duration (ms) | Share |'); lines.push('|---|---|---|'); @@ -101,15 +107,23 @@ export function formatInsightsMarkdown(insights: InsightsResult): string { lines.push(`- **Navigation ID**: \`${insights.navigationId}\``, ''); } + if (insights.insightSetId) lines.push(`- **Insight Set ID**: \`${insights.insightSetId}\``, ''); + if (insights.lcpBreakdown) { const l = insights.lcpBreakdown; lines.push('## LCP Breakdown'); lines.push(''); lines.push(`- **Total LCP**: \`${formatMs(l.totalLcpMs)}\``); - lines.push(`- **TTFB**: \`${formatMs(l.ttfbMs)}\``); - lines.push(`- **Resource Load Delay**: \`${formatMs(l.loadDelayMs)}\``); - lines.push(`- **Resource Load Duration**: \`${formatMs(l.loadDurationMs)}\``); - lines.push(`- **Element Render Delay**: \`${formatMs(l.renderDelayMs)}\``); + lines.push(`- **TTFB**: \`${l.ttfbMs === null ? 'N/A' : formatMs(l.ttfbMs)}\``); + lines.push( + `- **Resource Load Delay**: \`${l.loadDelayMs === null ? 'N/A' : formatMs(l.loadDelayMs)}\``, + ); + lines.push( + `- **Resource Load Duration**: \`${l.loadDurationMs === null ? 'N/A' : formatMs(l.loadDurationMs)}\``, + ); + lines.push( + `- **Element Render Delay**: \`${l.renderDelayMs === null ? 'N/A' : formatMs(l.renderDelayMs)}\``, + ); if (l.lcpElement) lines.push(`- **Element**: \`${l.lcpElement}\``); if (l.lcpUrl) lines.push(`- **Resource URL**: \`${l.lcpUrl}\``); lines.push(''); @@ -181,6 +195,8 @@ export function formatInsightsMarkdown(insights: InsightsResult): string { export function formatTasksMarkdown(tasksResult: TasksResult): string { const lines: string[] = [`# Tasks (${tasksResult.mode})`, '']; + if (tasksResult.threadId) lines.push(`Renderer thread \`${tasksResult.threadId}\``, ''); + if (tasksResult.warnings?.length) lines.push(...tasksResult.warnings, ''); if (tasksResult.tasks.length === 0) { lines.push('No tasks found matching criteria.'); return lines.join('\n'); @@ -191,7 +207,7 @@ export function formatTasksMarkdown(tasksResult: TasksResult): string { for (const t of tasksResult.tasks) { const loc = t.url ? `${t.url}${t.lineNumber !== undefined ? `:${t.lineNumber}` : ''}` : '-'; lines.push( - `| ${formatMs(t.startTimeMs)} | ${formatMs(t.durationMs)} | ${formatMs(t.selfTimeMs)} | \`${t.functionName}\` | \`${loc}\` |`, + `| ${formatMs(t.startTimeMs)} | ${formatMs(t.durationMs)} | ${formatMs(t.selfTimeMs)} | \`${t.callPath?.join(' > ') ?? t.functionName}\` | \`${loc}\` |`, ); } return lines.join('\n'); @@ -245,18 +261,17 @@ export function formatInteractionsMarkdown(interactions: InteractionsResult): st return lines.join('\n'); } -function formatCulpritLabel(item: { - debugName?: string; - selector?: string; - tagName?: string; - nodeId?: number; -}): string { - const detail = - item.selector || - item.tagName || - (item.nodeId !== undefined ? `Node #${item.nodeId}` : undefined); +function formatCulpritLabel(item: ShiftImpactedNodeItem): string { + const detail = item.selector || item.tagName; if (item.debugName && detail) return `${item.debugName} (${detail})`; - return item.debugName || detail || 'unknown'; + if (item.debugName || detail) return item.debugName || detail || ''; + const node = item.nodeId !== undefined ? `Node #${item.nodeId}` : 'Unknown node'; + const process = item.processId !== undefined ? ` (PID ${item.processId})` : ''; + const movement = + item.oldRect && item.newRect + ? ` [${item.oldRect.join(', ')}] → [${item.newRect.join(', ')}]` + : ''; + return `${node}${process}${movement}`; } export function formatShiftsMarkdown(shifts: ShiftsResult): string { @@ -314,7 +329,12 @@ export function formatShiftsMarkdown(shifts: ShiftsResult): string { } export function formatFindMarkdown(findResult: FindResult): string { - const lines: string[] = [`# Function Execution: \`${findResult.functionQuery}\``, '']; + const heading = findResult.functionQuery + ? `# Function Execution: \`${findResult.functionQuery}\`` + : '# Function Execution'; + const lines: string[] = [heading, '']; + if (findResult.urlQuery) lines.push(`- **URL**: \`${findResult.urlQuery}\``); + if (findResult.threadId) lines.push(`- **Renderer thread**: \`${findResult.threadId}\``); lines.push(`- **Total Calls**: \`${findResult.totalCalls}\``); lines.push(`- **Total Duration**: \`${formatMs(findResult.totalDurationMs)}\``); lines.push(`- **Average Duration**: \`${formatMs(findResult.avgDurationMs)}\``); @@ -342,11 +362,13 @@ export function formatFindMarkdown(findResult: FindResult): string { if (findResult.calls.length > 0) { lines.push('## Call Invocations'); lines.push(''); - lines.push('| Timestamp (ms) | Duration (ms) | Location |'); - lines.push('|---|---|---|'); + lines.push('| Timestamp (ms) | Duration (ms) | Function | Location |'); + lines.push('|---|---|---|---|'); for (const c of findResult.calls) { const loc = c.url ? `${c.url}${c.lineNumber !== undefined ? `:${c.lineNumber}` : ''}` : '-'; - lines.push(`| ${formatMs(c.timestampMs)} | ${formatMs(c.durationMs)} | \`${loc}\` |`); + lines.push( + `| ${formatMs(c.timestampMs)} | ${formatMs(c.durationMs)} | ${c.functionName ?? '-'} | \`${loc}\` |`, + ); } lines.push(''); } @@ -360,6 +382,9 @@ export function formatProfileMarkdown(profileResult: ProfileResult): string { '', ]; + if (profileResult.threadId) lines.push(`Renderer thread \`${profileResult.threadId}\``, ''); + if (profileResult.warnings?.length) lines.push(...profileResult.warnings, ''); + if (profileResult.buckets.length === 0) { lines.push('No CPU activity found in the specified range.'); return lines.join('\n'); @@ -368,7 +393,9 @@ export function formatProfileMarkdown(profileResult: ProfileResult): string { lines.push('| Time Bucket | Active CPU (ms) | Top Functions (% of slice) |'); lines.push('|---|---|---|'); for (const b of profileResult.buckets) { - const topSummary = b.topFunctions.map((f) => `${f.name} (${f.selfTimePct}%)`).join(', '); + const topSummary = b.topFunctions + .map((f) => `${f.name} (${f.selfTimePct}%)${f.url ? ` \`${f.url}\`` : ''}`) + .join(', '); lines.push( `| ${formatMs(b.startMs)} - ${formatMs(b.endMs)} | ${formatMs(b.totalActiveMs)} | ${topSummary || 'None (Idle)'} |`, ); @@ -378,6 +405,7 @@ export function formatProfileMarkdown(profileResult: ProfileResult): string { export function formatDiffMarkdown(diffResult: DiffResult): string { const lines: string[] = ['# Differential Profile', '']; + if (diffResult.threadId) lines.push(`Renderer thread \`${diffResult.threadId}\``, ''); lines.push( `- **Window A**: \`${formatMs(diffResult.windowA.startMs)} - ${formatMs(diffResult.windowA.endMs)}\` (${formatMs(diffResult.windowA.durationMs)})`, ); @@ -562,6 +590,21 @@ export function formatScanMarkdown(result: ScanResult): string { return lines.join('\n'); } +function formatThreadsMarkdown({ threads }: ThreadsResult): string { + const lines = [ + '# Renderer Threads', + '', + '| PID:TID | Thread | Frame URLs | Default | CPU samples |', + '|---|---|---|---|---|', + ]; + for (const thread of threads) { + lines.push( + `| ${thread.id} | ${thread.name ?? '-'} | ${thread.frameUrls.join(', ') || '-'} | ${thread.isDefault ? 'Yes' : 'No'} | ${thread.hasCpuSamples ? 'Yes' : 'No'} |`, + ); + } + return lines.join('\n'); +} + export function formatMarkdown(data: unknown, type?: string): string { switch (type) { case 'overview': @@ -570,6 +613,8 @@ export function formatMarkdown(data: unknown, type?: string): string { return formatInsightsMarkdown(data as InsightsResult); case 'tasks': return formatTasksMarkdown(data as TasksResult); + case 'threads': + return formatThreadsMarkdown(data as ThreadsResult); case 'network': return formatNetworkMarkdown(data as NetworkResult); case 'interactions': diff --git a/test/cli.test.ts b/test/cli.test.ts index 2cc0daa..142515c 100644 --- a/test/cli.test.ts +++ b/test/cli.test.ts @@ -17,6 +17,7 @@ import { mkdtemp, rm, writeFile } from 'node:fs/promises'; import os from 'node:os'; import path from 'node:path'; import test, { describe } from 'node:test'; +import { decode } from '@toon-format/toon'; import { runCli } from '../src/cli.js'; async function captureCliOutput(args: string[]): Promise { @@ -68,6 +69,52 @@ describe('CLI Dispatcher', () => { assert.ok(output.includes('## Main Thread Time Breakdown')); }); + test('runs several requested report sections with the same range and result shapes', async () => { + const fixture = path.resolve(import.meta.dirname, 'fixtures/navigation-cwv-trace.json'); + const args = ['node', 'trace-engine-cli']; + const options = ['--time-range', '400,525']; + const reportArgs = [...args, 'report', fixture, '--sections', 'tasks,network,tasks']; + const output = decode(await captureCliOutput([...reportArgs, ...options])); + assert.deepEqual(output, { + tasks: decode(await captureCliOutput([...args, 'tasks', fixture, ...options])), + network: decode(await captureCliOutput([...args, 'network', fixture, ...options])), + }); + const jsonOptions = [...options, '--format', 'json']; + const json = JSON.parse(await captureCliOutput([...reportArgs, ...jsonOptions])); + assert.deepEqual(json, { + tasks: JSON.parse(await captureCliOutput([...args, 'tasks', fixture, ...jsonOptions])), + network: JSON.parse(await captureCliOutput([...args, 'network', fixture, ...jsonOptions])), + }); + + const markdown = await captureCliOutput([...args, 'report', fixture, '--format', 'md']); + assert.match(markdown, /# Trace Overview.*# Tasks.*# Layout Shifts.*# Network Requests/s); + const toon = await captureCliOutput([...args, 'report', fixture, '--sections', 'overview']); + assert.match(toon, /^overview:/); + }); + + test('rejects unknown report sections before parsing the trace', async () => { + const args = [ + 'node', + 'trace-engine-cli', + 'report', + 'unavailable.json', + '--sections', + 'tasks,unknown', + ]; + await assert.rejects(() => captureCliOutput(args), /Unknown report section "unknown"/); + }); + + test('honors relative time ranges for navigations and insight selection', async () => { + const fixture = path.resolve(import.meta.dirname, 'fixtures/navigation-cwv-trace.json'); + const args = ['node', 'trace-engine-cli']; + const range = ['--time-range', '0,1']; + assert.match(await captureCliOutput([...args, 'navigations', fixture, ...range]), /total: 0/); + assert.match( + await captureCliOutput([...args, 'insights', fixture, '--navigation', 'NAV_1', ...range]), + /No insights found for navigation ID: NAV_1 in the selected time range/, + ); + }); + test('runs tasks subcommand with relative time range', async () => { const fixturePath = path.resolve(import.meta.dirname, 'fixtures/basic-trace.json'); const output = await captureCliOutput([ @@ -100,6 +147,49 @@ describe('CLI Dispatcher', () => { assert.ok(output.includes('totalCalls: 1')); }); + test('lists renderer threads in Markdown and JSON', async () => { + const fixture = path.resolve(import.meta.dirname, 'fixtures/navigation-cwv-trace.json'); + const args = ['node', 'trace-engine-cli', 'threads', fixture]; + const threads = await captureCliOutput([...args, '--format', 'md']); + assert.match(threads, /100:1.*CrRendererMain/); + const json = JSON.parse(await captureCliOutput([...args, '--format', 'json'])); + assert.equal(json.threads[0].id, '100:1'); + assert.equal(json.threads[0].name, 'CrRendererMain'); + }); + + test('finds named calls by URL on the chosen renderer thread', async () => { + const fixture = path.resolve(import.meta.dirname, 'fixtures/navigation-cwv-trace.json'); + const cli = ['node', 'trace-engine-cli']; + const args = [...cli, 'find', fixture, '--thread', '100:1', '--url', 'app.js']; + assert.match(await captureCliOutput(args), /bootApp/); + assert.match(await captureCliOutput([...args, '--function', 'missing']), /totalCalls: 0/); + await assert.rejects( + () => captureCliOutput([...args.slice(0, 4), '--thread', '999:1', '--url', 'app.js']), + /Unknown renderer thread.*threads /, + ); + }); + + test('rejects non-positive result limits and bucket sizes', async () => { + const fixture = path.resolve(import.meta.dirname, 'fixtures/basic-trace.json'); + const args = ['node', 'trace-engine-cli']; + await assert.rejects( + () => captureCliOutput([...args, 'tasks', fixture, '--top', '-5']), + /positive integers/, + ); + await assert.rejects( + () => captureCliOutput([...args, 'profile', fixture, '--bucket', '0']), + /--bucket must be at least/, + ); + }); + + test('requires a function or URL for find', async () => { + const fixture = path.resolve(import.meta.dirname, 'fixtures/basic-trace.json'); + await assert.rejects( + () => captureCliOutput(['node', 'trace-engine-cli', 'find', fixture]), + /Specify --function or --url/, + ); + }); + test('runs profile subcommand with relative range and bucket units', async () => { const fixturePath = path.resolve(import.meta.dirname, 'fixtures/basic-trace.json'); const output = await captureCliOutput([ diff --git a/test/extractors/insights.test.ts b/test/extractors/insights.test.ts index 4af5c27..c27981c 100644 --- a/test/extractors/insights.test.ts +++ b/test/extractors/insights.test.ts @@ -13,10 +13,24 @@ // limitations under the License. import assert from 'node:assert/strict'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import os from 'node:os'; import path from 'node:path'; -import test, { before, describe } from 'node:test'; +import test, { before, describe, type TestContext } from 'node:test'; import { extractInsights } from '../../src/engine/extractors/insights.js'; -import { type LoadedTrace, loadTraceFromFile } from '../../src/engine/loader.js'; +import { extractOverview } from '../../src/engine/extractors/overview.js'; +import { getPropertyByPath } from '../../src/engine/helpers.js'; +import { type LoadedTrace, loadTraceFromFile, parseTraceJson } from '../../src/engine/loader.js'; + +async function loadCwvVariant(t: TestContext, update: (events: unknown[]) => unknown[]) { + const fixture = path.resolve(import.meta.dirname, '../fixtures/navigation-cwv-trace.json'); + const events = parseTraceJson(await readFile(fixture, 'utf8')); + const directory = await mkdtemp(path.join(os.tmpdir(), 'trace-cwv-')); + t.after(() => rm(directory, { recursive: true, force: true })); + const file = path.join(directory, 'trace.json'); + await writeFile(file, JSON.stringify(update(events))); + return loadTraceFromFile(file); +} describe('Insights Extractor', () => { let loaded: LoadedTrace; @@ -28,20 +42,18 @@ describe('Insights Extractor', () => { test('extractInsights returns structured insights result', () => { const result = extractInsights(loaded); - assert.ok(result.thirdParties); - assert.equal(result.thirdParties.length, 1); - assert.equal(result.thirdParties[0].entityName, 'example.com'); - assert.equal(result.thirdParties[0].transferSizeBytes, 5000); + assert.equal(result.navigationId, 'NAV_1'); + assert.equal(result.insightSetId, 'NAVIGATION_0'); + assert.equal(result.lcpBreakdown, undefined); + assert.equal(result.thirdParties, undefined); }); test('extractInsights routes to specific navigationId when specified', () => { const result = extractInsights(loaded, { - navigationId: 'NAV_1', + navigationId: 'NAVIGATION_0', }); assert.equal(result.navigationId, 'NAV_1'); - assert.ok(result.thirdParties); - assert.equal(result.thirdParties.length, 1); - assert.equal(result.thirdParties[0].entityName, 'example.com'); + assert.equal(result.insightSetId, 'NAVIGATION_0'); }); test('extractInsights adds warning when navigationId is not found', () => { @@ -108,6 +120,107 @@ describe('Insights Extractor', () => { ); }); + test('uses the same default navigation across paint metrics and CLS insights', async (t) => { + function shift(ts: number, selector: string) { + return { + name: 'LayoutShift', + cat: 'loading', + ph: 'I', + pid: 100, + tid: 1, + ts, + args: { + frame: 'FRAME_0', + data: { + is_main_frame: true, + had_recent_input: false, + score: 0.15, + weighted_score_delta: 0.15, + impacted_nodes: [{ node_id: 42, selector }], + }, + }, + }; + } + const trace = await loadCwvVariant(t, (events) => { + const navIndex = events.findIndex( + (event) => getPropertyByPath(event, 'name') === 'navigationStart', + ); + assert.ok(navIndex >= 0); + events.splice( + navIndex, + 0, + { + name: 'navigationStart', + cat: 'blink.user_timing', + ph: 'R', + pid: 100, + tid: 1, + ts: 1010000, + args: { + frame: 'FRAME_0', + data: { + documentLoaderURL: 'https://example.com/earlier', + navigationId: 'NAV_EARLY', + isOutermostMainFrame: true, + isInPrimaryMainFrame: true, + }, + }, + }, + { + name: 'firstContentfulPaint', + cat: 'loading,rail,devtools.timeline', + ph: 'R', + pid: 100, + tid: 1, + ts: 1025000, + args: { frame: 'FRAME_0', data: { navigationId: 'NAV_EARLY' } }, + }, + shift(1030000, '#early'), + ); + return [...events, shift(1475000, '#selected')]; + }); + + const insights = extractInsights(trace); + assert.equal(insights.navigationId, 'NAV_1'); + assert.equal(insights.clsCulprits?.length, 1); + assert.deepEqual(insights.clsCulprits[0].culpritNodes, ['#selected']); + const earlier = extractInsights(trace, { navigationId: 'NAV_EARLY' }); + assert.equal(earlier.clsCulprits?.length, 1); + assert.deepEqual(earlier.clsCulprits[0].culpritNodes, ['#early']); + + const { fcp, lcp } = extractOverview(trace).coreWebVitals; + assert.equal(fcp?.valueMs, 2050); + assert.equal(lcp?.valueMs, 4150); + }); + + test('keeps measured LCP when document request timing is missing', async (t) => { + const trace = await loadCwvVariant(t, (events) => + events.filter((event) => getPropertyByPath(event, 'args.data.requestId') !== 'NAV_1'), + ); + const result = extractInsights(trace, { category: 'lcp' }); + assert.equal(result.lcpBreakdown?.totalLcpMs, 4150); + assert.equal(result.lcpBreakdown?.ttfbMs, null); + assert.equal(result.lcpBreakdown?.loadDelayMs, null); + assert.equal(result.lcpBreakdown?.loadDurationMs, null); + assert.equal(result.lcpBreakdown?.renderDelayMs, null); + assert.match(result.warnings?.join('\n') ?? '', /LCP was measured.*no timing breakdown/); + }); + + test('reports third-party requests and excludes first-party requests', async (t) => { + const trace = await loadCwvVariant(t, (events) => + parseTraceJson( + JSON.stringify(events).replaceAll( + 'https://example.com/font.woff2', + 'https://cdn.example.net/font.woff2', + ), + ), + ); + const result = extractInsights(trace, { category: 'third-parties' }); + assert.equal(result.thirdParties?.length, 1); + assert.match(result.thirdParties[0].entityName, /example\.net/); + assert.equal(result.thirdParties[0].transferSizeBytes, 16384); + }); + test('extractInsights parses CLS culprits for shifts-trace', async () => { const fixturePath = path.resolve(import.meta.dirname, '../fixtures/shifts-trace.json'); const shiftsLoaded = await loadTraceFromFile(fixturePath); @@ -117,7 +230,7 @@ describe('Insights Extractor', () => { { score: 0.15, timestampMs: 1300, - culpritNodes: ['Node #42'], + culpritNodes: ['#banner'], }, ]); }); diff --git a/test/extractors/network.test.ts b/test/extractors/network.test.ts index 5a07b70..2db790d 100644 --- a/test/extractors/network.test.ts +++ b/test/extractors/network.test.ts @@ -13,6 +13,8 @@ // limitations under the License. import assert from 'node:assert/strict'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import os from 'node:os'; import path from 'node:path'; import test, { describe } from 'node:test'; import { extractNetwork } from '../../src/engine/extractors/network.js'; @@ -112,4 +114,25 @@ describe('Network Extractor', () => { assert.equal(result.initiatorChain[2].initiatorUrl, undefined); assert.equal(result.initiatorChain[2].type, 'text/html'); }); + + test('strips queries in the request list and each initiator hop after matching', async (t) => { + const source = path.resolve(import.meta.dirname, '../fixtures/navigation-cwv-trace.json'); + const original = await readFile(source, 'utf8'); + const dir = await mkdtemp(path.join(os.tmpdir(), 'trace-cli-')); + t.after(() => rm(dir, { recursive: true, force: true })); + const fixture = path.join(dir, 'query-strings.json'); + const withQueries = original + .replaceAll('https://example.com/font.woff2', 'https://example.com/font.woff2?token=font') + .replaceAll('https://example.com/styles.css', 'https://example.com/styles.css?token=css'); + await writeFile(fixture, withQueries); + const loaded = await loadTraceFromFile(fixture); + + const result = extractNetwork(loaded, { initiatorChain: 'token=font', stripQuery: true }); + assert.equal(result.initiatorChain?.length, 3); + assert.equal(result.initiatorChain[0].url, 'https://example.com/font.woff2'); + assert.equal(result.initiatorChain[0].initiatorUrl, 'https://example.com/styles.css'); + assert.ok(result.requests.some((request) => request.url === 'https://example.com/font.woff2')); + assert.doesNotMatch(JSON.stringify(result), /token=/); + assert.match(JSON.stringify(extractNetwork(loaded)), /token=font/); + }); }); diff --git a/test/extractors/overview.test.ts b/test/extractors/overview.test.ts index 1dce1ca..24d8ff3 100644 --- a/test/extractors/overview.test.ts +++ b/test/extractors/overview.test.ts @@ -13,11 +13,13 @@ // limitations under the License. import assert from 'node:assert/strict'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import os from 'node:os'; import path from 'node:path'; import test, { describe } from 'node:test'; import { extractOverview } from '../../src/engine/extractors/overview.js'; import { extractShifts } from '../../src/engine/extractors/shifts.js'; -import { loadTraceFromFile } from '../../src/engine/loader.js'; +import { loadTraceFromFile, parseTraceJson } from '../../src/engine/loader.js'; describe('Overview Extractor', () => { test('extractOverview parses metadata and main thread stats for basic-trace', async () => { @@ -82,6 +84,61 @@ describe('Overview Extractor', () => { assert.equal(activeAndIdleSum, overview.mainThreadBreakdown.totalMs); }); + test('keeps timeline categories stable when CPU samples add nested calls', async (t) => { + const fixture = path.resolve(import.meta.dirname, '../fixtures/navigation-cwv-trace.json'); + const events = parseTraceJson(await readFile(fixture, 'utf8')); + const dir = await mkdtemp(path.join(os.tmpdir(), 'trace-cli-')); + t.after(() => rm(dir, { recursive: true, force: true })); + const sampled = path.join(dir, 'sampled.json'); + const profile = { + cat: 'disabled-by-default-v8.cpu_profiler', + ph: 'P', + pid: 100, + tid: 1, + id: '0x1', + }; + const profileStart = { + ...profile, + name: 'Profile', + ts: 1405000, + args: { data: { startTime: 1405000 } }, + }; + const profileChunk = { + ...profile, + name: 'ProfileChunk', + ts: 1465000, + args: { + data: { + cpuProfile: { + nodes: [ + { id: 1, callFrame: { functionName: '(root)', scriptId: '0' } }, + { + id: 2, + parent: 1, + callFrame: { + functionName: 'sampledFunction', + scriptId: '1', + url: 'https://example.com/app.js', + lineNumber: 43, + columnNumber: 1, + }, + }, + ], + samples: [2, 2, 2], + }, + timeDeltas: [10000, 20000, 30000], + }, + }, + }; + await writeFile(sampled, JSON.stringify([...events, profileStart, profileChunk])); + + const original = extractOverview(await loadTraceFromFile(fixture)); + const withSamples = extractOverview(await loadTraceFromFile(sampled)); + assert.ok(original.warnings.some((warning) => warning.includes('cpu_profiler'))); + assert.ok(!withSamples.warnings.some((warning) => warning.includes('cpu_profiler'))); + assert.deepEqual(withSamples.mainThreadBreakdown, original.mainThreadBreakdown); + }); + test('extractOverview parses interaction-to-next-paint for interaction-trace', async () => { const fixturePath = path.resolve(import.meta.dirname, '../fixtures/interaction-trace.json'); const loaded = await loadTraceFromFile(fixturePath); diff --git a/test/extractors/shifts.test.ts b/test/extractors/shifts.test.ts index 19ed696..d5b78c2 100644 --- a/test/extractors/shifts.test.ts +++ b/test/extractors/shifts.test.ts @@ -13,10 +13,14 @@ // limitations under the License. import assert from 'node:assert/strict'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import os from 'node:os'; import path from 'node:path'; import test, { describe } from 'node:test'; import { extractShifts } from '../../src/engine/extractors/shifts.js'; -import { loadTraceFromFile } from '../../src/engine/loader.js'; +import { getPropertyByPath } from '../../src/engine/helpers.js'; +import { loadTraceFromFile, parseTraceJson } from '../../src/engine/loader.js'; +import { formatResult } from '../../src/formatters/index.js'; describe('Layout Shifts Extractor', () => { test('extractShifts warns when the recording has no layout shift data', async () => { @@ -62,6 +66,48 @@ describe('Layout Shifts Extractor', () => { assert.equal(insideResult.clusters.length, 1); }); + test('keeps distinct culprits with missing node metadata and shows recorded rectangles', async (t) => { + const fixture = path.resolve(import.meta.dirname, '../fixtures/shifts-trace.json'); + const events = parseTraceJson(await readFile(fixture, 'utf8')).filter( + (event) => getPropertyByPath(event, 'name') !== 'LayoutShift', + ); + events.push({ + name: 'LayoutShift', + cat: 'loading', + ph: 'I', + pid: 100, + tid: 1, + ts: 1300000, + args: { + frame: 'FRAME_0', + data: { + is_main_frame: true, + had_recent_input: false, + weighted_score_delta: 0.15, + impacted_nodes: [ + { selector: '#first' }, + { debug_name: 'second' }, + { node_id: 42, old_rect: [0, 1, 40, 20], new_rect: [0, 7, 40, 20] }, + { old_rect: [0, 2, 5, 6], new_rect: [0, 4, 5, 6] }, + { old_rect: [0, 3, 5, 6], new_rect: [0, 5, 5, 6] }, + ], + }, + }, + }); + const directory = await mkdtemp(path.join(os.tmpdir(), 'trace-shifts-')); + t.after(() => rm(directory, { recursive: true, force: true })); + const file = path.join(directory, 'trace.json'); + await writeFile(file, JSON.stringify(events)); + + const result = extractShifts(await loadTraceFromFile(file)); + assert.equal(result.clusters[0].culprits.length, 5); + const output = formatResult(result, { format: 'md', type: 'shifts' }); + assert.match(output, /#first/); + assert.match(output, /second/); + assert.match(output, /Node #42 \(PID 100\) \[0, 1, 40, 20\] → \[0, 7, 40, 20\]/); + assert.match(output, /DOM names and selectors were unavailable/); + }); + test('extractShifts extracts individual events when includeEvents is true', async () => { const fixturePath = path.resolve(import.meta.dirname, '../fixtures/shifts-trace.json'); const loaded = await loadTraceFromFile(fixturePath); diff --git a/test/extractors/tasks.test.ts b/test/extractors/tasks.test.ts index bf44e5e..48deda4 100644 --- a/test/extractors/tasks.test.ts +++ b/test/extractors/tasks.test.ts @@ -13,10 +13,12 @@ // limitations under the License. import assert from 'node:assert/strict'; +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; +import os from 'node:os'; import path from 'node:path'; import test, { describe } from 'node:test'; import { extractTasks } from '../../src/engine/extractors/tasks.js'; -import { loadTraceFromFile } from '../../src/engine/loader.js'; +import { loadTraceFromFile, parseTraceJson } from '../../src/engine/loader.js'; describe('Tasks Extractor', () => { test('extractTasks extracts top-level tasks above threshold', async () => { @@ -89,10 +91,56 @@ describe('Tasks Extractor', () => { assert.equal(bottomUpResult.mode, 'bottom-up'); assert.ok(bottomUpResult.tasks.length > 0); assert.equal(bottomUpResult.tasks[0].functionName, 'bootApp'); + assert.equal(bottomUpResult.tasks[0].startTimeMs, 1405); assert.equal(bottomUpResult.tasks[0].selfTimeMs, 70); assert.equal(bottomUpResult.tasks[0].durationMs, 70); assert.equal(bottomUpResult.tasks[0].url, 'https://example.com/app.js'); assert.equal(bottomUpResult.tasks[0].lineNumber, 42); assert.equal(bottomUpResult.tasks[0].columnNumber, 8); }); + + test('shows nested calls in the top-down tree and applies URL filtering before the limit', async () => { + const fixture = path.resolve(import.meta.dirname, '../fixtures/navigation-cwv-trace.json'); + const loaded = await loadTraceFromFile(fixture); + const topDown = extractTasks(loaded, { tree: 'top-down' }); + const call = topDown.tasks.find((task) => task.functionName === 'bootApp'); + assert.deepEqual(call?.callPath, ['RunTask', 'bootApp']); + + const filtered = extractTasks(loaded, { tree: 'top-down', top: 1, urlOnly: true }); + assert.equal(filtered.tasks[0]?.url, 'https://example.com/app.js'); + assert.equal(filtered.tasks[0]?.functionName, 'bootApp'); + }); + + test('preserves ranking ties and applies URL eligibility before the limit', async (t) => { + const fixture = path.resolve(import.meta.dirname, '../fixtures/navigation-cwv-trace.json'); + const events = parseTraceJson(await readFile(fixture, 'utf8')); + const dir = await mkdtemp(path.join(os.tmpdir(), 'trace-cli-')); + t.after(() => rm(dir, { recursive: true, force: true })); + const file = path.join(dir, 'ranked-tasks.json'); + const runTask = { + name: 'RunTask', + cat: 'disabled-by-default-devtools.timeline', + ph: 'X', + pid: 100, + tid: 1, + }; + const taskEvents = [ + { ...runTask, ts: 5500000, dur: 180000 }, + { ...runTask, ts: 5800000, dur: 180001 }, + { ...runTask, ts: 6100000, dur: 180002 }, + { ...runTask, ts: 6400000, dur: 180003 }, + ]; + await writeFile(file, JSON.stringify([...events, ...taskEvents])); + + const loaded = await loadTraceFromFile(file); + const longest = extractTasks(loaded, { top: 2 }); + const startTimes = longest.tasks.map((task) => task.startTimeMs); + assert.deepEqual(startTimes, [5500, 5800]); + + const withUrl = extractTasks(loaded, { top: 1, urlOnly: true }); + assert.equal(withUrl.tasks[0]?.functionName, 'RunTask > bootApp'); + + const bottomUp = extractTasks(loaded, { tree: 'bottom-up', top: 1, urlOnly: true }); + assert.equal(bottomUp.tasks[0]?.functionName, 'bootApp'); + }); });