More work on report subcommand
diff --git a/0021-trace-processor-report-subcommand.md b/0021-trace-processor-report-subcommand.md index 6a2a05f..c026422 100644 --- a/0021-trace-processor-report-subcommand.md +++ b/0021-trace-processor-report-subcommand.md
@@ -19,7 +19,7 @@ None of these serve the "I just collected a trace, what's in it?" use case well. Users coming from `perf report` expect to point a tool at a data file -and immediately see an opinionated, useful summary — no query authoring, no +and immediately see an opinionated, useful summary: no query authoring, no spec files, no UI. This gap is especially felt by: @@ -28,12 +28,7 @@ - **AI tools** that need structured trace summaries to reason about performance. - **CI pipelines** that want a human-readable (or machine-parseable) trace - summary as a build artifact. - -The Firefox Profiler project is exploring a similar direction with their -experimental `pq` CLI tool (PR 5663 in the firefox-devtools/profiler repo), -which provides opinionated per-dimension views of profiling data from the -command line. + summary as a build artifact, and to compare traces across builds. ## Decision @@ -43,48 +38,68 @@ ### Relationship to `summarize` -`report` is a higher-level, opinionated cousin of `summarize`: +The two subcommands split by who decides what is computed: -- **`summarize`** is the general-purpose engine — users author custom - `TraceSummarySpec` protos to define exactly what to compute. -- **`report`** ships built-in specs that produce useful defaults across - known trace dimensions. +- **`summarize`** is for summaries the user asks for themselves: they + author `TraceSummarySpec` protos defining exactly what to compute. +- **`report`** is opinionated summaries we give the user: built-in views + over the common trace dimensions, with no authoring required. -Under the hood, `report` is built entirely on top of the summarization -machinery. Each dimension's report is a pre-authored `TraceSummarySpec` that -gets fed into the same engine that `summarize` uses. +Under the hood both are PerfettoSQL. `summarize` converts query results to +protos as directed by the user's spec, with typed schemas and semantics +such as higher/lower-is-better; `report` is a presentation layer over the +PerfettoSQL standard library, where each view is computed by curated +stdlib queries and rendered from a hand-written proto schema +(`report.proto`, one message per view). Two things follow: -### Built-in spec embedding - -Report specs are authored as human-readable textproto files in the source tree -(e.g. `src/trace_processor/shell/report_specs/*.textproto`). A build rule -converts these to binary proto and embeds them as byte arrays in the binary, -following the existing `perfetto_cc_proto_descriptor` pattern used for metric -and trace descriptors. This means: - -- Zero file I/O at runtime — specs are baked into the binary. -- Specs are human-editable in the source tree. -- The same build infrastructure that handles descriptor embedding is reused. +- **Typed output from a fixed proto.** `summarize` derives its output + schema from the user's spec because the computation is arbitrary; + `report`'s views are a closed, compile-time set, so a hand-written proto + gives stable text/JSON output directly. +- **Report improvements land in the stdlib.** Every gap a report exposes is + fixed in a module all SQL users benefit from, and the queries shown in + the SQL drill-down hints are the same ones the tool runs. ### CLI surface ``` -trace_processor_shell report [dimension] [FLAGS] <trace_file> +tp report [<noun> [<view>]] [FLAGS] (<trace_file> | --remote <addr>) ``` -When no dimension is specified, produce an overview covering all applicable -dimensions (skipping those with no data in the trace). When a dimension is -specified, produce a detailed per-dimension report. +The first positional is always a noun (what data you are looking at); the +second is always a view (how to see it), from a closed per-noun set with a +default: -#### Dimensions +| Noun | Views (default first) | +| --------------- | -------------------------------------------- | +| (none) | overview | +| `processes` | `table` | +| `tracks` | `table` | +| `slices` | `table`, `histogram`, `timeline`, `inspect` | +| `stack-samples` | `functions`, `top-down`, `bottom-up` | +| `heap-profile` | `functions`, `top-down`, `bottom-up` | +| `heap-dump` | `types` | +| `scheduling` | `summary`, `timeline`, `inspect` | +| `counters` | `list`, `timeline`, `histogram`, `inspect` | -| Dimension | Description | -| ---------------- | -------------------------------------------------- | -| `slices` | Slice aggregations (wall duration, count, max) | -| `stack-samples` | CPU profiling samples (self/total time) | -| `heap-profile` | Heap allocation profiling (bytes, count) | -| `heap-dump` | Heap snapshot analysis (retained size, objects) | -| `scheduling` | Thread scheduling (CPU time, runnable, wait time) | +``` +tp report trace.pftrace # overview +tp report stack-samples top-down trace.pftrace +tp report slices histogram --name doFrame --remote calm-blue-otter +tp report scheduling inspect --thread RenderThread --ts 2.734s trace.pftrace +tp report slices --baseline base.pftrace target.pftrace # diff two traces +``` + +With no noun, the overview covers every noun that has data in the trace, +skipping the rest. `processes`, `tracks`, and `counters list` exist for +structural discovery: they enumerate the names that the scoping flags and +the other views take as input, so a user can orient in an unfamiliar trace +before drilling down. Output shape is statically known per (noun, view). + +The nouns are deliberately generic trace primitives, applicable to any +trace regardless of origin. Domain-specific reporting (e.g. Android frame +timeline, ANRs, battery) is out of scope for now; it may be added in the +future. #### Output format flags @@ -92,8 +107,19 @@ --format text|json Output format (default: text). ``` -- **`text`**: Human-readable tables, similar to `perf report --stdio`. -- **`json`**: Structured JSON object, for tool/AI consumption. +- **`text`**: human-readable tables, similar to `perf report --stdio`. +- **`json`**: one stable schema per (noun, view), generated from + `report.proto`, with the scope context embedded and richer stats than + text (e.g. p50/p95/p99 where text shows min/avg/max). Text output is + explicitly not a stable interface; JSON is. + +``` +--show-sql Print the stdlib queries the view executed. +``` + +`--show-sql` is what makes the SQL escape hatch concrete: the printed +queries are the ones the view ran, and a starting point for going beyond +what `report` offers. #### Scoping flags @@ -106,199 +132,364 @@ --thread <name> Scope to a thread by name. --track <name> Scope to a track by name. --cpu <cpu> Scope to a specific CPU. ---time <start>,<end> Scope to a time range. - Accepts raw nanoseconds or human-friendly - format (e.g. 2.7s,3.1s). +--time <start>,<end> Scope to a time range. Accepts raw nanoseconds or + human-friendly format (e.g. 2.7s,3.1s). +--ts <ts> Scope to a point in time (same formats as --time). + Used by the inspect views. ``` -Scoping flags are translated into structured query filters and -`interval_intersect` clauses in the underlying `TraceSummarySpec`, using the -existing DSL primitives — no raw SQL WHERE clauses. +Scoping flags are translated into structured filters on the underlying +stdlib queries. Every invocation carries its full scope in flags; there is +no sticky state, so commands are reproducible, CI-safe, and safe for +concurrent clients of one warm session. -#### Aggregation control +#### Filtering and aggregation control ``` ---top <N> Number of entries per section (default: 10). +--name <glob> Row filter, same meaning in every view. In + bottom-up, rows are roots, so the callers of + function X are: stack-samples bottom-up --name X. + Single-series views (histogram, timeline, inspect + on slices/counters) require the glob to match + exactly one distinct name; if it matches several, + the command fails and lists them. Many instances + of one name are fine (see inspect). +--min-duration <dur> Hide rows below a duration threshold. +--min-count <N> Hide rows with fewer instances. +--sort <key> Sort key, enumerated per view (e.g. dur|count|max). +--top <N> Entries per table section (default: 10). +--max-lines <N> Node budget for tree views (default: 100). +--buckets <N> Bucket count for timeline/histogram views. +--group-by <key> Grouping key where a view supports one + (scheduling summary: thread (default) | process). ``` +#### Warm sessions + +``` +--remote <addr> Run against a warm session (see RFC 0031); + mutually exclusive with a trace path. +``` + +RFC 0031's `--remote` subcommand list grows `report`. The interactive +drill-down loop (overview -> view -> inspect) is a rapid sequence of +invocations against one trace, the pattern warm sessions exist to serve. + ### Overview output -When invoked without a dimension, the overview produces a one-line trace -context followed by per-dimension aggregated highlights. +When invoked without a noun, the overview produces a one-line trace context +followed by per-noun highlights, ending with drill-down hints. The overview +accepts the scoping flags and `--top` (applied per section); `--name` and +`--sort` are rejected, since the sections have disjoint name spaces and +sort keys. Example (`--format text`): ``` -Trace: 12.3s | Android 14 | Pixel 7 Pro | 12 processes | 48 threads | 156 tracks +[trace.pftrace | full trace | 12.3s | Android 14 | Pixel 7 Pro | 12 processes] Slices (12.3M total): Name Count Total dur % of trace Max dur - Choreographer#doFrame 83.2k 4.1s 33.2% 128ms - DrawFrame 83.1k 3.5s 28.4% 96ms - measure 41.6k 890ms 7.2% 42ms - layout 41.6k 620ms 5.0% 38ms - dequeueBuffer 24.9k 310ms 2.5% 12ms - eglSwapBuffers 24.9k 280ms 2.3% 8ms - RenderThread::draw 24.9k 240ms 1.9% 6ms - BinderTransaction 12.1k 180ms 1.5% 52ms - animation 8.3k 120ms 1.0% 4ms - inflate 2.1k 95ms 0.8% 18ms + Choreographer#doFrame 83.2k 4.1s 33.2% 128ms @ 2.734s (RenderThread) + DrawFrame 83.1k 3.5s 28.4% 96ms @ 5.101s (RenderThread) + measure 41.6k 890ms 7.2% 42ms @ 8.913s (main) + ... + * 210 more slice names below --top 10; rerun with --top 50 Stack Samples (3.2k total): Function Self% Total% Samples art::Thread::RunRootClock 18.2% 42.1% 583 __epoll_pwait 12.1% 12.1% 387 - art::interpreter::Execute 8.4% 31.2% 269 ... Scheduling: Thread CPU time Runnable Sleeping % of trace RenderThread 3.2s 120ms 8.9s 26.0% mali-cmar-backe 1.8s 45ms 10.4s 14.6% - HeapTaskDaemon 890ms 12ms 11.3s 7.2% ... Heap Profile: (not present in trace) Heap Dump: (not present in trace) + +Next: tp report processes trace.pftrace + tp report slices trace.pftrace --top 50 + tp report slices inspect --name "Choreographer#doFrame" --ts 2.734s --thread RenderThread trace.pftrace ``` -### Per-dimension detail +### View semantics -Per-dimension reports provide a deeper view. For example, -`tp report slices <trace>` would show the same columns as the overview but -with a higher default `--top` and potentially additional breakdowns (e.g. -per-thread grouping). +#### Tree views -The exact content of per-dimension reports is left as an open question for -now. As noted below, the call-tree views for stack samples (top-down / -bottom-up, as seen in `perf report` and the Firefox Profiler's `pq` tool) -are a natural fit here but the exact interaction model needs more thought. +The tree views (`top-down`, `bottom-up`) are never dumped in full. Nodes +are included best-first under the `--max-lines` budget, scored by total% +with a depth decay (internal, not user-configurable). Single-child chains +are collapsed to the same indent, and pruned subtrees are summarized: -### Per-dimension column definitions +``` +└─ ... (5 more children: combined 12.3%, max 4.1%) +``` -#### Slices +`functions` and the tree views show both self and total, with self +displayed more prominently. -Default aggregation key: slice name. +#### `timeline` and `histogram` -| Column | Description | -| ------------ | ---------------------------------------------------- | -| Name | Slice name | -| Count | Number of instances | -| Total dur | Sum of wall durations across all instances | -| % of trace | Total duration as percentage of trace duration | -| Max dur | Maximum single-instance duration (outlier detection) | +`timeline` buckets the trace (or the `--time` range) and renders one line +per bucket: an ASCII bar, the value, and the dominant entries in that +bucket. `scheduling timeline` needs no selection (CPU load per bucket, +plus the threads that dominate each bucket); `slices timeline` and +`counters timeline` take a `--name` and show +count and total duration, or average value, per bucket. -#### Stack Samples +`histogram` renders the distribution of a single series: durations of a +slice name, or values of a counter. The output labels its scale (linear or +logarithmic) so bucket widths are not misread; JSON carries percentile +stats alongside the buckets. -Modeled after `perf report --stdio`. +#### `inspect` -| Column | Description | -| --------- | ------------------------------------------------------- | -| Function | Function name (symbol) | -| Self% | Samples where this function is at the top of the stack | -| Total% | Samples where this function appears anywhere in stack | -| Samples | Absolute sample count | +`inspect` drills from a noun's aggregate into instances. The noun's normal +filters select the entity; scoping flags narrow which instances match. The +output shape is fixed per noun: aggregate stats over the matched instances +plus the instances themselves, rendered in full detail when exactly one +matches. `--name` must resolve to one distinct name (usually copy-pasted +from report output); many instances of that name are not an error: the +output is a wider instance table plus a hint on how to narrow. -#### Heap Profile +- `slices inspect --name <n>`: instance count, duration percentiles, top + instances by duration (each carrying its ts/thread for narrowing). A + unique match shows dur, self dur, args, parent chain, and children. +- `counters inspect --name <n>`: value at `--ts`, plus min/avg/max and + delta over the scoped range. +- `scheduling inspect --thread <t> --ts <ts>`: state, CPU, end state, + waker thread, and wakeup latency: "why was this thread not running at T". -Same shape as stack samples but with bytes instead of sample count. +Adding an inspect view to a noun must pass a two-part test: (1) instances +are addressable by name or coordinates, never tool-emitted ids; (2) it +answers an instance-level question the aggregate views do not. -| Column | Description | -| ------------ | ---------------------------------------------------- | -| Allocator | Allocation site / function | -| Self bytes | Bytes allocated directly by this function | -| Total bytes | Bytes allocated by this function and its callees | -| Count | Number of allocations | -| Avg size | Average allocation size | +**Coverage gap:** `stack-samples`, `heap-profile`, and `heap-dump` have no +inspect view yet. A per-source function inspect (callers/callees of one +function) is tractable under this grammar, since the noun fixes the data +source, but is left as future work; heap-graph objects remain id-only. +Until then, the tree views plus `--name` and the SQL hints cover these. -#### Heap Dump +### Diffing -Point-in-time memory snapshot. +Comparing two traces of the same workload (before/after a change, across +builds in CI) is an explicit goal. Any table view, and the overview, +accepts: -| Column | Description | -| ------------- | --------------------------------------------------- | -| Type/Alloc | Type or allocator | -| Retained size | Total retained memory | -| Live objects | Count of live objects | +``` +--baseline <trace|addr> Compare against a baseline trace. Accepts a path + or a warm-session address (as with --remote). +``` -#### Scheduling +Diff is a presentation-layer join: the view's queries run independently on +each trace, and the two typed results are joined on the view's aggregation +key (slice name, function, thread, process, track, or counter name) in the +shell. Neither engine ever sees both traces; multi-trace queries are out +of scope, both because they are very hard to reason about and because +sessions are single-trace (RFC 0031). Because the two runs are +independent, target and baseline may each be a path or a warm-session +address, in any combination. -Per-thread scheduling summary. +Semantics: -| Column | Description | -| ------------ | ---------------------------------------------------- | -| Thread | Thread name | -| CPU time | Total time spent running on a CPU | -| Runnable | Total time in runnable state (waiting for CPU) | -| Sleeping | Total time sleeping | -| % of trace | CPU time as percentage of trace duration | +- Value columns become Baseline / Target / Delta with signed formatting. + Default sort is the duration delta; `--sort` selects others (e.g. count + delta). Rows present on only one side are marked added/removed and rank + by absolute delta. +- Deltas are raw values. The context header shows both trace durations and + a footnote warns when they differ materially; `report` does not attempt + to normalize for workload size and makes no claim of statistical + significance. +- The join is by exact name. Canonicalizing names that embed dynamic + values (tids, addresses) is an open question. +- Scoping flags apply to both traces identically. -### Sources of inspiration +``` +[target.pftrace 10.1s | baseline base.pftrace 12.3s | full trace] -- **`perf report`** (Linux perf): Opinionated defaults, hierarchical views, - sort-by-overhead, `--stdio` output. The gold standard for "point at data, - get useful summary." -- **Firefox Profiler `pq`**: CLI profile querying with per-dimension - formatters, top-down/bottom-up call trees, scoping via time ranges, dual - human/JSON output (PR 5663 in the firefox-devtools/profiler repo). -- **`pprof`** (Go): `-top`, `-text` views for CPU/heap profiles. Ergonomic - top-N function summaries. -- **`heaptrack`** (KDE): CLI heap profile summaries — peak consumption, top - allocators, leak candidates. +Slices (vs base.pftrace): + Name Baseline dur Target dur Delta + Choreographer#doFrame 4.1s 5.0s +900ms + inflate 95ms 2ms -93ms + BitmapDecode (absent) 310ms +310ms + * baseline is 22% longer than target; deltas are not normalized +``` + +`tp report --baseline base.pftrace target.pftrace` diffs the overview: a +per-noun digest of the biggest movements, the intended CI build artifact. +Tree and single-series views do not accept `--baseline` initially; +structural tree diffs need their own design (see open questions). A flag +is easier to miss than a subcommand, so `--baseline` is included in the +`report` help text's example block (as above). + +### Per-view column definitions + +#### Slices `table` + +Aggregation key: slice name. + +| Column | Description | +| ---------- | ------------------------------------------------------- | +| Name | Slice name | +| Count | Number of instances | +| Total dur | Sum of wall durations across all instances | +| % of trace | Total duration as percentage of trace duration | +| Max dur | Maximum single-instance duration, with the instance's | +| | ts/thread (outlier detection, feeds the inspect hint) | + +The noun covers sync and async slice tracks; instants (zero duration) +contribute to Count only. Durations sum per instance without overlap +deduplication, so % of trace can exceed 100 on async-heavy traces. + +#### `stack-samples functions` + +| Column | Description | +| -------- | ------------------------------------------------------- | +| Function | Function name (symbol) | +| Self% | Samples where this function is at the top of the stack | +| Total% | Samples where this function appears anywhere in stack | +| Samples | Absolute sample count | + +#### `heap-profile functions` + +| Column | Description | +| ----------- | ---------------------------------------------------- | +| Function | Allocation site / function | +| Self bytes | Bytes allocated directly by this function | +| Total bytes | Bytes allocated by this function and its callees | +| Count | Number of allocations | +| Avg size | Average allocation size | + +#### `heap-dump types` + +| Column | Description | +| ------------- | -------------------------------------------------- | +| Type | Type or allocator | +| Retained size | Total retained memory | +| Live objects | Count of live objects | + +Retained size needs a dominator-tree computation and the Total%/Total +bytes columns need callstack ancestry walks; these are the most expensive +views to compute, and a further reason `report` benefits from warm +sessions. + +#### `scheduling summary` + +| Column | Description | +| ---------- | ----------------------------------------------------- | +| Name | Thread (or process, with --group-by process) | +| CPU time | Total time spent running on a CPU | +| Runnable | Total time in runnable state (waiting for CPU) | +| Sleeping | Total time sleeping | +| % of trace | CPU time as percentage of trace duration | + +#### `processes` / `tracks` / `counters list` + +Discovery tables: Process, PID, CPU time, % of trace, top threads; Track, +Type, Event count, Process/Thread; Counter name, Track/Process, Sample +count, Min, Avg, Max. + +### Output contract + +Every view emits: + +1. **A context header** echoing the effective scope, derived purely from + the invocation (see the overview example). +2. **Truncation footnotes**: what was hidden and the flag to reveal it. +3. **Next-command hints**: complete, copy-pasteable drill-down invocations + (including `--remote` and scope flags), including `tp query` commands + for raw data. The intended progression is: overview -> aggregate view + -> inspect -> SQL. +4. **Terminal-aware tables**: columns are width-fitted when stdout is a + TTY; piped or redirected output is untruncated. +5. **Errors fail fast**: an unknown noun/view combination, a missing + required flag (e.g. `--ts` for `scheduling inspect`), or a malformed + value exits non-zero with a message listing the valid options; with + `--format json` the error is a structured object. + +### Testing + +Text output is locked by trace processor diff tests: one golden per +(noun, view) over reference traces, so any output change is explicit in +review and documented behavior cannot drift from shipped behavior. JSON is +validated against `report.proto` by construction. ## Alternatives considered -### Ship report specs as external files +### Build on the summarize machinery Pro: -* Users can inspect and modify specs without rebuilding. +- Reuses an existing declarative engine; specs shareable with `summarize`. Con: -* Requires distributing spec files alongside the binary. -* File discovery and path resolution adds complexity. -* Embedded binary protos are zero-overhead and follow existing precedent - (metric descriptors, trace descriptors). +- Tree views have no representation in the flat-row summary output. +- Scoping flags rewrite the computation per invocation, so the + "pre-authored spec" is really a template rewritten per invocation. +- Typed output does not require the engine: a fixed proto provides it. -### Hardcode aggregation queries in C++ +### Alternative grammars -Pro: +Two other grammars were drafted: a `--view` flag on a noun positional, and +view-first positionals with data-selection flags (`top-down +--heap-profile`). Both rejected: noun-then-view matches how users approach +a trace (pick the data, then the presentation) and how comparable +heterogeneous profiling tools have converged, keeps each view a closed +per-noun set, and has in-repo precedent (`tp server http`). -* No proto serialization overhead. +### Sticky server-side session state -Con: +Holding analysis state (a selected thread, zoom/filter stacks) in a +long-lived server makes interactive drill-down terse, but single +invocations become non-reproducible and concurrent clients unsafe. Warm +sessions (RFC 0031) already give the real benefit, parse-once performance, +so `report` keeps all state in flags. -* Loses the declarative nature of the summarization DSL. -* Cannot be reused by the `summarize` subcommand. -* Harder to maintain and review. +### Id-based inspect addressing -### Combine with `summarize` +`inspect slice:187432` style handles: rejected. Ids are only knowable if +the tool emitted them and are unstable across trace loads and versions; +names and coordinates (thread, ts) are legible from any output. Id-precision +lookups remain available via the SQL bridge. -Pro: +### A `--focus <function>` stack filter -* One subcommand to learn. +A pprof-style focus transform (keep stacks containing X, re-root per view): +rejected as a second drill-down mechanism with subtle semantics (recursion +double-counting, per-view re-rooting). `bottom-up --name` covers the +callers workflow; re-rooting is left as an open question. -Con: +### A separate `diff` subcommand -* `summarize` is for custom specs; overloading it with opinionated defaults - muddies its purpose. -* Different flag surfaces (scoping flags vs spec paths) would conflict. +`tp report diff <base> <target>` was considered. Rejected: it would need to +replicate every noun/view under itself, whereas `--baseline` composes with +the existing views and the overview for free, and inherits scoping flags. ## Open questions -* **Per-dimension drill-down interaction model:** For stack samples, top-down - and bottom-up call trees (à la `perf report` and Firefox Profiler's `pq` - tool) are a natural fit. Should these be sub-sub-commands - (`tp report stack-samples top-down <trace>`), flags - (`--view top-down`), or sections within the same output? -* **Exact per-dimension report content:** The overview columns are defined - above. The detailed per-dimension reports may include additional breakdowns - (e.g. per-thread slice grouping, per-process scheduling). Exact content - TBD. -* **Spec authoring:** The built-in specs need to be written against the - existing PerfettoSQL stdlib tables and modules. The exact table/module - references for each dimension need to be determined. -* **Trace metadata extraction:** The one-line context line (OS, device, - duration, process/thread/track counts) may require queries outside the - summarization DSL. How to handle this cleanly? +- **Domain-specific reports:** Android frame timeline, ANRs, battery and + similar have stdlib modules but are out of scope for now; they may be + added in the future. +- **Inspect for the callstack/heap nouns:** per-source function inspect + (callers/callees) and heap-graph object drill-down (see coverage gap). +- **Tree diffs:** delta-annotated call trees under `--baseline`; the + rendering and matching rules need their own design. +- **Per-trace time windows in diffs:** `--time` applies to both traces; + comparing different windows (e.g. startup vs startup at different + offsets) may need a `--baseline-time`. +- **Diff join canonicalization:** names embedding dynamic values (tids, + addresses) break the exact-name join into spurious added/removed pairs. +- **Diff noise:** accepting multiple baseline traces (min/median/max) to + separate real movement from run-to-run variance. +- **`--root <function>` on tree views:** pure re-rooting for "what does X + call", if hop-wise navigation proves insufficient. +- **A heap-dump retainers view:** retaining paths for a type. +- **Cross-sectional view:** "what was happening at time T" across all + nouns (the CLI analog of a vertical line in the UI). +- **JSON schema discovery:** how consumers find the per-view schemas + (help section vs a dedicated subcommand).