Add public docs for TP Python API
Bug: 163311671
Change-Id: I41eaa351a1591eac52908625edd1280793dc19b5
diff --git a/docs/analysis/trace-processor.md b/docs/analysis/trace-processor.md
index 1214d59..060dc3d 100644
--- a/docs/analysis/trace-processor.md
+++ b/docs/analysis/trace-processor.md
@@ -421,6 +421,185 @@
and alerts added on these instead; this is because the trace processor
storage is monotonic-append-only.
+## Python API
+
+The trace processor Python API is built on the existing HTTP interface of `trace processor`
+and is available as part of the standalone build. The API allows you to load in traces and
+query tables and run metrics without requiring the `trace_processor` binary to be
+downloaded or installed.
+
+### Setup
+Note: The API is only compatible with Python3.
+
+```
+from trace_processor.api import TraceProcessor
+# Initialise TraceProcessor with a trace file
+tp = TraceProcessor(file_path='trace.pftrace')
+```
+
+NOTE: The TraceProcessor can be initialized in a combination of ways including:
+ <br> - An address at which there exists a running instance of `trace_processor` with a
+ loaded trace (e.g. `TraceProcessor(addr='localhost:9001')`)
+ <br> - An address at which there exists a running instance of `trace_processor` and
+ needs a trace to be loaded in
+ (e.g. `TraceProcessor(addr='localhost:9001', file_path='trace.pftrace')`)
+ <br> - A path to a `trace_processor` binary and the trace to be loaded in
+ (e.g. `TraceProcessor(bin_path='./trace_processor', file_path='trace.pftrace')`)
+
+
+### API
+
+The `trace_processor.api` module contains the `TraceProcessor` class which provides various
+functions that can be called on the loaded trace. For more information on how to use
+these functions, see this [`example`](/src/trace_processor/python/example.py).
+
+#### Query
+The query() function takes an SQL query as input and returns an iterator through the rows
+of the result.
+
+```
+from trace_processor.api import TraceProcessor
+tp = TraceProcessor(file_path='trace.pftrace')
+
+qr_it = tp.query('SELECT ts, dur, name FROM slice')
+for row in qr_it:
+ print(row.ts, row.dur, row.name)
+```
+**Output**
+```
+261187017446933 358594 eglSwapBuffersWithDamageKHR
+261187017518340 357 onMessageReceived
+261187020825163 9948 queueBuffer
+261187021345235 642 bufferLoad
+261187121345235 153 query
+...
+```
+The QueryResultIterator can also be converted to a Pandas DataFrame, although this
+requires you to have both the `NumPy` and `Pandas` modules installed.
+```
+from trace_processor.api import TraceProcessor
+tp = TraceProcessor(file_path='trace.pftrace')
+
+qr_it = tp.query('SELECT ts, dur, name FROM slice')
+qr_df = qr_it.as_pandas()
+print(qr_df.to_string())
+```
+**Output**
+```
+ts dur name
+-------------------- -------------------- ---------------------------
+ 261187017446933 358594 eglSwapBuffersWithDamageKHR
+ 261187017518340 357 onMessageReceived
+ 261187020825163 9948 queueBuffer
+ 261187021345235 642 bufferLoad
+ 261187121345235 153 query
+ ...
+```
+Furthermore, you can use the query result in a Pandas DataFrame format to easily
+make visualisations from the trace data.
+```
+from trace_processor.api import TraceProcessor
+tp = TraceProcessor(file_path='trace.pftrace')
+
+qr_it = tp.query('SELECT ts, value FROM counter WHERE track_id=50')
+qr_df = qr_it.as_pandas()
+qr_df = qr_df.replace(np.nan,0)
+qr_df = qr_df.set_index('ts')['value'].plot()
+```
+**Output**
+
+[](/docs/images/example_pd_graph.png)
+
+
+#### Metric
+The metric() function takes in a list of trace metrics and returns the results as a Protobuf.
+
+```
+from trace_processor.api import TraceProcessor
+tp = TraceProcessor(file_path='trace.pftrace')
+
+ad_cpu_metrics = tp.metric(['android_cpu'])
+print(ad_cpu_metrics)
+```
+**Output**
+```
+metrics {
+ android_cpu {
+ process_info {
+ name: "/system/bin/init"
+ threads {
+ name: "init"
+ core {
+ id: 1
+ metrics {
+ mcycles: 1
+ runtime_ns: 570365
+ min_freq_khz: 1900800
+ max_freq_khz: 1900800
+ avg_freq_khz: 1902017
+ }
+ }
+ core {
+ id: 3
+ metrics {
+ mcycles: 0
+ runtime_ns: 366406
+ min_freq_khz: 1900800
+ max_freq_khz: 1900800
+ avg_freq_khz: 1902908
+ }
+ }
+ ...
+ }
+ ...
+ }
+ process_info {
+ name: "/system/bin/logd"
+ threads {
+ name: "logd.writer"
+ core {
+ id: 0
+ metrics {
+ mcycles: 8
+ runtime_ns: 33842357
+ min_freq_khz: 595200
+ max_freq_khz: 1900800
+ avg_freq_khz: 1891825
+ }
+ }
+ core {
+ id: 1
+ metrics {
+ mcycles: 9
+ runtime_ns: 36019300
+ min_freq_khz: 1171200
+ max_freq_khz: 1900800
+ avg_freq_khz: 1887969
+ }
+ }
+ ...
+ }
+ ...
+ }
+ ...
+ }
+}
+```
+
+### HTTP
+The `trace_processor.http` module contains the `TraceProcessorHttp` class which
+provides methods to make HTTP requests to an address at which there already
+exists a running instance of `trace_processor` with a trace loaded in. All
+results are returned in Protobuf format
+(see [`trace_processor_proto`](/protos/perfetto/trace_processor.proto)).
+Some functions include:
+* `execute_query()` - Takes in an SQL query and returns a `QueryResult` Protobuf
+ message
+* `compute_metric()` - Takes in a list of trace metrics and returns a
+ `ComputeMetricResult` Protobuf message
+* `status()` - Returns a `StatusResult` Protobuf message
+
+
## Testing
Trace processor is mainly tested in two ways:
diff --git a/docs/images/example_pd_graph.png b/docs/images/example_pd_graph.png
new file mode 100644
index 0000000..e8d55c2
--- /dev/null
+++ b/docs/images/example_pd_graph.png
Binary files differ
diff --git a/docs/quickstart/trace-analysis.md b/docs/quickstart/trace-analysis.md
index 3f3a41f..5d637a2 100644
--- a/docs/quickstart/trace-analysis.md
+++ b/docs/quickstart/trace-analysis.md
@@ -1,14 +1,16 @@
# Quickstart: SQL-based analysis and trace-based metrics
-_This quickstart explains how to use `trace_processor` to programmatically query
-the trace contents through SQL and compute trace-based metrics._
+_This quickstart explains how to use `trace_processor` as well as its Python API to
+programmatically query the trace contents through SQL and compute trace-based metrics._
-## Get Trace Processor
+## Trace Processor
TraceProcessor is a multi-format trace importing and query engine based on
SQLite. It comes both as a C++ library and as a standalone executable:
`trace_processor_shell` (or just `trace_processor`).
+### Setup
+
```bash
# Download prebuilts (Linux and Mac only)
curl -LO https://get.perfetto.dev/trace_processor
@@ -30,12 +32,12 @@
See [Trace Processor docs](/docs/analysis/trace-processor.md) for the full
TraceProcessor guide.
-## Sample queries
+### Sample queries
For more exhaustive examples see the _SQL_ section of the various _Data sources_
docs.
-### Slices
+#### Slices
Slices are stackable events which have name and span some duration of time.
@@ -53,7 +55,7 @@
...
```
-### Counters
+#### Counters
Counters are events with a value which changes over time.
@@ -72,7 +74,7 @@
...
```
-### Scheduler slices
+#### Scheduler slices
Scheduler slices indicate which thread was scheduled on which CPU at which time.
@@ -90,7 +92,7 @@
...
```
-## Trace-based metrics
+### Trace-based metrics
Trace Processor offers also a higher-level query interface that allows to run
pre-baked queries, herein called "metrics". Metrics are generally curated by
@@ -104,7 +106,7 @@
The corresponding SQL queries live in
[/src/trace_processor/metrics](/src/trace_processor/metrics/).
-### Run a single metric
+#### Run a single metric
Let's run the [`android_cpu`](/protos/perfetto/metrics/android/cpu_metric.proto)
metric. This metrics computes the total CPU time and the total cycles
@@ -175,7 +177,7 @@
}
```
-### Running multiple metrics
+#### Running multiple metrics
Multiple metrics can be flagged using comma separators to the `--run-metrics`
flag. This will output a text proto with the combined result of running both
@@ -236,7 +238,7 @@
}
```
-### JSON and binary output
+#### JSON and binary output
The trace processor also supports binary protobuf and JSON as alternative output
formats. This is useful when the intended reader is an offline tool.
@@ -313,6 +315,99 @@
}
```
+## Python API
+
+The API can be run without requiring the `trace_processor` binary to be
+downloaded or installed.
+
+### Example functions
+See the Python API section of
+[Trace Processor (SQL)](/docs/analysis/trace-processor.md) to get
+more details on all available functions.
+
+#### Query
+```
+from trace_processor.api import TraceProcessor
+tp = TraceProcessor(file_path='trace.pftrace')
+
+qr_it = tp.query('SELECT name FROM slice')
+for row in qr_it:
+ print(row.name)
+```
+**Output**
+```
+eglSwapBuffersWithDamageKHR
+onMessageReceived
+queueBuffer
+bufferLoad
+query
+...
+```
+#### Query as Pandas DataFrame
+```
+from trace_processor.api import TraceProcessor
+tp = TraceProcessor(file_path='trace.pftrace')
+
+qr_it = tp.query('SELECT ts, name FROM slice')
+qr_df = qr_it.as_pandas()
+print(qr_df.to_string())
+```
+**Output**
+```
+ts name
+-------------------- ---------------------------
+ 261187017446933 eglSwapBuffersWithDamageKHR
+ 261187017518340 onMessageReceived
+ 261187020825163 queueBuffer
+ 261187021345235 bufferLoad
+ 261187121345235 query
+ ...
+```
+#### Metric
+```
+from trace_processor.api import TraceProcessor
+tp = TraceProcessor(file_path='trace.pftrace')
+
+cpu_metrics = tp.metric(['android_cpu'])
+print(cpu_metrics)
+```
+**Output**
+```
+metrics {
+ android_cpu {
+ process_info {
+ name: "/system/bin/init"
+ threads {
+ name: "init"
+ core {
+ id: 1
+ metrics {
+ mcycles: 1
+ runtime_ns: 570365
+ min_freq_khz: 1900800
+ max_freq_khz: 1900800
+ avg_freq_khz: 1902017
+ }
+ }
+ core {
+ id: 3
+ metrics {
+ mcycles: 0
+ runtime_ns: 366406
+ min_freq_khz: 1900800
+ max_freq_khz: 1900800
+ avg_freq_khz: 1902908
+ }
+ }
+ ...
+ }
+ ...
+ }
+ ...
+ }
+}
+```
+
## Next steps
There are several options for exploring more of the trace analysis features
