Improve documentation Bug: 73625827 Change-Id: I9d2e5d222abc8383e14f7f1172aa3f2833bd92c3
diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..7bc5a28 --- /dev/null +++ b/docs/architecture.md
@@ -0,0 +1,121 @@ +# Perfetto key concepts and architecture + +Producer <> Service <> Consumer model +------------------------------------- + + +**Service** +The tracing service is a long-lived entity (a system daemon on Linux/Android, +a service in Chrome) that has the following responsibilities: +- Maintains a registry of active producers and their data sources. +- Owns the trace buffers. +- Handles multiplexing of several tracing sessions. +- Routes the trace config from the consumers to the corresponding producers. +- Tells the Producers when and what to trace. +- Moves data from the Producer's shared memory buffer to the central non-shared + trace buffers. + +**Producer** +A producer is an untrusted entity that offers the ability to contribute to the +trace. In a multiprocess model, a producer almost always corresponds to a client +process of the tracing service. It advertises its ability to contribute to the trace with one or more data sources. +Each producer has exactly: +- One shared memory buffer, shared exclusively with the tracing service. +- One IPC channel with the tracing service. + +A producer is completely decoupled (both technically and conceptually) from +consumer(s). A producer knows nothing about: +- How many consumer(s) are connected to the service. +- How many tracing sessions are active. +- How many other producer(s) are registered or active. +- Trace data written by other producer(s). + +*** aside +In rare circumstances a process can host more than one producer and hence more +than one shared memory buffer. This can be the case for a process bundling +third-party libraries that in turn include the Perfetto client library. +Concrete example: at some point in the future Chrome might expose one Producer for tracing within the main project, one for V8 and one for Skia (for each child +process). +*** + +**Consumer** +A consumer is a trusted entity (a cmdline client on Linux/Android, an interface +of the Browser process in Chrome) that controls (non-exclusively) the tracing service and reads back (destructively) the trace buffers. +A consumer has the ability to: +- Send a [trace config](trace-config.md) to the service, determining: + - How many trace buffers to create. + - How big the trace buffers should be. + - The policy for each buffer (*ring-buffer* or *stop-when-full*). + - Which data sources to enable. + - The configuration for each data source. + - The target buffer for the data produced by each data source configured. +- Enable and disable tracing. +- Read back the trace buffers: + - Streaming data over the IPC channel. + - Passing a file descriptor to the service and instructing it to periodically + save the trace buffers into the file. + +**Data source** +A data source is a capability, exposed by a Producer, of providing some tracing +data. A data source almost always defines its own schema (a protobuf) consisting +of: +- At most one `DataSourceConfig` sub-message + ([example](/protos/perfetto/config/ftrace/ftrace_config.proto)) +- One or more `TracePacket` sub-messages + ([example](/protos/perfetto/trace/ps/process_tree.proto)) + +Different producers may expose the same data source. Concrete example: +*** aside +At some point in the near future we might offer, as part of Perfetto, a library +for in-process heap profiling. In such case more than one producer, linking +against the updated Perfetto library, will expose the heap profiler data source, +for its own process. +*** + +**IPC channel** +In a multiprocess scenario, each producer and each consumer interact with the +service using an IPC channel. IPC is used only in non-fast-path interactions, +mostly handshakes such as enabling/disabling trace (consumer), (un)registering +and starting/stopping data sources (producer). The IPC is typically NOT employed +to transport the protobufs for the trace. +Perfetto provides a POSIX-friendly IPC implementation, based on protobufs over a +UNIX socket (see [ipc.md](ipc.md)). That IPC implementation is not mandated. +Perfetto allows the embedder: +- Wrap its own IPC subsystem (e.g., Perfetto in Chromium uses Mojo) +- Not use an IPC mechanism at all and just short circuit the + Producer <> Service <> Consumer interaction via `PostTask(s)`. + +See [embedder-guide.md](embedder-guide.md) for more details. + + +**Shared memory buffer** +Producer(s) write tracing data, in the form of protobuf-encoded binary blobs, +directly into its shared memory buffer, using a special library called +[ProtoZero](protozero.md). The shared memory buffer: +- Has a fixed and typically small size (configurable, default: 128 KB). +- Is an ABI and must maintain backwards compatibility. +- Is shared by all data sources of the producer. +- Is independent of the number and the size of the trace buffers. +- Is independent of the number of Consumer(s). +- Is partitioned in *chunks* of variable size. + +Each chunk: +- Is owned exclusively by one Producer thread (or shared through a mutex). +- Contains a linear sequence of [`TracePacket(s)`](trace-format.md), or + fragments of that. A `TracePacket` can span across several chunks, the + fragmentation is not exposed to the consumers (consumers always see whole + packets as if they were never fragmented). +- Can be owned and written by exactly one `TraceWriter`. +- Is part of a reliable and ordered sequence, identified by the `WriterID`: + packets in a sequence are guaranteed to be read back in order, without gaps + and without repetitions (see [trace-format.md](trace-format.md) for more). + +See the comments in +[shared_memory_abi.h](/include/perfetto/tracing/core/shared_memory_abi.h) +for more details about the binary format of this buffer. + +Other resources +--------------- +* [Life of a tracing session](life-of-a-tracing-session.md) +* [Trace config](trace-config.md) +* [Trace format](trace-format.md)
diff --git a/docs/benchmarks.md b/docs/benchmarks.md new file mode 100644 index 0000000..0280a54 --- /dev/null +++ b/docs/benchmarks.md
@@ -0,0 +1,35 @@ +# Perfetto benchmarks + + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): Summarize results of perfetto_benchmarks --> +*** + +This doc should show the charts of `perfetto_benchmarks`, showing cpu usage and +tracing bandwidth for both writing (producer->service) and reading +(service->file / service->consumer). + +In two modes: +- Measure peak tracing bandwidth saturating the cpu: the producer(s) write as + much data as they can, reaching 100% cpu usage. +- Measure CPU overhead vs constant bandwidth: the producer(s) writes data at a + pre-defined rate. + +Tweaking the various parameters, such as: +- Number of writers +- Size of the shared memory buffer +- Size of each TracePacket. + +**TL;DR:** +Peak producer-to-service tracing bandwidth: +* Linux desktop: ~1.3 GB/s +* Android Pixel: ~1 GB/s + +Producer-to-service CPU overhead when writing ~3 MB/s: 0.01 - 0.03 +(0.01 := 1% cpu time of one core) + +CPU overhead for translating ftrace raw pipe into protobuf: +* Android Pixel: 0.00-0.01 when idle. +* Android Pixel: 0.02-0.04 with 8 cores @ 8.0 CPU usage (raytracer). +* Linux desktop: TBD
diff --git a/docs/build_instructions.md b/docs/build-instructions.md similarity index 60% rename from docs/build_instructions.md rename to docs/build-instructions.md index c938cd6..823f05b 100644 --- a/docs/build_instructions.md +++ b/docs/build-instructions.md
@@ -1,82 +1,100 @@ -# Perfetto - Build instructions +# Perfetto build instructions -Perfetto can be built both from the Android tree and standalone. -Standalone builds are meant only for local testing and are not shipped. +The source of truth for the Perfetto codebase currently lives in AOSP: +https://android.googlesource.com/platform/external/perfetto/ + +Perfetto can be built both from the Android tree (AOSP) and standalone. +Standalone builds are meant only for local testing and are not shipped. Due to the reduced dependencies they are faster to iterate on and the suggested way to work on Perfetto. -# Prerequisites -All dependent libraries are self-hosted and pulled by the -`tools/install-build-deps` script. -The only requirements on the host are -python and git: - -``` -$ sudo apt-get update && sudo apt-get install git python -``` - - -# Standalone checkout - Get the code ------------ +**Standalone checkout**: ``` -$ git clone https://android.googlesource.com/platform/external/perfetto.git +$ git clone https://android.googlesource.com/platform/external/perfetto/ ``` -Build ------ +**Android tree**: +Perfetto lives in `external/perfetto` in the AOSP tree. + + +Prerequisites +------------- +**Standalone checkout**: +All dependent libraries are self-hosted and pulled through: +``` +$ tools/install-build-deps [--no-android] +``` + +**Android tree**: +See https://source.android.com/setup + + +Building +-------- +**Standalone checkout**: If you are a chromium developer and have depot_tools installed you can avoid the `tools/` prefix below and just use gn/ninja from depot_tools. -`$ tools/install-build-deps` to install third-party build deps (NDK etc) - `$ tools/gn args out/android` to generate build files and enter in the editor: ``` -target_os = "android" # Leave empty for local testing -target_cpu = "arm" or "arm64" # Only when building for Android +target_os = "android" # Only when building for Android +target_cpu = "arm" / "arm64" / "x64" # Only when building for Android +is_debug = true / false ``` (See the [Build Configurations](#build-configurations) section below for more) ``` -$ tools/ninja -C out/android all +$ tools/ninja -C out/android ``` -This will generate artifacts under `out/android`. - - -# Inside the Android tree - -Get the code ------------- -See https://source.android.com/setup/building. -Perfetto lives in `external/perfetto` in the Android tree. - -Build ------ +**Android tree**: `$ mmma external/perfetto` or -`$ m perfetto traced traced_probes # put other target names here` +`$ m perfetto traced traced_probes` This will generate artifacts `out/target/product/XXX/system/`. Executables and shared libraries are stripped by default by the Android build -system. The unstripped versions are kept into `out/target/product/XXX/symbols`. +system. The unstripped artifacts are kept into `out/target/product/XXX/symbols`. -# GN -> Android.bp generator +Build files +----------- The source of truth of our build file is in the BUILD.gn files, which are based on [GN][gn-quickstart]. The Android build file ([Android.bp](../Android.bp)) is autogenerated from the GN files through `tools/gen_android_bp`, which needs to be invoked whenever a change -touches GN files or introduces new ones. +touches GN files or introduces new ones. A presubmit check checks that the Android.bp is consistent with GN files when -submitting a CL through `git cl upload`. +submitting a CL through `git cl upload`. The generator has a whitelist of root targets that will be translated into the Android.bp file. If you are adding a new target, add a new entry to the `default_targets` variable inside [tools/gen_android_bp](../tools/gen_android_bp). -# Build configurations +Supported platforms +------------------- +**Linux desktop** (Debian Rodete): + - Hermetic clang + libcxx toolchain (both following chromium's revisions) + - GCC-7 and libstdc++ 6 + +**Android**: + - Android's NDK r15c (using NDK's libcxx) + - AOSP's in-tree clang (using in-tree libcxx) + +**Mac**: + - XCode 9 / clang (currently maintained best-effort). + + + +Build configurations +-------------------- +*** aside +`tools/build_all_configs.py` can be used to generate out/XXX folders for most of +the supported configurations. +*** + The following [GN args][gn-quickstart] are supported: `target_os = "android" | "linux" | "mac"`: @@ -90,9 +108,10 @@ Toggles Debug (default) / Release mode. `is_clang = true | false`: -Use Clang (default) / GCC. It requires clang 3.5+ to be installed on the host. -Clang is the default compiler on Mac (% having installed Xcode). -On Linux: `sudo apt-get update && sudo apt-get install clang` +Use Clang (default: true) or GCC (false). +On Linux, by default it uses the self-hosted clang (see `is_hermetic_clang`). +On Android, by default it uses clang from the NDK (in `buildtools/ndk`). +On Mac, by default it uses the system version of clang (requires Xcode). `is_hermetic_clang = true | false`: Use bundled toolchain from `buildtools/` rather than system-wide one.
diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..d4da7bb --- /dev/null +++ b/docs/contributing.md
@@ -0,0 +1,23 @@ +# Contributing to Pefetto +This project uses [Android AOSP Gerrit][perfetto-gerrit] for code reviews and +uses the [Google C++ style][google-cpp-style] and targets `-std=c++11`. + +`git cl upload` from [Chromium depot tools][depot-tools] is the preferred +workflow to upload patches, as it supports presubmits and code formatting via +`git cl format`. + +See https://source.android.com/source/contributing for more details about +external contributions and CLA signing. + + +### Continuous integration + +Continuous build and test coverage is available at +[perfetto-ci.appspot.com](https://perfetto-ci.appspot.com). + +**Trybots**: CLs uploaded to gerrit are automatically submitted to TravisCI +within one minute and available on the CI page. + +[perfetto-gerrit]: https://android-review.googlesource.com/q/project:platform%252Fexternal%252Fperfetto+status:open +[google-cpp-style]: https://google.github.io/styleguide/cppguide.html +[depot-tools]: https://dev.chromium.org/developers/how-tos/depottools
diff --git a/docs/embedder-guide.md b/docs/embedder-guide.md new file mode 100644 index 0000000..3b6f3d8 --- /dev/null +++ b/docs/embedder-guide.md
@@ -0,0 +1,29 @@ +# Embedding Perfetto in your own project + +*** note +**This doc is WIP**, stay tuned +<!-- TODO(primiano): write embedder guide doc. --> +*** + + +This doc should: +- Contain tech details of the Producer(Endpoint), Consumer(Endpoint) and Service + interfaces. +- Explain how they are supposed to be wired up together, with or without + using an IPC transport. +- Explain the basic embedder requirements (e.g. [`TaskRunner`](/include/perfetto/base/task_runner.h)) +- Point out the relevant GN targets: + `//src/tracing`, `//src/tracing:ipc`, `//src/ipc`. +- Explain the API surface: + - [producer.h](/include/perfetto/tracing/core/producer.h) + - [consumer.h](/include/perfetto/tracing/core/consumer.h) + - [service.h](/include/perfetto/tracing/core/service.h) +- Explain the ABI surface: + - [shared_memory_abi.h](/include/perfetto/tracing/core/shared_memory_abi.h) + - IPC's [wire protocol](/src/ipc/wire_protocol.proto) (if used) + - The input [config protos](/protos/perfetto/config) + - The output [trace protos](/protos/perfetto/trace) + +Other resources +--------------- +* How we wrap our own IPC transport in Android: [/src/tracing/ipc](/src/tracing/ipc).
diff --git a/docs/ftrace.md b/docs/ftrace.md new file mode 100644 index 0000000..6a95d9a --- /dev/null +++ b/docs/ftrace.md
@@ -0,0 +1,52 @@ +# Perfetto <-> Ftrace interoperability + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): write ftrace doc. --> +*** + +This doc should: +- Describe the ftrace trace_pipe_raw -> protobuf translation. +- Describe how we deal with kernel ABI (in)stability and ftrace fields changing + over kernel versions (we process `event/**/format files on-device`). +- Describe how to generate ftrace protos (`tools/pull_ftrace_format_files.py`, + `tools/udate_protos.py`) +- Describe how session multiplexing works. +- Describe the page-by-page scheduling algorithm that uses vmsplice() + +Code lives in [/src/ftrace_reader](/src/ftrace_reader/). + +From https://android-review.googlesource.com/c/platform/external/perfetto/+/603793/ +``` + + main thread [drain] [unblock] + /: | + post .-----' : | + / : v + worker #0 [splice ...] [wakeup] [block ............] [splice] + : + worker #1 [splice ...] [wakeup] [block ........] [splice] + : + worker #2 [splice ..........................................] + : + : + drain period (100ms) + +In other words, the splice(2) system call is used to move data from +the raw kernel ftrace pipe into an intermediate pipe at a page +granularity. This call allows every per-cpu worker to sleep until there +is at least one page of data available. + +When a worker wakes up, it will attempt to move as many pages as +possible to its staging pipe (up to 64K, depending on the +system's pipe buffer size) in a non-blocking way. After this, it +will notify the main thread that data is available. This notification +will block the calling worker until the main thread has drained the +data. + +When at least one worker has woken up, we schedule a drain operation +on the main thread for the next drain period (every 100ms by default). +The drain operation parses ftrace data from the staging pipes of +every worker having pending data. After this, each waiting worker is +allowed to issue another call to splice(), restarting the cycle. +```
diff --git a/docs/ipc.md b/docs/ipc.md new file mode 100644 index 0000000..33d3a5c --- /dev/null +++ b/docs/ipc.md
@@ -0,0 +1,45 @@ +# Perfetto IPC + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): write IPC doc. --> +*** + +**TL;DR** +We needed an IPC for Android and Linux which was small, simple, controllable, +predictable, C++11 friendly and debuggable. +Our IPC transport is not mandatory outside of Android, you can wrap your own IPC +transport (e.g., Perfetto uses Mojo in chromium) or just short circuit the +Perfetto `{Service,Producer,Consumer}` interfaces for IPC-less full in-process +use. + +Key features: +- Protobuf over a unix-socket. +- Allows to send file descriptors over the wire: for setting up shared memory + and passing the FD for the output trace from a consumer to the service. +- Service definition uses same protobuf rpc syntax of [gRPC](https://grpc.io) +- Extremely simple [wire protocol](/src/ipc/wire_protocol.proto). +- C++11 friendly, allows to bind `std::function` to each request. +- Leak (un)friendly: tries hard to guarantee that callbacks are left unresolved, + using C++11 move semantics. +- Shutdown / destruction friendly: tries hard to guarantee that no callbacks are + issued after the IPC channel is destroyed. +- Disconnection-friendly: all outstanding requests (and hence pending callbacks) + are nack-ed in case of a disconnection (e.g., if the endpoint crashes). +- Memory friendly: one virtually contiguous cache-friendly rx buffer, + madvise()'d when when not used. +- Debugging friendly: single-thread only, based on non-blocking socket I/O. +- Binary size friendly: generates one protobuf per message, doesn't have any + external dependency. +- Hopefully safe: + - The rx buffer has guard regions around. + - The wire protocol is based on protobuf. + - [Fuzzed](/src/ipc/buffered_frame_deserializer_fuzzer.cc) +- Offers direct control of socket buffers and overrun/stalling policy. +- ABI stable. + +Realistically will never support: + - Multithreading / thread pools. + - Congestion or flow control. + - Non-data object brokering (e.g. sending a remote interface). + - Introspection / reflection.
diff --git a/docs/life-of-a-tracing-session.md b/docs/life-of-a-tracing-session.md new file mode 100644 index 0000000..3ed34df --- /dev/null +++ b/docs/life-of-a-tracing-session.md
@@ -0,0 +1,83 @@ +# Life of a Perfetto tracing session + +This document explains how producer, service and consumer interact end-to-end +during a tracing session, with references to code and IPC requests / responses. + +1. One or more producers connect to the tracing service and sets up their IPC + channels. +2. Each producer advertises one or more data sources through the + [`RegisterDataSource`](/protos/perfetto/ipc/producer_port.proto#34) IPC. + Nothing more happens on the Producer until this point. Nothing traces by + default. +3. A consumer connects to the tracing service and sets up the IPC channel. +4. The consumer starts a tracing session sending a + [trace config](trace-config.md) to the service through the + [`EnableTracing`](/protos/perfetto/ipc/consumer_port.proto#65) IPC. +6. The service creates as many new trace buffers as specified in the config. +7. The service iterates through the + [`data_sources`](/protos/perfetto/config/trace_config.proto#50) section of + the trace config: for each entry, if a matching data source is found in the + producer(s) (accordingly to what advertised in step 2): +8. The service sends a + [`SetupTracing`](/protos/perfetto/ipc/producer_port.proto#112) IPC message, + passing a shared memory buffer to the producer(s) (only once for each + producer). +9. The service sends a + [`StartDataSource`](/protos/perfetto/ipc/producer_port.proto#105) IPC message + to each producer, for each data source configured in the trace config and + present in the producer (if any). +10. The producer creates one or more data source instance, as instructed in + the previous step. +11. Each data source instance creates one or more + [`TraceWriter`](/include/perfetto/tracing/core/trace_writer.h) (typically + one for each thread). +12. Each `TraceWriter` writes one or more + [`TracePacket`](/protos/perfetto/trace/trace_packet.proto). +13. While doing so, each `TraceWriter` takes ownership of shared memory buffer's + chunks, using the [`SharedMemoryArbiter`](/include/perfetto/tracing/core/shared_memory_arbiter.h). +14. While writing a `TracePacket`, the `TraceWriter` will unavoidably cross the + chunk boundary (typically 4KB, but can configured to be smaller). +15. When this happens the `TraceWriter` will release the current chunk and + acquire a new chunk through the `SharedMemoryArbiter`. +16. The `SharedMemoryArbiter` will, out-of-band, send a + [`CommitDataRequest`](/protos/perfetto/ipc/producer_port.proto#41) to the + service, requesting it to move some chunks of the shared memory buffer into + the final trace buffer. +17. If one or more long `TracePacket` were fragmented over several chunks, it is + possible that some of these chunks are gone from the the shared memory + buffer and committed into the final trace buffer (step 16). In this case, + the `SharedMemoryArbiter` will send an other `CommitDataRequest` IPC message + to request the out-of-band patching of the chunk data into the final trace + buffer. +18. The service will check if the given chunk, identified by the tuple + `{ProducerID (unspoofable), WriterID, ChunkID}` is still present in the + trace buffer and if so will proceed to patch it (% sanity checks). +19. The consumer sends a [`FlushRequest`](/perfetto/ipc/consumer_port.proto#52) + to the service, asking it commit all data on flight in the trace buffers. +20. The service, in turn, issues a + [`Flush`](/protos/perfetto/ipc/producer_port.proto#132) request to all + producers involved in the trace session. +21. The producer(s) will `Flush()` all their `TraceWriter` and reply to the + service flush request. +22. Once the service has received an ACK to all flush requests from all + producers (or the + [flush timeout](/protos/perfetto/ipc/consumer_port.proto#117) has expired) + it replies to the consumer's `FlushRequest` +23. The consumer optionally sends a + [`DisableTracing`](/protos/perfetto/ipc/consumer_port.proto#38) IPC request + to stop tracing and freeze the content of the trace buffers. +24. The service will in turn broadcast a + [`StopDataSource`](/protos/perfetto/ipc/producer_port.proto#110) request for + each data source in each Producer. +23. At this point the Consumer issues a + [`ReadBuffers`](/protos/perfetto/ipc/consumer_port.proto#41) IPC request + (unless it did previously pass a file descriptor to the service when + enabling the trace through the `write_into_file` field of the trace config). +24. The service reads the trace buffers and streams all the `TracePacket(s)` + back to the consumer. +25. If a trace packet stored in the trace buffer is incomplete (e.g., a fragment + is missing) or is marked as pending out-of-band patching, the given writer + sequence is interrupted and no more packets for that sequence are read. + Other packets for other `TraceWriter` sequences will be unaffected. +26. The consumer sends a `FreeBuffers` (or simply disconnects). +27. The service tears down all the trace buffers for the session.
diff --git a/docs/multi-layer-tracing.md b/docs/multi-layer-tracing.md new file mode 100644 index 0000000..48a51af --- /dev/null +++ b/docs/multi-layer-tracing.md
@@ -0,0 +1,22 @@ +# Multi layer tracing + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): write multi-layer tracing doc. --> +*** + +This doc should explain how is possible to compose a hierarchy of tracing +services. The concrete use case is combining multiprocess tracing in Chromium +with Android's tracing daemons (think to hypervisors' nested page tables). + +The TL;DR of the trick is: +- ABI stability of the + [shared_memory_abi.h](/include/perfetto/tracing/core/shared_memory_abi.h) +- ABI stability of the IPC surface. + +The tracing service in chromium should proxy Producer connections (adapting Mojo +to our IPC) towards the Android's `traced` service, passing back the shared +memory buffers to the real producers (the Chrome child process). + +Conceptually it is simple and straightforward, requires *some* care to implement +correctly ownership of the shared memory buffers though.
diff --git a/docs/protozero.md b/docs/protozero.md new file mode 100644 index 0000000..95b92a4 --- /dev/null +++ b/docs/protozero.md
@@ -0,0 +1,28 @@ +ProtoZero +--------- + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): write protozero doc. --> +*** + +ProtoZero is an almost* zero-copy zero-malloc append-only protobuf library. +It's designed to be fast and efficient at the cost of a reduced API +surface for generated stubs. The main limitations consist of: +- Append-only interface: no readbacks are possible from the stubs. +- No runtime checks for duplicated or missing mandatory fields. +- Mandatory ordering when writing of nested messages: once a nested message is + started it must be completed before adding any fields to its parent. + +*** aside +Allocations and library calls will happen only when crossing the boundary of a +contiguous buffer (e.g., to request a new buffer to continue the write). +Assuming a chunk size (a trace *chunk* is what becomes a *contiguous buffer* +within ProtoZero) of 4KB, and an average event size of 32 bytes, only 7 out of +1000 events will hit malloc / ipc / library calls. +*** + + +Other resources +--------------- +* [Design doc](https://goo.gl/EKvEfa])
diff --git a/docs/running.md b/docs/running.md new file mode 100644 index 0000000..df98a8a --- /dev/null +++ b/docs/running.md
@@ -0,0 +1,77 @@ +# Running Perfetto + +In order to run Perfetto and get a meaningful trace you need to build +(see [build instructions](build-instructions.md)) and run the following: + +`traced`: +The unprivileged trace daemon that owns the log buffers and maintains +a registry of Producers and Consumers connected. + +`traced_probes`: +The privileged daemon that has access to the Kernel tracefs +(typically mounted under `/sys/kernel/debug/tracing`). It drives +[Ftrace](https://source.android.com/devices/tech/debug/ftrace) and writes its +protobuf-translated contents into `traced`. + +`perfetto`: +A command line utility client that drive the trace and save back +the results (either to a file or to [Android's Dropbox][dropbox]) + +Running from a standalone checkout (Linux, Mac or Android) +------------------------------------------------------------- +A convenience script allows to run Perfetto daemons (`traced`, `traced_probes`) +and the command line client (`perfetto`) in a tmux-based terminal: +``` +CONFIG=ftrace.cfg OUT=out/default ./tools/tmux +``` + +The script will automatically serialize the trace config defined in the +`CONFIG` variable (e.g., [this](https://android.googlesource.com/platform/external/perfetto/+/master/test/configs/ftrace.cfg)) into a protobuf and setup the right paths. +Furthermore it will automatically rebuild if necessary. + +Running from an Android P+ in-tree build +---------------------------------------- +Make sure that Perfetto daemons (`traced` / `traced_probes`) are running. +They are enabled by default on Pixel and Pixel 2 (walleye, taimen, marlin, +sailfish). On other devices start them manually by doing: +``` +adb shell setprop persist.traced.enable 1 +``` + +If this works you will see something like this in the logs: +``` +$ adb logcat -s perfetto +perfetto: service.cc:45 Started traced, listening on /dev/socket/traced_producer /dev/socket/traced_consumer +perfetto: probes.cc:25 Starting /system/bin/traced_probes service +perfetto: probes_producer.cc:32 Connected to the service +``` + +At which point you can grab a trace by doing: + +``` +$ adb shell perfetto --config :test --out /data/misc/perfetto-traces/trace +``` + +For more advanced configurations see the [Trace Config](#trace-config) section. + +*** aside +If the output file is not under `/data/misc/perfetto-traces`, tracing will +fail due to SELinux. +*** + +*** aside +For security reasons the trace file is written with 0600 (rw-------) permissions +and owned by shell. On rooted (`userbuild`) devices it is possible to just +`adb pull` the file after `adb root`. On `user` devices instead, in order to get +the trace out of the device, do the following: +`adb shell cat /data/misc/perfetto-traces/trace > ~/trace` +*** + +Trace config +------------ +`--config :test` uses a hard-coded test trace config. It is possible to pass +an arbitrary trace config. See instructions in the +[trace config](trace-config.md) page. + + +[dropbox]: https://developer.android.com/reference/android/os/DropBoxManager.html
diff --git a/docs/running_perfetto.md b/docs/running_perfetto.md deleted file mode 100644 index 0763505..0000000 --- a/docs/running_perfetto.md +++ /dev/null
@@ -1,98 +0,0 @@ -# Running Perfetto - -In order to run Perfetto and get a meaningful trace you need to build -(see [build instructions](build_instructions.md)) and run the following: - -`traced`: The unprivileged trace daemon that owns the log buffers and maintains -a registry of Producers and Consumers connected. - -`traced_probes`: The privileged daemon that has access to the Kernel tracefs -(typically mounted under `/sys/kernel/debug/tracing`), can drive -[Ftrace](https://source.android.com/devices/tech/debug/ftrace) and writes its -output into `traced`. - -`perfetto`: A command line utility client that drive the trace and save back -the results (either to a file or to [Android's Dropbox][dropbox]) - - -## Instructions: -Make sure that Perfetto daemons (`traced` / `traced_probes`) are started. -They are enabled by default on Pixel and Pixel 2 (walleye, taimen, marlin, -sailfish). On other devices start them manually by doing: -``` -adb shell setprop persist.traced.enable 1 -``` - -If this works you will see something like: - -``` -$ adb logcat -s perfetto -perfetto: service.cc:45 Started traced, listening on /dev/socket/traced_producer /dev/socket/traced_consumer -perfetto: probes.cc:25 Starting /system/bin/traced_probes service -perfetto: probes_producer.cc:32 Connected to the service -``` - -At which point you can grab a trace by doing: - -``` -$ adb shell perfetto --config :test --out /data/misc/perfetto-traces/trace -``` - -Note: If the output file is not under `/data/misc/perfetto-traces`, it might -fail due to SELinux (shell is allowed to read that directory). - -Alternatively, it can be saved to [Android's Dropbox][dropbox] -(to test the uploader): - -``` -$ adb shell perfetto --config :test --dropbox perfetto -``` - -### Trace config -`--config :test` uses a hard-coded test trace config. It is possible to pass -an arbitrary trace config by doing the following: -``` -cat > /tmp/config.txpb <<EOF -# This is a text-encoded protobuf for /protos/perfetto/config/trace_config.proto -duration_ms: 10000 - -buffers { - size_kb: 10240 -} - -data_sources { - config { - name: "linux.ftrace" - target_buffer: 0 - ftrace_config { - buffer_size_kb: 40 # Kernel ftrace buffer size. - ftrace_events: "sched_switch" - ftrace_events: "print" - } - } -} - -data_sources { - config { - name: "linux.process_stats" - target_buffer: 0 - } -} -EOF - -protoc=$(pwd)/out/android/gcc_like_host/protoc - -$protoc --encode=perfetto.protos.TraceConfig \ - -I$(pwd)/external/perfetto/protos \ - $(pwd)/external/perfetto/protos/perfetto/config/perfetto_config.proto \ - < /tmp/config.txpb \ - > /tmp/config.pb - -cat /tmp/config.pb | adb shell perfetto -c - -o /data/misc/perfetto-traces/trace.pb -adb pull /data/misc/perfetto-traces/trace.pb /tmp/ -out/android/trace_to_text systrace < /tmp/trace.pb > /tmp/trace.json - -# The file can now be viewed in chrome://tracing -``` - -[dropbox]: https://developer.android.com/reference/android/os/DropBoxManager.html
diff --git a/docs/running_tests.md b/docs/running_tests.md deleted file mode 100644 index a37b83c..0000000 --- a/docs/running_tests.md +++ /dev/null
@@ -1,48 +0,0 @@ -# Running Perfetto tests - -Perfetto has two main test targets: `perfetto_unittests` and -`perfetto_integrationtests`. These can either be built using standalone or in -the Android tree; we provide instructions for running both below. CTS tests can -also be run when building in the Android tree; they cannot be built using -standalone. - -First, ennsure that you can build Perfetto (see [build -instructions](build_instructions.md)). On Android, also setup the service and -the kernel event producer (see [running Perfetto](running_perfetto.md)) - -## Standalone build -On Linux and Mac, out/$target/perfetto_unittests and -out/$target/perfetto_integrationtests can be run directly. - -On Android, the following commands should be run (with the appropriate target in -curly braces depending on which tests suites are to be run): -``` -adb push out/$target/perfetto_{unittests, integrationtests} /data/local/tmp/ -adb shell out/$target/perfetto_{unittests, integrationtests}. -``` - -# Inside the Android tree -Unit tests, once built, can be run using the following commands: -``` -adb push $OUT/data/nativetest/perfetto_unittests/perfetto_unittests /data/local/tmp/ -adb shell /data/local/tmp/perfetto_unittests -``` - -Integration tests, once built, can be run using the following commands: -``` -adb push $OUT/data/nativetest/perfetto_integrationtests/perfetto_integrationtests /data/local/tmp/ -adb shell /data/local/tmp/perfetto_integrationtests -``` - -Building in the Android tree also allows building of CTS tests. The relevant -targets are `CtsPerfettoProducerApp` and `CtsPerfettoTestCases`. Once these are -built, the following commands should be run (to manually run these tests): -``` -adb push $ANDROID_HOST_OUT/cts/android-cts/testcases/CtsPerfettoTestCases64 /data/local/tmp/ -adb install -r $ANDROID_HOST_OUT/cts/android-cts/testcases/CtsPerfettoProducerApp.apk -``` -Next, the app with name 'android.perfetto.producer' should be run on device. -Finally, the following command should be run: -``` -adb shell /data/local/tmp/CtsPerfettoTestCases64 -```
diff --git a/docs/security-model.md b/docs/security-model.md new file mode 100644 index 0000000..ee772ac --- /dev/null +++ b/docs/security-model.md
@@ -0,0 +1,71 @@ +# Perfetto security model + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): expand security model doc. --> +*** + + + +**TL;DR** +The tracing service has two endpoints (in Chromium: Mojo services, on Android: +UNIX sockets): one for producer(s) and one for consumer(s). +The former is typically public, the latter is restricted only to trusted +consumers. + +**Producers** +Producers are never trusted. We assume they will try their best to DoS / crash / +exploit the tracing service. We do so at the +[core/service_impl.cc](/src/tracing/core/service_impl.cc) so that the the same +level of security and testing is applied regardless of the embedder and the IPC +transport. + +**Tracing service** +- The tracing service has to validate all inputs. +- In the worst case a bug in the tracing service allowing remote code execution, + the tracing service should have no meaningful capabilities to exploit. +- The tracing service, by design, has a limited syscall surface to simplify + its sandboxing: + - It doesn't open or create files (% tmpfs). + - It writes only onto file descriptors passed over the IPC channel. + - It doesn't open or create sockets (on Android the IPC sockets are passed by + init, see [perfetto.rc](/perfetto.rc)) + - On Android it runs as nobody:nobody and is allowed to do very little + see [traced.te](https://android.googlesource.com/platform/system/sepolicy/+/master/private/traced.te). + - In Chromium it should run as a utility process. + - TODO: we could use BPF syscall sandboxing both in Chromium and Android. + [Proof of concept](https://android-review.googlesource.com/c/platform/external/perfetto/+/576563) + +**Consumers** +Consumers are always trusted. They still shouldn't be able to crash or exploit +the service. They can easily DoS it though, but that is WAI. + - In Chromium the trust path is established through service manifest. + - In Android the trust path is established locking down the consumer socket + to shell through SELinux. + +**Shared memory isolation** +Memory is shared only point-to-point between each producer and the tracing +service. We should never ever share memory across producers (in order to not +leak trace data belonging to different producers) nor between producers and +consumers (that would open a hard to audit path between +untrusted-and-unprivileged and trusted-and-more-privileged entities). + +**Attestation of trace contents** +The tracing service guarantees that the `TracePacket` fields defined also in +[trusted_packet.proto](/protos/perfetto/trace/trusted_packet.proto) cannot be +spoofed by the Producer(s). Packets that try to define those fields are rejected +(modulo the clock snapshots). +See [PacketStreamValidator](/src/tracing/core/packet_stream_validator.cc) and +[its unit test](/src/tracing/core/packet_stream_validator_unittest.cc) for more +details. +At the moment nothing prevents that a producer writes `TracePacket(s)` that do +not belong to its data sources. Realistically the service will never prevent +that because doing so would imply that the service knows about all the possible +types of packets, which doesn't scale. +However, the service appends the POSIX uid of the producer to each `TracePacket` +to perform offline attestation of the contents of the trace. + +Internal docs +------------- +* [Android security review doc](go/perfetto-asec) +* [Chromium security review doc](go/perfetto-csec)
diff --git a/docs/testing.md b/docs/testing.md index 5a2015f..ccb05fb 100644 --- a/docs/testing.md +++ b/docs/testing.md
@@ -1,42 +1,96 @@ # Testing Perfetto -The testing strategy for Perfetto is rather complex considering the wide variety -of build configurations and places it is expected to be integrated. We discuss -all testing related information in this doc. +The testing strategy for Perfetto is rather complex due to the wide variety +of build configurations and embedding targets. -# Test runners -Perfetto can be tested in a variety of locations: -* Travis: CI for Perfetto only. Found at https://perfetto-ci.appspot.com/ -* APCT: see go/apct and go/apct-guide -* Treehugger: Android's presubmit system. Run before submission of every CL. -* CTS: Android test suite used run to ensure API compatibility. Rolling runs -internally. +Common test targets (all platforms / checkouts): + +`perfetto_unittests`: +Platform-agnostic unit-tests. + +`perfetto_integrationtests`: +End-to-end tests, involving the protobuf-based IPC transport and ftrace +integration (Linux/Android only). + +`perfetto_benchmarks`: +Benchmarks tracking the performance of: (i) trace writing, (ii) trace readback +and (iii) ftrace raw pipe -> protobuf translation. + +Running tests on Linux / MacOS +------------------------------ +``` +$ tools/ninja -C out/default perfetto_{unittests,integrationtests,benchmarks} +$ out/default/perfetto_unittests --gtest_help +``` + +`perfetto_integrationtests` requires that the ftrace debugfs directory is +is readable/writable by the current user on Linux: +``` +sudo chown -R $USER /sys/kernel/debug/tracing +``` + +Running tests on Android +------------------------ +1A) Connect a device through `adb` +1B) Start the build-in emulator (supported on Linux and MacOS): +``` +$ tools/install-build-deps +$ tools/run_android_emulator & +``` + +2) Run the tests (either on the emulator or physical device): +``` +$ tools/run_android_test out/default perfetto_unittests +``` + + +Continuous testing +------------------ +Perfetto is tested in a variety of locations: + +**Travis CI**: https://perfetto-ci.appspot.com/ +Builds and runs perfetto_{unittests,integrationtests,benchmarks} from then standalone checkout. Benchmarks are ran in a reduced form for smoke testing. + +**Android CI** (see go/apct and go/apct-guide): +runs only `perfetto_integrationtests` + +**Android presubmits (TreeHugger)**: +Runs before submission of every AOSP CL of `external/perfetto`. + + +**Android CTS** (Android test suite used run to ensure API compatibility): Rolling runs internally. Note that Travis uses the standalone build system and the others build as part of the Android tree. -# Unit tests +Unit tests +---------- Unit tests exist for most of the code in Perfetto on the class level. They ensure that each class broadly works as expected. -Unit tests are run on Travis, APCT (pending) and Treehugger (pending). +Unit tests are currently ran only on Travis. +Running unit tests on APCT and Treehugger is WIP. -# Integration tests -Integration tests ensure that subsystems (importantly ftrace) and Perfetto -as a whole (end to end testing) is working correctly. Note that we do not -test integration with any third parties, just within Perfetto. +Integration tests +----------------- +Integration tests ensure that subsystems (importantly ftrace and the IPC layer) +and Perfetto as a whole is working correctly end-to-end. There are two configurations in which integration tests can be run: -1. Assuming that the daemons (the service and ftrace) already running and - checking the test is able to interact correctly with them. This is usually - the way in which we test things on Android. -2. Starting up the daemons in the test itself and then testing against them. - This is usually how standalone builds are tested. -Integration tests are run on Travis (excluding ftrace) in mode 2, APCT (pending) -in mode 1, Treehugger (pending) in mode 1 and CTS in mode 1. +**1. Production mode** (Android-only) +This mode assumes that both the tracing service (`traced`) and the OS probes +service (`traced_probes`) are already running. In this mode the test enables +only the consumer endpoint and tests the interaction with the production +services. This is the way our Android CTS and APCT tests work. -# CTS tests +**2. Standalone mode**: +Starting up the daemons in the test itself and then testing against them. +This is how standalone builds are tested. This is the only supported way to +run integration tests on Linux and MacOS. + +Android CTS tests +----------------- CTS tests ensure that any vendors who modify Android remain compliant with the platform API. @@ -44,5 +98,19 @@ more complex tests which ensure interaction between platform (e.g. Android apps etc.) and Perfetto is not broken. -CTS tests currently are not run anywhere as these tests are not included in -the suite. +The relevant targets are `CtsPerfettoProducerApp` and `CtsPerfettoTestCases`. Once these are built, the following commands should be run: +``` +adb push $ANDROID_HOST_OUT/cts/android-cts/testcases/CtsPerfettoTestCases64 /data/local/tmp/ +adb install -r $ANDROID_HOST_OUT/cts/android-cts/testcases/CtsPerfettoProducerApp.apk +``` + +Next, the app named `android.perfetto.producer` should be run on the device. + +Finally, the following command should be run: +``` +adb shell /data/local/tmp/CtsPerfettoTestCases64 +``` + +Chromium waterfall +------------------ +Coming soon!
diff --git a/docs/trace-config.md b/docs/trace-config.md new file mode 100644 index 0000000..0c5f7ba --- /dev/null +++ b/docs/trace-config.md
@@ -0,0 +1,82 @@ +# Perfetto trace config + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): write trace config doc. --> +*** + + + +The [`TraceConfig`](/protos/perfetto/config/trace_config.proto) is an extensible +protobuf message, sent by the consumer to the service, that defines: +- The number and size of the trace buffer. +- The duration of the trace. +- [optionally] a file descriptor for the output trace and a periodic write + interval. If omitted the trace is kept only in memory. +- The producers involved in the trace session. +- The data sources involved in the trace session. +- The configuration of each data source. +- The crossbar mapping between each data source and the trace buffers. + +Each data source can create its own specialized schema for the config, like +[this](/protos/perfetto/config/ftrace/ftrace_config.proto) + +See [`trace_config.proto`](/protos/perfetto/config/trace_config.proto) for more +details. + +For convenience, a vulcanized trace config where all the nested protobuf +sub-message definitions are squashed together is available in +[`perfetto_config.proto`](/protos/perfetto/config/perfetto_config.proto). + + +Specifying a custom trace config +-------------------------------- +``` +cat > /tmp/config.txpb <<EOF +# This is a text-encoded protobuf for /protos/perfetto/config/trace_config.proto +duration_ms: 10000 + +# For long traces set the following variables. It will periodically drain the +# trace buffers into the output file, allowing to save a trace larger than the +# buffer size. +write_into_file: true +file_write_period_ms: 5000 + +buffers { + size_kb: 10240 +} + +data_sources { + config { + name: "linux.ftrace" + target_buffer: 0 + ftrace_config { + buffer_size_kb: 40 # Kernel ftrace buffer size. + ftrace_events: "sched_switch" + ftrace_events: "print" + } + } +} + +data_sources { + config { + name: "linux.process_stats" + target_buffer: 0 + } +} +EOF + +protoc=$(pwd)/out/android/gcc_like_host/protoc + +$protoc --encode=perfetto.protos.TraceConfig \ + -I$(pwd)/external/perfetto/protos \ + $(pwd)/external/perfetto/protos/perfetto/config/perfetto_config.proto \ + < /tmp/config.txpb \ + > /tmp/config.pb + +cat /tmp/config.pb | adb shell perfetto -c - -o /data/misc/perfetto-traces/trace.pb +adb shell cat /data/misc/perfetto-traces/trace.pb > /tmp/trace.pb +out/android/trace_to_text json < /tmp/trace.pb > /tmp/trace.json + +# The file can now be viewed in chrome://tracing +```
diff --git a/docs/trace-format.md b/docs/trace-format.md new file mode 100644 index 0000000..ca5db49 --- /dev/null +++ b/docs/trace-format.md
@@ -0,0 +1,33 @@ +# Perfetto trace format + +*** note +**This doc is WIP**, stay tuned. +<!-- TODO(primiano): write trace format doc. --> +*** + +A Perfetto trace is guaranteed to be a a linear sequence of `TracePacket(s)` +(see [trace_packet.proto](/protos/perfetto/trace/trace_packet.proto)). + +As a key part of the Perfetto design, the tracing service is agnostic of the +content of TracePacket, modulo the few fields defined in +[trusted_packet.proto](/protos/perfetto/trace/trusted_packet.proto) that are +produced by the service itself. + +Each data source can extend the trace with their app-specific protobuf schema. +*** aside +TODO(primiano): we should reserve an extension range and figure out / comment a +hash to assign sub-message IDs, even without checking them into +trace_packet.proto. +*** + + +**Linearity guarantees** +The tracing service guarantees that all `TracePacket(s)` written by a given +`TraceWriter` are seen in-order, without gaps or duplicates. If, for any reason, +a `TraceWriter` sequence becomes invalid, no more packets are returned to the +Consumer (or written into the trace file). + +However, `TracePacket(s)` written by different `TraceWriter` (hence even +different producers) can be seen in no particular order. +The consumer can re-establish a total order, if interested, using the packet +timestamps (after having synchronized the different clocks onto a global clock).