Primiano Tucci | b808373 | 2018-04-09 15:38:37 +0100 | [diff] [blame] | 1 | # Perfetto key concepts and architecture |
| 2 | |
| 3 | Producer <> Service <> Consumer model |
| 4 | ------------------------------------- |
| 5 |  |
| 6 | |
| 7 | **Service** |
| 8 | The tracing service is a long-lived entity (a system daemon on Linux/Android, |
| 9 | a service in Chrome) that has the following responsibilities: |
| 10 | - Maintains a registry of active producers and their data sources. |
| 11 | - Owns the trace buffers. |
| 12 | - Handles multiplexing of several tracing sessions. |
| 13 | - Routes the trace config from the consumers to the corresponding producers. |
| 14 | - Tells the Producers when and what to trace. |
| 15 | - Moves data from the Producer's shared memory buffer to the central non-shared |
| 16 | trace buffers. |
| 17 | |
| 18 | **Producer** |
| 19 | A producer is an untrusted entity that offers the ability to contribute to the |
| 20 | trace. In a multiprocess model, a producer almost always corresponds to a client |
| 21 | process of the tracing service. It advertises its ability to contribute to the trace with one or more data sources. |
| 22 | Each producer has exactly: |
| 23 | - One shared memory buffer, shared exclusively with the tracing service. |
| 24 | - One IPC channel with the tracing service. |
| 25 | |
| 26 | A producer is completely decoupled (both technically and conceptually) from |
| 27 | consumer(s). A producer knows nothing about: |
| 28 | - How many consumer(s) are connected to the service. |
| 29 | - How many tracing sessions are active. |
| 30 | - How many other producer(s) are registered or active. |
| 31 | - Trace data written by other producer(s). |
| 32 | |
| 33 | *** aside |
| 34 | In rare circumstances a process can host more than one producer and hence more |
| 35 | than one shared memory buffer. This can be the case for a process bundling |
| 36 | third-party libraries that in turn include the Perfetto client library. |
| 37 | Concrete example: at some point in the future Chrome might expose one Producer for tracing within the main project, one for V8 and one for Skia (for each child |
| 38 | process). |
| 39 | *** |
| 40 | |
| 41 | **Consumer** |
| 42 | A consumer is a trusted entity (a cmdline client on Linux/Android, an interface |
| 43 | of the Browser process in Chrome) that controls (non-exclusively) the tracing service and reads back (destructively) the trace buffers. |
| 44 | A consumer has the ability to: |
| 45 | - Send a [trace config](trace-config.md) to the service, determining: |
| 46 | - How many trace buffers to create. |
| 47 | - How big the trace buffers should be. |
| 48 | - The policy for each buffer (*ring-buffer* or *stop-when-full*). |
| 49 | - Which data sources to enable. |
| 50 | - The configuration for each data source. |
| 51 | - The target buffer for the data produced by each data source configured. |
| 52 | - Enable and disable tracing. |
| 53 | - Read back the trace buffers: |
| 54 | - Streaming data over the IPC channel. |
| 55 | - Passing a file descriptor to the service and instructing it to periodically |
| 56 | save the trace buffers into the file. |
| 57 | |
| 58 | **Data source** |
| 59 | A data source is a capability, exposed by a Producer, of providing some tracing |
| 60 | data. A data source almost always defines its own schema (a protobuf) consisting |
| 61 | of: |
| 62 | - At most one `DataSourceConfig` sub-message |
| 63 | ([example](/protos/perfetto/config/ftrace/ftrace_config.proto)) |
| 64 | - One or more `TracePacket` sub-messages |
| 65 | ([example](/protos/perfetto/trace/ps/process_tree.proto)) |
| 66 | |
| 67 | Different producers may expose the same data source. Concrete example: |
| 68 | *** aside |
| 69 | At some point in the near future we might offer, as part of Perfetto, a library |
| 70 | for in-process heap profiling. In such case more than one producer, linking |
| 71 | against the updated Perfetto library, will expose the heap profiler data source, |
| 72 | for its own process. |
| 73 | *** |
| 74 | |
| 75 | **IPC channel** |
| 76 | In a multiprocess scenario, each producer and each consumer interact with the |
| 77 | service using an IPC channel. IPC is used only in non-fast-path interactions, |
| 78 | mostly handshakes such as enabling/disabling trace (consumer), (un)registering |
| 79 | and starting/stopping data sources (producer). The IPC is typically NOT employed |
| 80 | to transport the protobufs for the trace. |
| 81 | Perfetto provides a POSIX-friendly IPC implementation, based on protobufs over a |
| 82 | UNIX socket (see [ipc.md](ipc.md)). That IPC implementation is not mandated. |
| 83 | Perfetto allows the embedder: |
| 84 | - Wrap its own IPC subsystem (e.g., Perfetto in Chromium uses Mojo) |
| 85 | - Not use an IPC mechanism at all and just short circuit the |
| 86 | Producer <> Service <> Consumer interaction via `PostTask(s)`. |
| 87 | |
| 88 | See [embedder-guide.md](embedder-guide.md) for more details. |
| 89 | |
| 90 | |
| 91 | **Shared memory buffer** |
| 92 | Producer(s) write tracing data, in the form of protobuf-encoded binary blobs, |
| 93 | directly into its shared memory buffer, using a special library called |
| 94 | [ProtoZero](protozero.md). The shared memory buffer: |
| 95 | - Has a fixed and typically small size (configurable, default: 128 KB). |
| 96 | - Is an ABI and must maintain backwards compatibility. |
| 97 | - Is shared by all data sources of the producer. |
| 98 | - Is independent of the number and the size of the trace buffers. |
| 99 | - Is independent of the number of Consumer(s). |
| 100 | - Is partitioned in *chunks* of variable size. |
| 101 | |
| 102 | Each chunk: |
| 103 | - Is owned exclusively by one Producer thread (or shared through a mutex). |
| 104 | - Contains a linear sequence of [`TracePacket(s)`](trace-format.md), or |
| 105 | fragments of that. A `TracePacket` can span across several chunks, the |
| 106 | fragmentation is not exposed to the consumers (consumers always see whole |
| 107 | packets as if they were never fragmented). |
| 108 | - Can be owned and written by exactly one `TraceWriter`. |
| 109 | - Is part of a reliable and ordered sequence, identified by the `WriterID`: |
| 110 | packets in a sequence are guaranteed to be read back in order, without gaps |
| 111 | and without repetitions (see [trace-format.md](trace-format.md) for more). |
| 112 | |
| 113 | See the comments in |
| 114 | [shared_memory_abi.h](/include/perfetto/tracing/core/shared_memory_abi.h) |
| 115 | for more details about the binary format of this buffer. |
| 116 | |
| 117 | Other resources |
| 118 | --------------- |
| 119 | * [Life of a tracing session](life-of-a-tracing-session.md) |
| 120 | * [Trace config](trace-config.md) |
| 121 | * [Trace format](trace-format.md) |