docs: Add docs page on concurrent tracing
Change-Id: I680c583766f74c2dafc34c72c09ac9e04d02b360
diff --git a/docs/concepts/concurrent-tracing-sessions.md b/docs/concepts/concurrent-tracing-sessions.md
new file mode 100644
index 0000000..a9ab94a
--- /dev/null
+++ b/docs/concepts/concurrent-tracing-sessions.md
@@ -0,0 +1,75 @@
+# Concurrent tracing sessions
+
+Perfetto supports multiple concurrent tracing sessions.
+Sessions are isolated from each other each and each session can choose a different mix of producers and data sources in its [config](config.md) and, in general, it will only receive events specified by that config.
+This is a powerful mechanism which allows great flexibility when collecting traces from the lab or field.
+However there are a few caveats to bear in mind with concurrent tracing sessions:
+1. [Some data sources do not support concurrent sessions](#some-data-sources-do-not-support-concurrent-sessions)
+2. [Some settings are per session while others are per producer](#some-settings-are-per-session-while-others-are-per-producer)
+3. Due to the [way atrace works works](#atrace) if session requests *any* atrace category or app it receives *all* atrace events enabled on the device
+4. [Various limits](#various-limits) apply
+
+## Some data sources do not support concurrent sessions
+
+While most data sources implemented with the Perfetto SDK as most of the data sources provided by the Perfetto team support concurrent tracing sessions others do not. This can be due to:
+
+- HW or driver constraints
+- Difficulty of implementing the config muxing
+- Perfetto SDK: users may [opt-out of multiple session support](https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/include/perfetto/tracing/data_source.h;l=266;drc=f988c792c18f93841b14ffa71019fdedf7ab2f03)
+
+### Known to work
+- `traced_probes` datasources
+
+### Known to work with caveats
+- `heapprofd` supports multiple sessions but each process can only be in one session.
+- `traced_perf` in general supports but the kernel has a limit on counters so may reject a config.
+
+### Known not to work
+- `traced metatracing`
+
+## Some settings are per session while others are per producer
+
+Most buffer sizes and timings specified in the config are per session.
+For example the buffer [sizes](https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/protos/perfetto/config/trace_config.proto;l=32?q=f:perfetto%20f:trace_config&ss=android%2Fplatform%2Fsuperproject%2Fmain).
+
+However some parameters configure per-producer settings. For example the [size and layout](https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/protos/perfetto/config/trace_config.proto;l=182;drc=488df1649781de42b72e981c5e79ad922508d1e5) of the shmem buffer between the producer and traced.
+While that is general data source setting the same can apply to data source specific settings.
+For example the ftrace [kernel buffer size and drain period](https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/protos/perfetto/config/ftrace/ftrace_config.proto;l=32;drc=6a3d3540e68f3d5949b5d86ca736bfd7f811deff) are settings that have to be shared between all users of `traced_probes`.
+
+Bear in mind that
+- Some resources like the shmem buffers are shared by all sessions
+- As suggested by the comments in linked code above some settings are best treated as 'hints' since another config may have already set them before you get a chance to.
+
+## Atrace
+
+Atrace is an Android specific mechanism for doing userland instrumentation and the only available tracing method prior to the introduction of the Perfetto SDK into Android.
+It still powers [os.Trace](https://developer.android.com/reference/android/os/Trace) (as used by platform and application Java code) and [ATRACE_*](https://cs.android.com/android/platform/superproject/main/+/main:system/core/libcutils/include/cutils/trace.h;l=188;drc=0c44d8d68d56c7aecb828d8d87fba7dcb114f3d9) (as used by platform C++).
+
+
+Atrace (both prior to Perfetto and via Perfetto) works as follows:
+- Configuration:
+ - Users choose zero or more 'categories' from a hardcoded list
+ - Users choose zero of package names including globs
+- This sets:
+ - Some kernel ftrace events
+ - A [system property bitmask](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/native/cmds/atrace/atrace.cpp;l=306;drc=c8af4d3407f3d6be46fafdfc044ace55944fb4b7) (for the atrace categories)
+ - A [system property](https://cs.android.com/android/platform/superproject/main/+/main:frameworks/native/cmds/atrace/atrace.cpp;l=306;bpv=1;bpt=1) for each package.
+- When the Java or C++ tracing APIs are called we examine the system props.
+- If the relevant category or package is enabled we write the event to `trace_marker`
+
+As mentioned each category may enable a number of kernel ftrace events.
+For example the 'sched' atrace category enables the `sched/sched_switch` ftrace event.
+Kernel ftrace events do not suffer from the current session issues so will not be described further.
+
+For the userland instrumentation:
+- Perfetto ensures the union of all atrace packages categories are installed
+- However since:
+ - the atrace system properties are global
+ - we cannot tell which event comes from which category/package
+- every session that requests *any* atrace event gets *all* enabled atrace events.
+
+## Various limits
+- Perfetto SDK: Max 8 datasource intances per datasource type per producer
+- `traced`: Limit of [15 concurrent sessions](https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/src/tracing/service/tracing_service_impl.cc;l=114?q=kMaxConcurrentTracingSessions%20)
+- `traced`: Limit of [5 (10 for statsd) concurrent sessions per UID](https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/src/tracing/service/tracing_service_impl.cc;l=115;drc=17d5806d458e214bdb829deeeb08b098c2b5254d)
+
diff --git a/docs/toc.md b/docs/toc.md
index 2ac2751..f828874 100644
--- a/docs/toc.md
+++ b/docs/toc.md
@@ -66,6 +66,7 @@
* [Service model](concepts/service-model.md)
* [Clock synchronization](concepts/clock-sync.md)
* [Detached mode](concepts/detached-mode.md)
+ * [Concurrent tracing sessions](concepts/concurrent-tracing-sessions.md)
* [Reference](#)
* [Trace Config proto](reference/trace-config-proto.autogen)
