docs: Explain Java heap sampler This updates the memory docs a little bit: * Renames most mentions of Java heap profiler with Java heap dump or graph, to avoid confusion with the java heap sampler. * Updates the memory flamegraph images: we've updated some of the labels since, and the images where out of date. * It renames the "Native heap profiler" page to simply "Heap profiler". * It documents the java heap sampler under the "Heap profiler" page. Bug: 218414058 Bug: 256052262 Change-Id: Id4a7fdd652bf03bfbe55bbd72931bec4c87609ac
diff --git a/docs/README.md b/docs/README.md index 1ce43f8..0dac083 100644 --- a/docs/README.md +++ b/docs/README.md
@@ -49,7 +49,7 @@ memory to call-stacks, based on out-of-process unwinding, configurable sampling, attachable to already running processes. -* [Java heap profiling](/docs/data-sources/java-heap-profiler.md): an +* Capturing [Java heap dumps](/docs/data-sources/java-heap-profiler.md) with an out-of-process profiler tightly integrated with the Android RunTime that allows to get full snapshots of the managed heap retention graph (types, field names, retained size and references to other objects) without, however,
diff --git a/docs/case-studies/memory.md b/docs/case-studies/memory.md index de4e401..b55bdce 100644 --- a/docs/case-studies/memory.md +++ b/docs/case-studies/memory.md
@@ -307,7 +307,7 @@ **Native Heap Profiles require Android 10.** NOTE: For detailed instructions about the native heap profiler and - troubleshooting see the [Data sources > Native heap profiler]( + troubleshooting see the [Data sources > Heap profiler]( /docs/data-sources/native-heap-profiler.md) page. Applications usually get memory through `malloc` or C++'s `new` rather than @@ -353,42 +353,40 @@ The tabs that are available are -* **Unreleased size**: how many bytes were allocated but not freed at this - callstack the moment the dump was created. -* **Total size**: how many bytes were allocated (including ones freed at the - moment of the dump) at this callstack -* **Unreleased count**: how many allocations without matching frees were done at - this callstack. -* **Total count**: how many allocations (including ones with matching frees) - were done at this callstack. +* **Unreleased malloc size**: how many bytes were allocated but not freed at + this callstack the moment the dump was created. +* **Total malloc size**: how many bytes were allocated (including ones freed at + the moment of the dump) at this callstack. +* **Unreleased malloc count**: how many allocations without matching frees were + done at this callstack. +* **Total malloc count**: how many allocations (including ones with matching + frees) were done at this callstack. The default view will show you all allocations that were done while the profile was running but that weren't freed (the **space** tab). - + We can see that a lot of memory gets allocated in paths through -`ResourceManager.loadApkAssets`. To get the total memory that was allocated -this way, we can enter "loadApkAssets" into the Focus textbox. This will only -show callstacks where some frame matches "loadApkAssets". +`AssetManager.applyStyle`. To get the total memory that was allocated +this way, we can enter "applyStyle" into the Focus textbox. This will only +show callstacks where some frame matches "applyStyle". - + From this we have a clear idea where in the code we have to look. From the code we can see how that memory is being used and if we actually need all of -it. In this case the key is the `_CompressedAsset` that requires decompressing -into RAM rather than being able to (_cleanly_) memory-map. By not compressing -these data, we can save RAM. +it. ## {#java-hprof} Analyzing the Java Heap **Java Heap Dumps require Android 11.** -NOTE: For detailed instructions about the Java heap profiler and - troubleshooting see the [Data sources > Java heap profiler]( +NOTE: For detailed instructions about capturing Java heap dumps and + troubleshooting see the [Data sources > Java heap dumps]( /docs/data-sources/java-heap-profiler.md) page. -### {#capture-profile-java} Capturing the profile +### {#capture-profile-java} Dumping the java heap We can get a snapshot of the graph of all the Java objects that constitute the Java heap. We use the `tools/java_heap_dump` script. If you are having trouble make sure you are using the [latest version]( @@ -415,12 +413,12 @@ and is generally the highest-signal. The rightmost `[merged]` stacks is the sum of all objects that are too small to be displayed. - + The tabs that are available are -* **space**: how many bytes are retained via this path to the GC root. -* **objects**: how many objects are retained via this path to the GC root. +* **Size**: how many bytes are retained via this path to the GC root. +* **Objects**: how many objects are retained via this path to the GC root. If we want to only see callstacks that have a frame that contains some string, we can use the Focus feature. If we want to know all allocations that have to @@ -430,7 +428,7 @@ graph, we can filter by the names of the classes. If we wanted to see everything that could be caused by notifications, we can put "notification" in the Focus box. - + We aggregate the paths per class name, so if there are multiple objects of the same type retained by a `java.lang.Object[]`, we will show one element as its
diff --git a/docs/data-sources/java-heap-profiler.md b/docs/data-sources/java-heap-profiler.md index f67cf03..e3d5921 100644 --- a/docs/data-sources/java-heap-profiler.md +++ b/docs/data-sources/java-heap-profiler.md
@@ -1,15 +1,17 @@ -# Memory: Java heap profiler +# Memory: Java heap dumps -NOTE: The Java heap profiler requires Android 11 or higher +NOTE: Capturing Java heap dumps requires Android 11 or higher See the [Memory Guide](/docs/case-studies/memory.md#java-hprof) for getting -started with Java heap profiling. +started with Java heap dumps. -Conversely from the [Native heap profiler](native-heap-profiler.md), the Java -heap profiler reports full retention graphs of managed objects but not -call-stacks. The information recorded by the Java heap profiler is of the form: -_Object X retains object Y, which is N bytes large, through its class member -named Z_. +Conversely from [Native heap profiles](native-heap-profiler.md), Java heap dumps +report full retention graphs of managed objects but not call-stacks. The +information recorded in a Java heap dump is of the form: _Object X retains +object Y, which is N bytes large, through its class member named Z_. + +Java heap dumps are not to be confused with profiles taken by the +[Java heap sampler](native-heap-profiler.md#java-heap-sampling) ## UI @@ -17,13 +19,13 @@ diamond in the _"Heap Profile"_ track of a process. Each diamond corresponds to a heap dump. - + - + The native size of certain objects is represented as an extra child node in the flamegraph, prefixed with "[native]". The extra node counts as an extra object. -This is available only on Android T+. +This is available only on Android 13 or higher. ## SQL @@ -93,7 +95,7 @@ ## TraceConfig -The Java heap profiler is configured through the +The Java heap dump data source is configured through the [JavaHprofConfig](/docs/reference/trace-config-proto.autogen#JavaHprofConfig) section of the trace config.
diff --git a/docs/data-sources/native-heap-profiler.md b/docs/data-sources/native-heap-profiler.md index f051124..1859295 100644 --- a/docs/data-sources/native-heap-profiler.md +++ b/docs/data-sources/native-heap-profiler.md
@@ -1,13 +1,17 @@ -# Native heap profiler +# Heap profiler NOTE: **heapprofd requires Android 10 or higher** -Heapprofd is a tool that tracks native heap allocations & deallocations of an -Android process within a given time period. The resulting profile can be used to +Heapprofd is a tool that tracks heap allocations & deallocations of an Android +process within a given time period. The resulting profile can be used to attribute memory usage to particular call-stacks, supporting a mix of both native and java code. The tool can be used by Android platform and app developers to investigate memory issues. +By default, the tool records native allocations and deallocations done with +malloc/free (or new/delete). It can be configured to record java heap memory +allocations instead: see [Java heap sampling](#java-heap-sampling) below. + On debug Android builds, you can profile all apps and most system services. On "user" builds, you can only use it on apps with the debuggable or profileable manifest flag. @@ -25,7 +29,7 @@  - + ## SQL @@ -80,14 +84,14 @@ The resulting profile proto contains four views on the data -* **Unreleased size**: how many bytes were allocated but not freed at this - callstack the moment the dump was created. -* **Total size**: how many bytes were allocated (including ones freed at the - moment of the dump) at this callstack -* **Unreleased count**: how many allocations without matching frees were done at - this callstack. -* **Total count**: how many allocations (including ones with matching frees) - were done at this callstack. +* **Unreleased malloc size**: how many bytes were allocated but not freed at + this callstack the moment the dump was created. +* **Total malloc size**: how many bytes were allocated (including ones freed at + the moment of the dump) at this callstack. +* **Unreleased malloc count**: how many allocations without matching frees were + done at this callstack. +* **Total malloc count**: how many allocations (including ones with matching + frees) were done at this callstack. _(Googlers: You can also open the gzipped protos using http://pprof/)_ @@ -217,6 +221,41 @@ </manifest> ``` +## {#java-heap-sampling} Java heap sampling + +NOTE: **Java heap sampling is available on Android 12 or higher** + +NOTE: **Java heap sampling is not to be confused with [Java heap +dumps](/docs/data-sources/java-heap-profiler.md)** + +Heapprofd can be configured to track Java allocations instead of native one. +* By setting adding `heaps: "com.android.art"` in + [HeapprofdConfig](/docs/reference/trace-config-proto.autogen#HeapprofdConfig). +* By adding `--heaps com.android.art` to the invocation of + [`tools/heap_profile`](/docs/reference/heap_profile-cli). + +Unlike java heap dumps (which show the retention graph of a snapshot of the live +objects) but like native heap profiles, java heap samples show callstacks of +allocations over time of the entire profile. + +Java heap samples only show callstacks of when objects are created, not when +they're deleted or garbage collected. + + + +The resulting profile proto contains two views on the data: + +* **Total allocation size**: how many bytes were allocated at this callstack + over time of the profile until this point. The bytes might have been freed or + not, the tool does not keep track of that. +* **Total allocation count**: how many object were allocated at this callstack + over time of the profile until this point. The objects might have been freed + or not, the tool does not keep track of that. + +Java heap samples are useful to understand memory churn showing the call stack +of which parts of the code large allocations are attributed to as well as the +allocation type from the ART runtime. + ## DEDUPED frames If the name of a Java method includes `[DEDUPED]`, this means that multiple
diff --git a/docs/design-docs/heapprofd-sampling.md b/docs/design-docs/heapprofd-sampling.md index 3ebfb3d..9f238b5 100644 --- a/docs/design-docs/heapprofd-sampling.md +++ b/docs/design-docs/heapprofd-sampling.md
@@ -1,6 +1,6 @@ # heapprofd: Sampling for Memory Profiles -_tomlewin, fmayer **·** 2021-04-14_ +_tomlewin, fmayer **·** 2021-04-14_ ## Background @@ -173,7 +173,7 @@ [algorithm]: https://cs.android.com/android/platform/superproject/+/master:external/perfetto/src/profiling/memory/sampler.h -[example visualization]: /docs/images/syssrv-apk-assets-two.png +[example visualization]: /docs/images/native-heap-prof.png [Android Heap Profiler]: /docs/design-docs/heapprofd-design [mean absolute percentage error]: https://en.wikipedia.org/wiki/Mean_absolute_percentage_error -[full results]: https://gist.github.com/fmayer/3aafcaf58f8db09714ba09aa4ac2b1ac \ No newline at end of file +[full results]: https://gist.github.com/fmayer/3aafcaf58f8db09714ba09aa4ac2b1ac
diff --git a/docs/images/java-flamegraph-focus.png b/docs/images/java-flamegraph-focus.png deleted file mode 100644 index d75648f..0000000 --- a/docs/images/java-flamegraph-focus.png +++ /dev/null Binary files differ
diff --git a/docs/images/java-flamegraph.png b/docs/images/java-flamegraph.png deleted file mode 100644 index fa58843..0000000 --- a/docs/images/java-flamegraph.png +++ /dev/null Binary files differ
diff --git a/docs/images/java-heap-graph-focus.png b/docs/images/java-heap-graph-focus.png new file mode 100644 index 0000000..cd5d232 --- /dev/null +++ b/docs/images/java-heap-graph-focus.png Binary files differ
diff --git a/docs/images/java-heap-graph.png b/docs/images/java-heap-graph.png new file mode 100644 index 0000000..829c7fe --- /dev/null +++ b/docs/images/java-heap-graph.png Binary files differ
diff --git a/docs/images/java-heap-samples.png b/docs/images/java-heap-samples.png new file mode 100644 index 0000000..7bd4307 --- /dev/null +++ b/docs/images/java-heap-samples.png Binary files differ
diff --git a/docs/images/native-flamegraph.png b/docs/images/native-flamegraph.png deleted file mode 100644 index 88b92e3..0000000 --- a/docs/images/native-flamegraph.png +++ /dev/null Binary files differ
diff --git a/docs/images/native-heap-prof-focus.png b/docs/images/native-heap-prof-focus.png new file mode 100644 index 0000000..acb3da6 --- /dev/null +++ b/docs/images/native-heap-prof-focus.png Binary files differ
diff --git a/docs/images/native-heap-prof.png b/docs/images/native-heap-prof.png new file mode 100644 index 0000000..5b7f9fa --- /dev/null +++ b/docs/images/native-heap-prof.png Binary files differ
diff --git a/docs/images/profile-diamond.png b/docs/images/profile-diamond.png index 5f14756..8c12fbc 100644 --- a/docs/images/profile-diamond.png +++ b/docs/images/profile-diamond.png Binary files differ
diff --git a/docs/images/syssrv-apk-assets-focus.png b/docs/images/syssrv-apk-assets-focus.png deleted file mode 100644 index c5070dd..0000000 --- a/docs/images/syssrv-apk-assets-focus.png +++ /dev/null Binary files differ
diff --git a/docs/images/syssrv-apk-assets-two.png b/docs/images/syssrv-apk-assets-two.png deleted file mode 100644 index dea36ef..0000000 --- a/docs/images/syssrv-apk-assets-two.png +++ /dev/null Binary files differ
diff --git a/docs/quickstart/heap-profiling.md b/docs/quickstart/heap-profiling.md index 6f9692f..3a58f20 100644 --- a/docs/quickstart/heap-profiling.md +++ b/docs/quickstart/heap-profiling.md
@@ -60,7 +60,7 @@ script. Then start the profile: ```bash -python /path/to/heap_profile -n system_server +python /path/to/heap_profile -n system_server ``` ## View profile @@ -70,7 +70,7 @@ _"Heap profile"_.  - + ## Next steps
diff --git a/docs/toc.md b/docs/toc.md index f9e010e..dc3c717 100644 --- a/docs/toc.md +++ b/docs/toc.md
@@ -18,8 +18,8 @@ * [Data sources](#) * [Memory](#) * [Counters and events](data-sources/memory-counters.md) - * [Native heap profiler](data-sources/native-heap-profiler.md) - * [Java heap profiler](data-sources/java-heap-profiler.md) + * [Heap profiler](data-sources/native-heap-profiler.md) + * [Java heap dumps](data-sources/java-heap-profiler.md) * [CPU](#) * [Scheduling events](data-sources/cpu-scheduling.md) * [System calls](data-sources/syscalls.md)
