EXPERIMENT: evaluating compression vs interning on Wakelock
NOT FOR REVIEW
Prompt
We are going to do some investigation about interning vs compression.
You'll have to write some python or C++ throwaway scripts to help me with some research on the topic.
Pre-work:
Take the long_trace.pftrace file and extract out only the TracePacket that contain an AppWakelockBundle (defined in app_wakelock_data.proto). Save that in wakelock.pftrace.
Context:
you'll notice that wakelocks are stored in an efficient manner via a union of interning + columnar encoding
- There is an AppWakelockInfo structure which is interned.
- When a wakelock is detected and stored into the trace:
- If the wakelock is new, a new AppWakelockInfo is created and appended to TracePacket.interned_data.app_wakelock_info (see interned_data.proto)
- The index of the newly created intern entry is appended to intern_id
- The timestamp of the even is appended to encoded_ts
- Ignore the other two fiels AppWakelockInfo info = 3; and acquired = 4
So the way you interpret the data is that encoded_ts and intern_id have the same cardinality. intern_id refers to the entry previously emitted in interned_data
Assignment:
I want you to de-intern the trace, and pretend that the AppWakelockInfo was written as a record, in a dumb way, without any interning.
SO I want you to do this:
- Modify TrackEvent and add AppWakelockInfo as one of the possible fields
- Write a python/c++ script that parses the input trace and de-interns the data. FOr each entry in encoded_ts + intern_id, you shall create a new TrackEvent of type instant, with the appopriate timestamp, and same AppWakelockInfo fields (skip the iid field because it's only used for interning)
- Hence create a wakelock_uninterned.pftrace, taking the previous wakelock.pftrace and expanding each wakelock as described above
- Then I want you to update the importer code in trace_processor to import also this file. I want you to verify that you did a good job ensuring that at the trace_processor level, the data looks identical. You'll have to join the slice table with the tracks table, and filter for track of type 'app_wakelock_events', and then check the slice and its args
After you have done all this, I want you to take both traces (wakelock.pftrace and wakelock_uninterned.pftrace) and apply compression as follows. You have to try the combination of the following matrix
zstd levels={10, 15, 19} with block sizes of 512K, 1MB, 2MB, 8MB (that is, apply compression indpendently to blocks of these sizes)
then write me a report with comparing the reults of this compression matrix on both traces in a tabular format
Perfetto is an open-source suite of SDKs, daemons and tools which use tracing to help developers understand the behaviour of complex systems and root-cause functional and performance issues on client and embedded systems.
It is a production-grade tool that is the default tracing system for the Android operating system and the Chromium browser.
Perfetto is not a single tool, but a collection of components that work together:
Perfetto was designed to be a versatile and powerful tracing system for a wide range of use cases.
ftrace, allowing you to visualize scheduling, syscalls, interrupts, and custom kernel tracepoints on a timeline.chrome://tracing. Use it to debug and root-cause issues in the browser, V8, and Blink.We‘ve designed our documentation to guide you to the right information as quickly as possible, whether you’re a newcomer to performance analysis or an experienced developer.
New to tracing? If you're unfamiliar with concepts like tracing and profiling, start here:
Ready to dive in? Our “Getting Started” guide is the main entry point for all users. It will help you find the right tutorials and documentation for your specific needs:
Want the full overview? For a comprehensive look at what Perfetto is, why it's useful, and who uses it, see our main documentation page:
For users interested in the Debian distribution of Perfetto, the official source of truth and packaging efforts are maintained at Debian Perfetto Salsa Repository
Have questions? Need help?
We follow Google's Open Source Community Guidelines.