| # Perfetto Stress Test |
| |
| This is a test harness that to stress test the client library (DataSource-level |
| only for now). |
| |
| The test is based on a number of configs in /test/stress_test/configs/*.cfg |
| (NOTE: they must be listed in configs/BUILD.gn). |
| The config is a /protos/perfetto/config/stress_test_config.proto message, which |
| embeds the configuration of the test and a whole trace config. |
| |
| Each configs defines a testing scenario, determining the general trace config |
| and all the settings of the test (e.g., how many producer processes to spawn, |
| the write timings). |
| |
| The test is based on exec()-ing `traced` (the tracing service), `perfetto` (the |
| consumer cmdline client) and a variable number of `stress_producer` instances. |
| |
| `stress_producer` emits events at a configurable rate, writing predictable |
| sequences of numbers / string, so that the test harness can easily detect |
| corruptions, out-of-order events or gaps. |
| |
| After running each test, the `stress_test` binary reads back the trace and |
| performs a bunch of checks: |
| |
| - Checks that the number of sequences is exactly equal to #processes x #threads. |
| - Checks that each sequence has all the expected packets in the right sequence |
| - Checks the payload and correctness of proto nesting of each trace packet. |
| - Reports CPU/Memory/Context-switch numbers for the service and producer |
| processes. |
| |
| Each test config is isolated from the others. All processes are killed and |
| re-spawned for each test. |
| |
| The stdout/err of each process is saved in a dedicated /tmp/ folder, as well as |
| the resulting trace. |
| |
| ## Building and running the test |
| |
| ```bash |
| # This will recursively build traced, perfetto and stress_producer. |
| ninja -C out/default stress_test |
| |
| out/default/stress_test |
| ``` |
| |
| will output: |
| |
| ```txt |
| [307.909] stress_test.cc:116 Saving test results in /tmp/perfetto-ltIBJgA0 |
| |
| =============================================================== |
| Config: simple |
| =============================================================== |
| Metric Expected Actual |
| ------ -------- ------ |
| #Errors 0 0 |
| Duration [ms] 3000 3109 |
| Num threads 1 1 |
| Num packets 1000 1001 |
| Trace size [KB] 168 170 |
| Svc RSS [MB] 4 2 |
| Prod RSS [MB] --- 1 |
| Svc CPU [ms] --- 10 |
| Prod CPU [ms] --- 32 |
| Svc #ctxswitch --- 103 / 20 |
| Prod #ctxswitch --- 1022 / 1 |
| |
| =============================================================== |
| Config: bursts |
| =============================================================== |
| Metric Expected Actual |
| ------ -------- ------ |
| #Errors 0 0 |
| Duration [ms] 2000 2381 |
| Num threads 10 10 |
| Num packets 2675 20021 |
| Trace size [KB] 449 11063 |
| Svc RSS [MB] 32 17 |
| Prod RSS [MB] --- 1 |
| Svc CPU [ms] --- 98 |
| Prod CPU [ms] --- 17 |
| Svc #ctxswitch --- 704 / 1327 |
| Prod #ctxswitch --- 421 / 1 |
| ``` |
| |
| ```bash |
| $ ls -Rlh /tmp/perfetto-ltIBJgA0 |
| total 0 |
| drwxr-xr-x 16 primiano wheel 512B 5 Aug 09:16 bursts |
| drwxr-xr-x 9 primiano wheel 288B 5 Aug 09:16 simple |
| drwxr-xr-x 38 primiano wheel 1.2K 5 Aug 09:16 the_storm |
| |
| /tmp/perfetto-ltIBJgA0/bursts: |
| total 22752 |
| -rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 errors.log |
| -rw-r--r-- 1 primiano wheel 180B 5 Aug 09:16 perfetto.log |
| -rw-r--r-- 1 primiano wheel 441B 5 Aug 09:16 producer.0.log |
| ... |
| -rw-r--r-- 1 primiano wheel 441B 5 Aug 09:16 producer.9.log |
| -rw------- 1 primiano wheel 11M 5 Aug 09:16 trace |
| -rw-r--r-- 1 primiano wheel 407B 5 Aug 09:16 traced.log |
| |
| /tmp/perfetto-ltIBJgA0/simple: |
| total 400 |
| srwxr-xr-x 1 primiano wheel 0B 5 Aug 09:16 consumer.sock |
| -rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 errors.log |
| -rw-r--r-- 1 primiano wheel 178B 5 Aug 09:16 perfetto.log |
| -rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 producer.0.log |
| srwxr-xr-x 1 primiano wheel 0B 5 Aug 09:16 producer.sock |
| -rw------- 1 primiano wheel 167K 5 Aug 09:16 trace |
| -rw-r--r-- 1 primiano wheel 406B 5 Aug 09:16 traced.log |
| |
| /tmp/perfetto-ltIBJgA0/the_storm: |
| total 524432 |
| -rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 errors.log |
| -rw-r--r-- 1 primiano wheel 184B 5 Aug 09:16 perfetto.log |
| -rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 producer.0.log |
| ... |
| -rw-r--r-- 1 primiano wheel 0B 5 Aug 09:16 producer.127.log |
| -rw------- 1 primiano wheel 248M 5 Aug 09:16 trace |
| -rw-r--r-- 1 primiano wheel 408B 5 Aug 09:16 traced.log |
| ``` |
| |
| ## TODOs |
| |
| The following scenarios requires more coverage: |
| |
| - Nested messages. |
| - Force losses and check that the last_dropped flag is consistent. |
| - Flushes and scraping. |
| - Report data losses in the test output. |
| - Multibuffer scenarios. |
| - write_into_file=true. |
| - Vary page size, smb size. |
