Add some end to end tests to the CTS test suite
There are a bunch of end to end tests in test:integrationtests
(android_integrationtest.cc, cmdline_integrationtest.cc,
ftrace_integrationtest.cc, traced_integrationtest.cc).
These tests are linked in perfetto_integrationtests, which is run in CI
on a couple of devices.
I think the intention in the code base was for these tests to be
included in CtsPerfettoTestCases. All devices have to pass CTS, so that
gives us a better guarantee that perfetto is working.
Unfortunately, our build files did not include these tests correctly
into the CtsPerfettoTestCases.
Before this commit
GN:
```
//test/cts:
static_library("perfetto_cts_deps") {
deps = [ "test:integrationtests" ],
}
//test:
source_set("integrationtests") {
sources = [
"android_integrationtest.cc",
"cmdline_integrationtest.cc",
"ftrace_integrationtest.cc",
"traced_integrationtest.cc",
]
}
```
BP:
```
//test/cts/Android.bp (NOT autogenerated)
cc_test {
name: "CtsPerfettoTestCases"
static_libs: ["perfetto_cts_deps"]
}
//Android.bp (autogenerated from GN)
cc_library_static {
name: "perfetto_cts_deps"
srcs: [ "perfetto_test_integrationtests" ]
}
filegroup {
name: "perfetto_test_integrationtests"
srcs: [
"android_integrationtest.cc",
"cmdline_integrationtest.cc",
"ftrace_integrationtest.cc",
"traced_integrationtest.cc",
]
}
```
The files in the filegroup `perfetto_test_integrationtests` were
included into the cc_library_static `perfetto_cts_deps`. When
CtsPerfettoTestCases links `perfetto_cts_deps`, it doesn't include all
the objects, it includes only the one that strictly needs and it
excludes the one that only register initializers (i.e. there's no
--whole-archive).
This commit fixes the problem by listing the _integrationtest.cc
explicitly in `CtsPerfettoTestCases`.
In order to do that, this commit:
* Renames end_to_end_integrationtest_cts.cc to
producer_to_consumer_integrationtest_cts.cc so that the name
end_to_end can be reused later.
* Renames the source_set //test:integrationtests to
perfetto_end_to_end_integrationtests, so that it can be exported to
Android.bp (exported targets are not prefixed with their path).
Before this change CtsPerfettoTestCases was taking ~2 minutes and 30
seconds on my redfin. After this change CtsPerfettoTestCases is taking
2 minutex and 40 seconds, so this commit also updates the timeout in the
XML.
This is the diff between the old CTS test list and the new one:
```
HeapprofdJavaCtsTest.
DebuggableAppRuntime
ProfileableAppRuntime
ReleaseAppRuntime
DebuggableAppRuntimeByPid
DebuggableAppOom
ProfileableAppOom
ReleaseAppOom
DebuggableAppOomNotSelected
HeapprofdCtsTest.
DebuggableAppRuntime
DebuggableAppStartup
ProfileableAppRuntime
ProfileableAppStartup
ReleaseAppRuntime
ReleaseAppStartup
NonProfileableAppRuntime
NonProfileableAppStartup
JavaHeapRuntime
JavaHeapStartup
ProfilePlatformProcess
TracedPerfCtsTest.
SystemWideDebuggableApp
SystemWideProfileableApp
SystemWideNonProfileableApp
SystemWideReleaseApp
ProfilePlatformProcess
PerfettoDeviceFeatureTest.
TestMaxCpusForAtraceChmod
PerfettoCtsTest.
TestProducerActivity
TestProducerService
TestProducerIsolatedService
+PerfettoAndroidIntegrationTest.
+ TestKmemActivity
+ TestBatteryTracing
+PerfettoCmdlineTest.
+ InvalidCases
+ Version
+ TxtConfig
+ SimpleConfig
+ DetachAndAttach
+ StartTracingTrigger
+ StopTracingTrigger
+ NoDataNoFileWithoutTrigger
+ StopTracingTriggerFromConfig
+ TriggerFromConfigStopsFileOpening
+ Query
+ CmdTriggerWithUploadFlag
+ TriggerCloneSnapshot
+ SaveForBugreport
+ SaveForBugreport_WriteIntoFile
+ Clone
+ UnavailableBugreportLeavesNoEmptyFiles
+ DISABLED_SaveForBugreport_Triggers
+PerfettoFtraceIntegrationTest.
+ TestFtraceProducer
+ TestFtraceFlush
+ KernelAddressSymbolization
+ ReportFtraceFailuresInStats
+PerfettoTracedIntegrationTest.
+ TestFakeProducer
+ VeryLargePackets
+ UnresponsiveProducer
+ DetachAndReattach
+ ReattachFailsAfterTimeout
+ TestProducerProvidedSMB
+ QueryServiceStateLargeResponse
+ TraceFilterLargePackets
```
Change-Id: Idcd04d6239f276b95abeca09ebb0891d5cf98a3f
diff --git a/Android.bp b/Android.bp
index 4265016..77b3dda 100644
--- a/Android.bp
+++ b/Android.bp
@@ -1204,7 +1204,6 @@
":perfetto_src_ipc_perfetto_ipc",
":perfetto_src_kallsyms_kallsyms",
":perfetto_src_kernel_utils_syscall_table",
- ":perfetto_src_perfetto_cmd_bugreport_path",
":perfetto_src_protozero_filtering_bytecode_common",
":perfetto_src_protozero_filtering_bytecode_generator",
":perfetto_src_protozero_filtering_bytecode_parser",
@@ -1238,12 +1237,11 @@
":perfetto_src_tracing_ipc_default_socket",
":perfetto_src_tracing_ipc_producer_producer",
":perfetto_src_tracing_ipc_service_service",
- ":perfetto_test_integrationtests",
":perfetto_test_test_helper",
"test/cts/device_feature_test_cts.cc",
- "test/cts/end_to_end_integrationtest_cts.cc",
"test/cts/heapprofd_java_test_cts.cc",
"test/cts/heapprofd_test_cts.cc",
+ "test/cts/producer_to_consumer_integrationtest_cts.cc",
"test/cts/traced_perf_test_cts.cc",
],
static_libs: [
@@ -1705,6 +1703,17 @@
},
}
+// GN: //test:perfetto_end_to_end_integrationtests
+filegroup {
+ name: "perfetto_end_to_end_integrationtests",
+ srcs: [
+ "test/android_integrationtest.cc",
+ "test/cmdline_integrationtest.cc",
+ "test/ftrace_integrationtest.cc",
+ "test/traced_integrationtest.cc",
+ ],
+}
+
// GN: //test:perfetto_gtest_logcat_printer
cc_library_static {
name: "perfetto_gtest_logcat_printer",
@@ -1870,6 +1879,7 @@
name: "perfetto_integrationtests",
srcs: [
":perfetto_base_default_platform",
+ ":perfetto_end_to_end_integrationtests",
":perfetto_include_perfetto_base_base",
":perfetto_include_perfetto_ext_base_base",
":perfetto_include_perfetto_ext_base_version",
@@ -2131,7 +2141,6 @@
":perfetto_src_tracing_test_client_api_integrationtests",
":perfetto_src_tracing_test_test_support",
":perfetto_src_tracing_test_tracing_integration_test",
- ":perfetto_test_integrationtests",
":perfetto_test_test_helper",
],
shared_libs: [
@@ -11530,17 +11539,6 @@
],
}
-// GN: //test:integrationtests
-filegroup {
- name: "perfetto_test_integrationtests",
- srcs: [
- "test/android_integrationtest.cc",
- "test/cmdline_integrationtest.cc",
- "test/ftrace_integrationtest.cc",
- "test/traced_integrationtest.cc",
- ],
-}
-
// GN: //test/sanitizers:unittests
filegroup {
name: "perfetto_test_sanitizers_unittests",
diff --git a/gn/perfetto_integrationtests.gni b/gn/perfetto_integrationtests.gni
index ab788b8..ff7bb7f 100644
--- a/gn/perfetto_integrationtests.gni
+++ b/gn/perfetto_integrationtests.gni
@@ -31,7 +31,7 @@
# enable_perfetto_traced_probes implies enable_perfetto_platform_services.
perfetto_integrationtests_targets += [
"src/traced/probes/ftrace:integrationtests",
- "test:integrationtests",
+ "test:perfetto_end_to_end_integrationtests",
]
}
diff --git a/test/BUILD.gn b/test/BUILD.gn
index 6b995ed..32f7510 100644
--- a/test/BUILD.gn
+++ b/test/BUILD.gn
@@ -17,7 +17,7 @@
import("../gn/perfetto.gni")
if (!build_with_chromium && enable_perfetto_integration_tests) {
- source_set("integrationtests") {
+ source_set("perfetto_end_to_end_integrationtests") {
testonly = true
deps = [
":test_helper",
diff --git a/test/cts/Android.bp b/test/cts/Android.bp
index c05189a..ccd1745 100644
--- a/test/cts/Android.bp
+++ b/test/cts/Android.bp
@@ -11,10 +11,11 @@
name: "CtsPerfettoTestCases",
srcs: [
"device_feature_test_cts.cc",
- "end_to_end_integrationtest_cts.cc",
"heapprofd_java_test_cts.cc",
"heapprofd_test_cts.cc",
+ "producer_to_consumer_integrationtest_cts.cc",
"traced_perf_test_cts.cc",
+ ":perfetto_end_to_end_integrationtests",
":perfetto_protos_perfetto_config_cpp_gen",
":perfetto_protos_perfetto_common_cpp_gen"
],
diff --git a/test/cts/AndroidTest.xml b/test/cts/AndroidTest.xml
index 27ccef1..1afa927 100644
--- a/test/cts/AndroidTest.xml
+++ b/test/cts/AndroidTest.xml
@@ -40,9 +40,9 @@
<test class="com.android.tradefed.testtype.GTest" >
<option name="native-test-device-path" value="/data/local/tmp" />
<option name="module-name" value="CtsPerfettoTestCases" />
- <option name="runtime-hint" value="0m40s" />
+ <option name="runtime-hint" value="2m40s" />
<!-- test-timeout unit is ms -->
- <option name="native-test-timeout" value="40000" />
+ <option name="native-test-timeout" value="180000" />
</test>
<!-- When this test is run in a Mainline context (e.g. with `mts-tradefed`), only enable it if
diff --git a/test/cts/BUILD.gn b/test/cts/BUILD.gn
index 9294cdb..3feee32 100644
--- a/test/cts/BUILD.gn
+++ b/test/cts/BUILD.gn
@@ -20,7 +20,6 @@
complete_static_lib = true
testonly = true
deps = [
- "..:integrationtests",
"../..:libperfetto_client_experimental",
"../../gn:default_deps",
"../../gn:gtest_and_gmock",
@@ -31,13 +30,14 @@
"../../protos/perfetto/trace:cpp",
"../../protos/perfetto/trace/profiling:cpp",
"../../src/base:test_support",
+ "../../src/protozero/filtering:bytecode_generator",
"../../test:test_helper",
]
sources = [
"device_feature_test_cts.cc",
- "end_to_end_integrationtest_cts.cc",
"heapprofd_java_test_cts.cc",
"heapprofd_test_cts.cc",
+ "producer_to_consumer_integrationtest_cts.cc",
"traced_perf_test_cts.cc",
]
}
diff --git a/test/cts/end_to_end_integrationtest_cts.cc b/test/cts/producer_to_consumer_integrationtest_cts.cc
similarity index 100%
rename from test/cts/end_to_end_integrationtest_cts.cc
rename to test/cts/producer_to_consumer_integrationtest_cts.cc
diff --git a/tools/gen_android_bp b/tools/gen_android_bp
index b154ee3..e6021fe 100755
--- a/tools/gen_android_bp
+++ b/tools/gen_android_bp
@@ -73,6 +73,7 @@
'//test/cts:perfetto_cts_deps',
'//test/cts:perfetto_cts_jni_deps',
'//test:perfetto_gtest_logcat_printer',
+ '//test:perfetto_end_to_end_integrationtests',
'//test/vts:perfetto_vts_deps',
]
