| # Android Hardware Smoke Test | |
| An integration and compatibility smoke test suite | |
| designed to verify visual rendering correctness | |
| on Android hardware. | |
| ## Prerequisites & Initial Setup | |
| Because this integration test project | |
| follows a minimal-boilerplate pattern, | |
| standard binary Gradle wrappers and properties | |
| are not committed to the repository. | |
| Before compiling or running | |
| any connected JUnit/instrumented tests locally for the first time, | |
| simply run the standard Flutter project regeneration command | |
| in this directory to restore the missing wrappers: | |
| ```sh | |
| flutter create --platform=android --no-overwrite . | |
| ``` | |
| This will cleanly restore the missing wrapper scripts | |
| (`gradlew`, `gradlew.bat`) and wrapper configurations | |
| without modifying any of the customized build definitions, | |
| Java/Kotlin test harnesses, or package sources. | |
| ## 1. Overview & Purpose | |
| The primary objective of the `android_hardware_smoke_test` | |
| is to provide a **fully self-contained Android instrumented test suite**. | |
| This allows Android hardware manufacturers (OEMs) | |
| to run visual regression, performance, and GPU compatibility tests | |
| on a precompiled APK directly on their target devices. | |
| **The test execution requires no Flutter SDK, | |
| Dart CLI commands, or host-side orchestration.** | |
| Feedback is reported directly in standard native Android JUnit test reports. | |
| --- | |
| ## 2. Structural Differences: `android_hardware_smoke_test` vs. `android_engine_test` | |
| While both suites verify rendering correctness, they are architected differently to support different workflows: | |
| | Dimension | `android_engine_test` | `android_hardware_smoke_test` | | |
| | :--- | :--- | :--- | | |
| | **Verification Location** | **Host-Only** (CI PC) | **Dual-Mode**: On-Device (OEM) & On-Host (CI) | | |
| | **Golden Comparison** | Host-side only against Skia Gold. | **OEM Mode**: In-App pixel comparison against bundled assets.<br>**CI Mode**: Host-side comparisons against repository files. | | |
| | **Device Role** | Passive target rendering static views. | Active participant executing on-device JUnit orchestration. | | |
| | **Target Audience** | Core Engine Contributors & CI Shards. | Android Hardware Manufacturers (OEMs) & CI Shards. | | |
| --- | |
| ## 3. Dual-Mode Architecture | |
| ```mermaid | |
| flowchart TD | |
| TestInit(["Test Initiation"]) --> Mode{"Which Mode?"} | |
| Mode -->|"Host-Driven Mode"| HostScript["driver script<br/>(test_driver/driver_test.dart)"] | |
| Mode -->|"Instrumented Mode"| JUnit["native Android JUnit<br/>(FlutterActivityTest)"] | |
| HostScript --> RequestData["Driver script connects and requests<br/>testName via driver.requestData()"] | |
| JUnit --> SendMessage["JUnit test runner sends<br/>testName over Message Channel"] | |
| RequestData --> RenderHost["Dart app renders target state"] | |
| SendMessage --> RenderDevice["Dart app renders target state"] | |
| RenderHost --> HostRender["Dart app returns image bytes over channel"] | |
| RenderDevice --> OnDeviceCompare["Dart app performs local on-device golden<br/>comparison against bundled assets"] | |
| HostRender --> CompareHost["Driver script asserts exact match<br/>against local filesystem"] | |
| OnDeviceCompare --> Report["Dart app returns success/fail<br/>to JUnit test runner"] | |
| ``` | |
| ### Host-Driven Driver Mode (CI / Host-Driven) | |
| * **Orchestration**: Orchestrated by the host PC using `flutter drive`. | |
| * **Execution**: The host script (`test_driver/driver_test.dart`) | |
| commands the app (`lib/main.dart` | |
| through thin wrapper `integration_test/integration_test_wrapper.dart`) | |
| to transition states. | |
| The app captures the repaint boundary, | |
| base64-encodes it, and streams the bytes back to the host | |
| over the message channel. | |
| The host driver decodes the bytes | |
| and asserts visual matches against local repository baselines | |
| on the host filesystem. | |
| ### Instrumented On-Device Mode (OEM / Standalone) | |
| * **Orchestration**: Runs purely on the device under Android `AndroidJUnit4` runner. | |
| * **Execution**: Kotlin JUnit code (`FlutterActivityTest.kt`) | |
| launches the main activity and sends the test payload | |
| over a JSON message channel. | |
| The app (`lib/main.dart`) renders the widget, | |
| performs a local pixel-by-pixel on-device comparison | |
| against baseline images bundled within the APK assets, | |
| and replies with the status to the Java runner | |
| to pass or fail the JUnit assertion. | |
| --- | |
| ## 4. How to Run the Tests | |
| Unlock your connected Android device or emulator | |
| and ensure it is active before executing any of these commands. | |
| ### A. Running via Host Driver (CI / Host-Driven) | |
| This mode is used to execute visual assertions | |
| locally on your PC or in CI pipelines, | |
| and to manage the local golden baselines. | |
| * **Command to run the driver test suite**: | |
| ```sh | |
| # Execute from the android_hardware_smoke_test root directory | |
| flutter drive -v \ | |
| --driver=test_driver/driver_test.dart \ | |
| --target=integration_test/integration_test_wrapper.dart \ | |
| --no-dds | |
| ``` | |
| * **Command to capture/update reference golden baselines**: | |
| Running with `UPDATE_GOLDENS=true` | |
| writes or overwrites the local PNG baselines | |
| under `test_driver/goldens/` on the host. | |
| Because the statically compiled `AndroidManifest.xml` | |
| is the single source of truth, | |
| the app will automatically self-report its active backend variant. | |
| To capture or update the baseline for a specific graphics variant, | |
| simply edit `android/app/src/main/AndroidManifest.xml` | |
| to set the desired `io.flutter.embedding.android.ImpellerBackend` value, | |
| then execute: | |
| ```sh | |
| UPDATE_GOLDENS=true flutter drive -v \ | |
| --driver=test_driver/driver_test.dart \ | |
| --target=integration_test/integration_test_wrapper.dart \ | |
| --no-dds | |
| ``` | |
| --- | |
| ### B. Running Instrumented Tests (OEM / Self-Contained) | |
| > [!IMPORTANT] | |
| > **Asset Bundling Precondition**: | |
| > Because instrumented tests run completely standalone on the device, | |
| > they compare pixels against baseline images | |
| > bundled as read-only assets inside the APK. | |
| > You **must** first generate the local baselines | |
| > under `test_driver/goldens/` | |
| > using the **Host-Driven Driver Mode (with `UPDATE_GOLDENS=true`)** | |
| > before compiling and building the instrumented APK. | |
| * **Command to compile and run the native JUnit suite**: | |
| ```sh | |
| # Execute from the 'android' subdirectory | |
| cd android | |
| ./gradlew :app:connectedDebugAndroidTest \ | |
| -Pandroid.testInstrumentationRunnerArguments.class=com.example.android_hardware_smoke_test.FlutterActivityTest \ | |
| -s | |
| ``` | |
| > [!NOTE] | |
| > **Statically Compiled Single Source of Truth**: | |
| > The app's compiled `AndroidManifest.xml` `<meta-data>` tag | |
| > is the single source of truth | |
| > for the graphics backend configuration | |
| > under **both** Instrumented On-Device Mode (OEM) | |
| > and Host-Driven Driver Mode (CI). | |
| > | |
| > * **Instrumented On-Device Mode (OEM)**: | |
| > The native Kotlin JUnit harness (`FlutterActivityTest.kt`) | |
| > reads this value dynamically using the `PackageManager` API | |
| > and routes it to Dart. | |
| > * **Host-Driven Driver Mode (CI / Host)**: | |
| > The Dart app queries the native Android embedder | |
| > via a custom `MethodChannel` to self-discover its compiled backend | |
| > and self-reports it to the host test script | |
| > inside its JSON reply payload, | |
| > completely eliminating the need for environment variables on the host PC. | |
| > | |
| > To switch the active graphics backend manually for local runs, | |
| > open `android/app/src/main/AndroidManifest.xml` | |
| > and update the `io.flutter.embedding.android.ImpellerBackend` value: | |
| > | |
| > ```xml | |
| > <!-- Enable Vulkan: --> | |
| > <meta-data android:name="io.flutter.embedding.android.ImpellerBackend" android:value="vulkan" /> | |
| > | |
| > <!-- Enable OpenGLES: --> | |
| > <meta-data android:name="io.flutter.embedding.android.ImpellerBackend" android:value="opengles" /> | |
| > ``` | |
| > [!NOTE] | |
| > **Automated HTML Screenshot Embedding (`embedTestResultImages`)**: | |
| > When running the Gradle command above, | |
| > a custom Kotlin DSL task named **`embedTestResultImages`** | |
| > executes automatically once the tests finish. | |
| > | |
| > It performs the following actions seamlessly: | |
| > 1. Prevents Gradle from auto-uninstalling the APKs prematurely | |
| > (via a `gradle.properties` injection). | |
| > 2. Queries the device sandbox cache | |
| > to discover all rendered `.png` files dynamically. | |
| > 3. Streams the raw binary images directly onto the host PC | |
| > using zero-copy ADB piping. | |
| > 4. Dynamically parses the generated HTML reports | |
| > and injects Alternative `<img>` elements | |
| > right next to the test outcome table cells. | |
| > 5. Executes a manual `adb uninstall` cleanup | |
| > to leave the target device perfectly clean. | |
| > | |
| > Once finished, | |
| > open `app/build/reports/androidTests/connected/debug/index.html` | |
| > to view the interactive report | |
| > with all rendering result snapshots embedded natively! | |
| --- | |
| ### C. Manual Command-Line Debugging (Without Gradle Orchestration) | |
| If you prefer to bypass Gradle entirely for custom debugging, | |
| you can manually build, install, and run the instrumentation | |
| using raw `adb` shell calls: | |
| 1. **Build and Install the packages manually**: | |
| ```sh | |
| # Run from the 'android' subdirectory | |
| cd android | |
| ./gradlew installDebug installDebugAndroidTest | |
| ``` | |
| 2. **Manually launch the native Android instrumentation test**: | |
| ```sh | |
| adb shell am instrument -w \ | |
| -e class com.example.android_hardware_smoke_test.FlutterActivityTest \ | |
| com.example.android_hardware_smoke_test.test/androidx.test.runner.AndroidJUnitRunner | |
| ``` | |
| 3. **Manually pull the generated snapshot off the device's sandbox**: | |
| Since the app remains installed during raw `adb` runs, | |
| you can copy the rendering result files manually: | |
| ```sh | |
| adb exec-out "run-as com.example.android_hardware_smoke_test cat cache/results/blueRectangleTest.png" \ | |
| > test_driver/results/blueRectangleTest.png | |
| ``` | |
| --- | |
| ### D. Running the CI Shard Locally | |
| Since the test suite is registered | |
| inside the central repository test orchestrator (`dev/bots/test.dart`), | |
| you can execute the full CI runner pipeline locally | |
| using standard dev-bot scripts: | |
| ```sh | |
| # Run the Vulkan graphics backend shard locally | |
| SHARD=android_hardware_smoke_vulkan_tests bin/cache/dart-sdk/bin/dart dev/bots/test.dart | |
| # Run the OpenGLES graphics backend shard locally | |
| SHARD=android_hardware_smoke_opengles_tests bin/cache/dart-sdk/bin/dart dev/bots/test.dart | |
| ``` | |
| --- | |
| ## 5. Test Suite Coverage & Technical Rationales | |
| The suite is composed of targeted visual regression test cases | |
| designed to exercise distinct GPU graphics pipelines | |
| and verify compatibility with driver and hardware configurations: | |
| | Test Case | Rendering Pipeline | Impeller/Hardware Mechanism Exercised | Reference/Source | | |
| | :--- | :--- | :--- | :--- | | |
| | **`blueRectangleTest`** | **Solid Vector Fills** | Standard vector rasterization and layout transformation. | Simple `canvas.drawRect` | | |
| | **`trianglePathTest`** | **Complex Paths** | Path triangulation, rasterization, and hardware anti-aliasing (MSAA). | Simple `canvas.drawPath` | | |
| | **`textTest`** | **Font Rendering** | Text layout (`TextPainter`), glyph caching, shaping, and font atlas rendering. | Simple `TextPainter.paint` | | |
| | **`imageTest`** | **Texture Sampling** | Image decoding, GPU texture uploading, and texture sampler rendering. Uses a 32x32 4-color checkerboard PNG to verify RGB color channel correctness. | Simple `canvas.drawImage` | | |
| | **`advancedBlendTest`** | **Advanced Blending** | Fragment shader blending and framebuffer fetch tile-memory optimizations (e.g. Vulkan subpass inputs, `EXT_shader_framebuffer_fetch` in GLES). Uses `BlendMode.difference`. | Mirrors [animated_advanced_blend.dart](/dev/benchmarks/macrobenchmarks/lib/src/animated_advanced_blend.dart). | | |
| | **`backdropFilterBlurTest`** | **Compositing & Blur** | Offscreen texture allocation, layer downscale/upscale passes, and multi-pass Gaussian blur filter execution. Uses `ImageFilter.blur(sigmaX: 5, sigmaY: 5)`. | Mirrors [backdrop_filter.dart](/dev/benchmarks/macrobenchmarks/lib/src/backdrop_filter.dart). | | |
| | **`platformViewTextureLayerTest`** | **Platform Views (Texture Layer)** | Embedded native Android views composition using Texture Layer Hybrid Composition (TLHC) via `PlatformViewsService.initSurfaceAndroidView`. | Mirrors texture layer composition pathways. | | |
| | **`platformViewHybridCompositionTest`** | **Platform Views (Hybrid Composition)** | Embedded native Android views composition using legacy Hybrid Composition (HC) via `PlatformViewsService.initExpensiveAndroidView`. Uses `AndroidViewSurface` to overlay native UI. | Mirrors legacy hybrid composition pathways. | | |
| | **`platformViewHybridCompositionPlusPlusTest`** | **Platform Views (Hybrid Composition++)** | Embedded native Android views composition using Hybrid Composition++ (HCPP) via `PlatformViewsService.initHybridAndroidView`. Skips cleanly if the device lacks HCPP hardware support. | Mirrors modern HCPP composition pathways. | | |
| --- | |
| ## 6. Platform View Screenshot Strategy | |
| Testing platform views requires capturing a screenshot | |
| of the physical screen layout | |
| that contains both the Flutter-rendered UI and native Android views | |
| (e.g., a native `TextView` wrapped under hybrid composition). | |
| Because these two contexts are rendered on separate hardware surface layers, | |
| standard in-process widget screenshot methods | |
| (like `RenderRepaintBoundary.toImage()`) | |
| cannot see or capture the native platform view pixels. | |
| To address this, | |
| the test suite implements a **No-Compositing System Screenshot Strategy**: | |
| ```mermaid | |
| flowchart TD | |
| TestInit(["Test Initiation"]) --> Mode{"Which Mode?"} | |
| Mode -->|"Host-Driven Mode"| HostScript["driver script<br/>(test_driver/driver_test.dart)"] | |
| Mode -->|"Instrumented Mode"| JUnit["native Android JUnit<br/>(FlutterActivityTest)"] | |
| HostScript --> RequestData["Driver script connects and requests<br/>testName via driver.requestData()"] | |
| JUnit --> SendMessage["JUnit test runner sends<br/>testName over Message Channel"] | |
| RequestData --> RenderHost["Dart app renders target state"] | |
| SendMessage --> RenderDevice["Dart app renders target state"] | |
| RenderHost -->|"Unique for PlatformView tests"| RenderHostPlatformView["Native Kotlin UI renders via PlatformView"]:::unique | |
| RenderDevice -->|"Unique for PlatformView tests"| RenderDevicePlatformView["Native Kotlin UI renders via PlatformView"]:::unique | |
| RenderHostPlatformView --> DrawSyncHost["Native view triggers onDraw;<br/>Dart awaits callback + 1 frame"]:::unique | |
| DrawSyncHost --> CoordsHost["Dart app returns widget<br/>crop coordinates"]:::unique | |
| RenderDevicePlatformView --> DrawSyncDevice["Native view triggers onDraw;<br/>Dart awaits callback + 1 frame"]:::unique | |
| DrawSyncDevice --> CoordsDevice["Dart app returns widget<br/>crop coordinates"]:::unique | |
| CoordsHost --> CaptureHost["Driver script takes ADB screencap<br/>and crops locally"]:::unique | |
| CoordsDevice --> CaptureDevice["JUnit test runner takes UiAutomation<br/>screencap and crops natively"]:::unique | |
| CaptureHost -->|"Remaining steps occur normally"| CompareHost["Driver script asserts exact match<br/>against local filesystem"] | |
| CaptureDevice --> SendBytes["JUnit test runner sends cropped<br/>base64 bytes to Dart app over message channel"]:::unique | |
| SendBytes --> ImmediateComparison["App handles cropped bytes immediately without new frame render"]:::unique | |
| ImmediateComparison -->|"Remaining steps occur normally"| OnDeviceCompare["Dart app performs local on-device golden<br/>comparison against bundled assets"] | |
| OnDeviceCompare --> Report["Dart app returns success/fail<br/>to JUnit test runner"] | |
| classDef unique fill:#fff3cd,stroke:#ffc107,color:#856404,stroke-width:2px; | |
| ``` | |
| ### Flow Breakdown | |
| The beginning and end of the test are the same | |
| as for other tests as described in Section 3 above. | |
| * **Native UI rendering and timing**: | |
| For platform view tests (starting with `platformView` prefix), | |
| the app renders an `AndroidViewLink` or `AndroidView` widget. | |
| This causes the Android OS to render the native Kotlin UI (`NativeTextView`) | |
| via the selected composition mode | |
| ([`NativeTextView.kt`](android/app/src/main/kotlin/com/example/android_hardware_smoke_test/NativeTextView.kt)). | |
| Because the native UI is rendered by the OS, | |
| `addPostFrameCallback` is no longer sufficient to guarantee | |
| that all platform view pixels have been fully rendered and composited. | |
| To solve this, | |
| the native `ObservableTextView` overrides `onDraw(Canvas)` | |
| to send an `"onDraw"` method channel callback | |
| once the native view actually paints. | |
| The Dart app awaits this callback via `settleFuture` | |
| in [`goldens.dart`](lib/goldens.dart) | |
| and then waits 1 additional frame using | |
| `await WidgetsBinding.instance.endOfFrame` | |
| to ensure the platform view composition is fully submitted. | |
| * **Crop coordinates and test-script-driven screenshot capture**: | |
| * **Both Modes:** The Dart app (`_handlePlatformViewRequest` | |
| in [`goldens.dart`](lib/goldens.dart)) | |
| computes the bounding box of the `RepaintBoundary` | |
| in physical device pixels | |
| and returns these coordinates back to the test runner. | |
| * **Host-Driven Mode:** The driver script takes a full-screen screenshot | |
| of the physical device using ADB | |
| (`adb shell screencap` wrapped inside `NativeDriver.screenshot()` | |
| in [`driver_test.dart`](test_driver/driver_test.dart)). | |
| Then crops it locally to the retrieved coordinates | |
| using the Dart `image` package. | |
| * **Instrumented Mode:** The JUnit test runner takes a full-screen screenshot | |
| using `UiAutomation.takeScreenshot()` | |
| in [`FlutterActivityTest.kt: captureAndSendScreenshot`](android/app/src/androidTest/java/com/example/android_hardware_smoke_test/FlutterActivityTest.kt). | |
| Then crops the bitmap natively using `Bitmap.createBitmap`. | |
| * **Extra round trip and encoding-independent pixel comparison for Instrumented Mode**: | |
| * **Instrumented Mode:** The Dart app is where we perform the golden comparison, | |
| so the JUnit test runner sends the cropped bytes back to the Dart app | |
| by base64-encoding them and sending them over the `BasicMessageChannel`. | |
| It sets a field on the JSON message called `command` | |
| with the value `compare_golden`. | |
| When the Dart app parses this request, | |
| it performs the golden comparison immediately | |
| instead of waiting for another frame through `addPostFrameCallback`. | |
| However, the normal `NaiveLocalFileComparator` compares all image bytes. | |
| This is a problem because the different methods of capturing screenshots | |
| may produce images with PNG encoding differences. | |
| To compare pixels regardless of encoding, | |
| we use a [`PixelExactLocalFileComparator`](lib/pixel_exact_local_file_comparator.dart). | |
| An additional benefit of this custom comparator is | |
| that it supports reading golden references directly | |
| from the package's bundled assets | |
| using the `asset://` URI scheme via `rootBundle.load()`, | |
| completely avoiding the need to copy files | |
| to temporary directories on the device. | |
| After comparison, it reports success or failure | |
| in the same way as it would for other tests. | |
| * **Host-Driven Mode:** The driver doesn't need a new round trip | |
| because it performs the comparison directly | |
| in the same way as it does for image bytes returned from the Dart app. | |
| ### Why the `PixelCopy` approach was not desirable | |
| An alternative approach using `PixelCopy` was considered: | |
| * **How it worked:** The native Kotlin app targeted the `FlutterSurfaceView` | |
| directly using `PixelCopy.request(surfaceView, ...)` | |
| and then manually traversed the Android sibling view hierarchy, | |
| drawing the visible native platform view boundaries | |
| on top of the captured bitmap using a Kotlin `Canvas`. | |
| While plausible and functioning, | |
| it was **not desirable** for several reasons: | |
| 1. **Manual Compositing Replicas:** Drawing views manually onto a canvas | |
| (`child.draw(canvas)`) relies on replicating the composition steps. | |
| If the operating system or graphics drivers apply specific shader effects, | |
| blending, custom overlays, or subpixel anti-aliasing | |
| during hardware composition, the manual Kotlin reconstruction | |
| might not match what the user actually sees. | |
| 2. **Missing Real Composition Bugs:** The primary goal of this smoke test | |
| is to catch platform-specific integration | |
| and composition rendering errors in GPU drivers. | |
| If we manually draw the sibling views ourselves, | |
| we bypass the OS hardware compositor (SurfaceFlinger) entirely | |
| for the screenshot, | |
| defeating the purpose of testing the system's actual composition pipeline. | |
| 3. **Fragility:** Manually translating coordinate spaces, handling layouts, | |
| view visibility states, and Z-orders in Kotlin | |
| is highly fragile and prone to emulation/rendering bugs. | |
| By capturing the entire screen | |
| using native system compositor APIs (`adb` / `UiAutomation`) and cropping, | |
| the test asserts on the **true, final composited image** | |
| rendered by the device's GPU and system composer. | |
| ### Why the Compile-Online / Run-Offline Split was not desirable | |
| An alternative execution model was considered: | |
| * **How it worked**: The build is split into | |
| two sequential phases: compiling the test APK online | |
| (`assembleDebugAndroidTest`), followed by | |
| running the test execution completely offline | |
| (`connectedDebugAndroidTest --offline`) to | |
| insulate the emulator execution from Maven network flakes. | |
| While beneficial for network isolation, | |
| it was **not desirable** for several reasons: | |
| 1. **UTP Dynamic Resolution Design**: The Android Gradle Plugin | |
| (AGP) 8.x/9.x Unified Test Platform (UTP) runner resolves and | |
| downloads its host-side runner plugins (like | |
| `com.android.tools.utp:android-test-plugin-host-additional-test-output`) | |
| dynamically via internal detached configurations at | |
| runtime. These dependencies are not resolved during | |
| compilation, causing the offline check to fail on | |
| cold caches. | |
| 2. **Maintenance Overhead / Double Execution**: To support | |
| offline runs, we would either have to run a try-catch | |
| online test execution first (which executes the entire | |
| test suite twice, wasting CI resources) or manually | |
| pin and declare all transitive UTP dependencies in the | |
| `build.gradle.kts` file. Hardcoding UTP plugin versions | |
| is highly brittle and breaks automatically whenever the | |
| project upgrades its AGP version. | |
| 3. **Redundant Safety**: Because the online compilation phase | |
| (`assembleDebugAndroidTest`) already queries remote Maven | |
| repositories for project dependencies, running the test | |
| execution online does not introduce any new network | |
| vectors that could flake. Gradle's internal dependency | |
| caching guarantees subsequent execution remains | |
| extremely fast. | |
| By executing `connectedDebugAndroidTest` online directly in | |
| a single step, the test suite executes exactly once, | |
| remains low maintenance across AGP upgrades, and | |
| cleanly bubbles up any legitimate compiler or runner | |
| errors without silent try-catch blocks. | |
| --- | |
| ## 7. Graphics Initialization & Screenshot Retry Mechanisms | |
| To guarantee high stability against flakiness on emulators and physical hardware, the test suite implements a two-layered retry mechanism: | |
| ### A. Process & Activity-Level Retries (Graphics Context Initialization) | |
| * **Goal**: Recovers from transient EGL or graphics context negotiation errors (e.g., `Failed to choose config with EGL_SWAP_BEHAVIOR_PRESERVED` or HWUI startup warnings) that occur globally when the Flutter Engine boots up. | |
| * **Mechanism**: | |
| * **On Host (CI)**: If the `flutter drive` run fails, the host suite runner | |
| ([`run_android_hardware_smoke_tests.dart`](dev/bots/suite_runners/run_android_hardware_smoke_tests.dart)) | |
| checks the device logcat. If transient EGL errors are detected, it restarts | |
| the test run process (up to 3 attempts) to initialize a new EGL context in | |
| a fresh process. | |
| * **On Device (JUnit)**: If an attempt fails, the JUnit runner | |
| ([`FlutterActivityTest.kt`](android/app/src/androidTest/java/com/example/android_hardware_smoke_test/FlutterActivityTest.kt)) | |
| recreates the Activity scenario and retries **only** on blank screenshot | |
| failures (up to 3 attempts). It does **not** retry on EGL/graphics errors, | |
| as recreating the activity within the same application process cannot | |
| recover from process-level EGL initialization failures; these fail | |
| immediately on the device to bubble up to the host runner. Standard test | |
| failures (e.g., golden pixel mismatches) also fail immediately. | |
| ### B. Screenshot-Level Retries (Blank Platform View Capture) | |
| * **Goal**: Prevents race conditions where a screenshot is taken before native platform view compositing has fully settled, resulting in a blank/transparent frame. | |
| * **Mechanism**: | |
| * **Both Modes**: The screenshot capture loop (both native `UiAutomation` and | |
| host-side `NativeDriver`) checks the cropped image. If the cropped image is | |
| completely transparent or solid black, it retries the capture up to 3 | |
| times, sleeping 200ms between attempts. | |
| * **Short-circuiting**: If an EGL/graphics error is detected in the logcat | |
| during screenshot attempts, the loop is terminated early (bypassing | |
| remaining screenshot retries) to fail the test immediately and bubble the | |
| failure up to the host runner as quickly as possible. |