Add a skill for flake analysis (#187530) Example output for issue https://github.com/flutter/flutter/issues/185070 # Flake Analysis Report: `android_engine_vulkan_tests` This report analyzes the failures of the `android_engine_vulkan_tests` shard in the `flutter/flutter` repository across 15 recent failed builds between Build 1658 and Build 2495. --- ## Executive Summary The failures in the `android_engine_vulkan_tests` builder fall into three distinct categories: 1. **Primary Blocker (Skia Gold Image Mismatch)**: A deterministic/semi-deterministic failure where 5 different integration tests all output the **exact same** image digest (`5ba6df7acef516f7dab78587cf30ce20`). This strongly points to a frame buffer rendering or screenshot capture failure (e.g., capturing a completely blank, black, or transparent frame) on the Vulkan emulator. 2. **Infrastructure / Emulator Issues (Transient Flakes)**: Classical ADB/emulator connection issues, such as failing to uninstall/install the application APK, or failure to initialize the native driver wrapper. 3. **Timeout / Aborted Runs**: Builds that failed during the test execution without printing any specific stdout failure details before termination, indicating runner-level timeouts. --- ## Build Failure Summary Table | Build Number | Failure Category | Details / Primary Failure Symptom | |---|---|---| | **2495** | Skia Gold Mismatch | All 5 tests matched hash `5ba6df7acef516f7dab78587cf30ce20` | | **2392** | Timeout / Aborted | Shard failed without specific error logs | | **2337** | Infrastructure | `Failed to uninstall app` (Process exited abnormally) | | **2317** | Infrastructure | `Failed to uninstall app` (Process exited abnormally) | | **2313** | Skia Gold Mismatch | All 5 tests matched hash `5ba6df7acef516f7dab78587cf30ce20` | | **2283** | Infrastructure | `Failed to uninstall app` (Process exited abnormally) | | **2280** | Infrastructure | `Failed to uninstall app` (Process exited abnormally) | | **2268** | Infrastructure | `Failed to uninstall app`, `nativeDriver has not been initialized` | | **2198** | Infrastructure | `Failed to uninstall app`, `Failed to fulfill WaitFor due to remote error` | | **1965** | Skia Gold Mismatch | All 5 tests matched hash `5ba6df7acef516f7dab78587cf30ce20` | | **1857** | Timeout / Aborted | Shard failed without specific error logs | | **1813** | Skia Gold Mismatch | All 5 tests matched hash `5ba6df7acef516f7dab78587cf30ce20` | | **1810** | Timeout / Aborted | Shard failed without specific error logs | | **1691** | Skia Gold Mismatch | All 5 tests matched hash `5ba6df7acef516f7dab78587cf30ce20` | | **1658** | Skia Gold Mismatch | All 5 tests matched hash `5ba6df7acef516f7dab78587cf30ce20` | --- ## Detailed Findings ### 1. Skia Gold Image Mismatch (Primary Blocker) In builds **2495, 2313, 1965, 1813, 1691, and 1658**, the following 5 tests failed because the screenshot captured during the test did not match any approved golden image: * `android_engine_test.vulkan.external_texture_surface_texture_smiley_face` * `android_engine_test.vulkan.external_texture_surface_producer_smiley_face` * `android_engine_test.vulkan.platform_view_tap_color_change_initial` * `android_engine_test.vulkan.fluttered_rendered_blue_rectangle` * `android_engine_test.vulkan.texture_layer_hybrid_composition_platform_view.blue_orange_gradient_portrait` > [!IMPORTANT] > **Identical Hash Anomaly:** > For all 5 tests, the screenshot captured had the exact same hash: `5ba6df7acef516f7dab78587cf30ce20`. > Since these tests render completely different UI components (e.g. a blue rectangle vs. a blue/orange gradient vs. a smiley face texture), it is mathematically impossible for them to produce the same valid image digest. This indicates that the screenshot capture mechanism (or the emulator's Vulkan rendering context) returned a blank, black, or default-cleared buffer. ### 2. Infrastructure / Emulator Issues (Transient Flakes) In builds **2337, 2317, 2283, 2280, 2268, and 2198**, the tests failed due to VM/emulator communication errors: * **ADB Failures:** Logs show `Failed to uninstall app` followed by `Process exited abnormally with exit code 1`. * **Driver Failures:** In build 2268, the runner output `Local 'nativeDriver' has not been initialized` and `Failed to fulfill NativeCommand due to remote error`. * **Wait Failures:** In build 2198, the runner output `Failed to fulfill WaitFor due to remote error`. These are standard emulator health and connectivity flakes, where the test runner loses connection to the emulator device during apk installation/uninstallation or command dispatch. --- ## Recommendations & Next Steps 1. **Verify Vulkan Screenshot/Buffer Capture:** Investigate if the screenshotting code in the Flutter Driver / engine Vulkan integration tests is correctly capturing the Vulkan swapchain. The identical hash suggests a blank/empty frame buffer is being captured. 2. **Review/Triage on Flutter Gold:** Check the [Flutter Gold console](https://flutter-gold.skia.org/) for digest `5ba6df7acef516f7dab78587cf30ce20` to visually confirm if the images are blank/black. 3. **Emulator Stability Improvements:** To address the secondary infrastructure failures, consider adding retry logic for emulator ADB connection, or updating the Android emulator image version used on the LUCI bots. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Flutter is Google's SDK for crafting beautiful, fast user experiences for mobile, web, and desktop from a single codebase. Flutter works with existing code, is used by developers and organizations around the world, and is free and open source.
For release and other announcements, join the flutter-announce mailing list. Our documentation also tracks breaking changes across releases.
The Flutter tool may occasionally download resources from Google servers. By downloading or using the Flutter SDK, you agree to the Google Terms of Service: https://policies.google.com/terms
For example, when installed from GitHub (as opposed to from a prepackaged archive), the Flutter tool will download the Dart SDK from Google servers immediately when first run, as it is used to execute the flutter tool itself. This will also occur when Flutter is upgraded (e.g. by running the flutter upgrade command).
We think Flutter will help you create beautiful, fast apps, with a productive, extensible and open development model, whether you're targeting iOS or Android, web, Windows, macOS, Linux or embedding it as the UI toolkit for a platform of your choice.
We want to enable designers to deliver their full creative vision without being forced to water it down due to limitations of the underlying framework. Flutter‘s layered architecture gives you control over every pixel on the screen and its powerful compositing capabilities let you overlay and animate graphics, video, text, and controls without limitation. Flutter includes a full set of widgets that deliver pixel-perfect experiences whether you’re building for iOS (Cupertino) or other platforms (Material), along with support for customizing or creating entirely new visual components.
Flutter is fast. It's powered by hardware-accelerated 2D graphics libraries like Skia (which underpins Chrome and Android) and Impeller. We architected Flutter to support glitch-free, jank-free graphics at the native speed of your device.
Flutter code is powered by the world-class Dart programming language, which enables compilation to 32-bit and 64-bit ARM machine code for iOS and Android, JavaScript and WebAssembly for the web, as well as Intel x64 and ARM for desktop devices.
Flutter offers stateful hot reload, allowing you to make changes to your code and see the results instantly without restarting your app or losing its state.
Flutter works with any development tool (or none at all), and also includes editor plug-ins for both Visual Studio Code and IntelliJ / Android Studio. Flutter provides tens of thousands of packages to speed your development, regardless of your target platform. And accessing other native code is easy, with support for both FFI (on Android, on iOS, on macOS, and on Windows) as well as platform-specific APIs.
Flutter is a fully open-source project, and we welcome contributions. Information on how to get started can be found in our contributor guide.