Upgrade android_hardware_smoke_test CI to run instrumented tests (#189390) This PR adds native Android instrumented tests to the `android_hardware_smoke_test` CI suite in parallel to existing driver tests (Part of https://github.com/flutter/flutter/issues/187538). Adds two new shards temporarily, both with `bringup: true`: * `android_hardware_smoke_vulkan_instrumented_tests` * `android_hardware_smoke_opengles_instrumented_tests` These shards run the driver tests first, similar to existing shards, then subsequently run the instrumented tests. They do this via a new optional `runInstrumented` param in `run_android_hardware_smoke_tests.dart`. We'll remove the parameter and put this behavior in the existing shards after it proves stable. ## Why do we need to run the instrumented tests sequentially after the driver tests? I generally prefer to avoid doing multiple things sequentially in an integration test. It tends to introduce flakiness. So for this suite, why can't we have one shard for driver tests and a separate one for instrumented tests? The driver tests use a skia gold comparator to compare result images to skia gold at runtime. However, no such comparator is available for instrumented tests. `goldctl` isn't designed to run on android. So we need to get the goldens somehow at APK build time. In this PR, the integration tests run in two sequential passes. We first execute a host-driven driver test to save reference screenshots, and then run native JUnit tests to compare against those local files. This is the same way we will generate instrumented test APKs to be delivered to OEMs. I considered optimizing this by downloading the latest approved goldens directly from the Skia Gold server. It turns out it's possible to calculate the public URLs based on the test name and some other parameters. So we could theoretically write a script which cheaply fetches all the goldens for the suite. Then we could configure a shard which runs that script instead of the driver tests, achieving an independent shard for instrumented tests only. However, this downloader approach is incompatible with presubmit. Any PR that introduces a change to the goldens would fail because the script would only return the latest approved goldens. Even after a developer approved the change on the dashboard, subsequent presubmit runs would still download the old goldens, deadlocking the PR checks. Technically that's not a problem for post-submit runs, but I'd rather keep presubmit and postsubmit identical instead of optimizing only postsubmit with something like this. Running the driver pass first, as we do in this PR, avoids the problem by dynamically capturing reference screenshots generated by the PR's own code. This keeps the skew at zero and allows tests to pass once approved. Also, I'm not that comfortable relying on calculating URLs, since there would be no guarantee that the URL scheme will remain stable. The URL scheme hasn't changed since 2020, but compared to relying only on goldctl, it just feels more fragile to me. That being said, I've implemented the download script in a personal agent skill instead, where I can use it at my own risk, and it isn't weight bearing in any way. ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
Flutter is Google's SDK for crafting beautiful, fast user experiences for mobile, web, and desktop from a single codebase. Flutter works with existing code, is used by developers and organizations around the world, and is free and open source.
For release and other announcements, join the flutter-announce mailing list. Our documentation also tracks breaking changes across releases.
The Flutter tool may occasionally download resources from Google servers. By downloading or using the Flutter SDK, you agree to the Google Terms of Service: https://policies.google.com/terms
For example, when installed from GitHub (as opposed to from a prepackaged archive), the Flutter tool will download the Dart SDK from Google servers immediately when first run, as it is used to execute the flutter tool itself. This will also occur when Flutter is upgraded (e.g. by running the flutter upgrade command).
We think Flutter will help you create beautiful, fast apps, with a productive, extensible and open development model, whether you're targeting iOS or Android, web, Windows, macOS, Linux or embedding it as the UI toolkit for a platform of your choice.
We want to enable designers to deliver their full creative vision without being forced to water it down due to limitations of the underlying framework. Flutter‘s layered architecture gives you control over every pixel on the screen and its powerful compositing capabilities let you overlay and animate graphics, video, text, and controls without limitation. Flutter includes a full set of widgets that deliver pixel-perfect experiences whether you’re building for iOS (Cupertino) or other platforms (Material), along with support for customizing or creating entirely new visual components.
Flutter is fast. It's powered by hardware-accelerated 2D graphics libraries like Skia (which underpins Chrome and Android) and Impeller. We architected Flutter to support glitch-free, jank-free graphics at the native speed of your device.
Flutter code is powered by the world-class Dart programming language, which enables compilation to 32-bit and 64-bit ARM machine code for iOS and Android, JavaScript and WebAssembly for the web, as well as Intel x64 and ARM for desktop devices.
Flutter offers stateful hot reload, allowing you to make changes to your code and see the results instantly without restarting your app or losing its state.
Flutter works with any development tool (or none at all), and also includes editor plug-ins for both Visual Studio Code and IntelliJ / Android Studio. Flutter provides tens of thousands of packages to speed your development, regardless of your target platform. And accessing other native code is easy, with support for both FFI (on Android, on iOS, on macOS, and on Windows) as well as platform-specific APIs.
Flutter is a fully open-source project, and we welcome contributions. Information on how to get started can be found in our contributor guide.