)]}'
{
  "commit": "aa6bb3d9477651a74138454bd64225a9f6d4cb58",
  "tree": "6c64bb17af190b76197b054964eb79455c7ef4a1",
  "parents": [
    "7bef8f54e62e79db15acc2eddab6c6ef8bc9d8d1"
  ],
  "author": {
    "name": "Andy Wolff",
    "email": "awolff@google.com",
    "time": "Fri Jul 17 10:01:56 2026 -0700"
  },
  "committer": {
    "name": "GitHub",
    "email": "noreply@github.com",
    "time": "Fri Jul 17 17:01:56 2026 +0000"
  },
  "message": "Upgrade android_hardware_smoke_test CI to run instrumented tests (#189390)\n\nThis PR adds native Android instrumented tests to the\n`android_hardware_smoke_test` CI suite in parallel to existing driver\ntests (Part of https://github.com/flutter/flutter/issues/187538).\n\nAdds two new shards temporarily, both with `bringup: true`:\n* `android_hardware_smoke_vulkan_instrumented_tests` \n* `android_hardware_smoke_opengles_instrumented_tests`\n\nThese shards run the driver tests first, similar to existing shards,\nthen subsequently run the instrumented tests. They do this via a new\noptional `runInstrumented` param in\n`run_android_hardware_smoke_tests.dart`.\n\nWe\u0027ll remove the parameter and put this behavior in the existing shards\nafter it proves stable.\n\n## Why do we need to run the instrumented tests sequentially after the\ndriver tests?\n\nI generally prefer to avoid doing multiple things sequentially in an\nintegration test. It tends to introduce flakiness. So for this suite,\nwhy can\u0027t we have one shard for driver tests and a separate one for\ninstrumented tests?\n\nThe driver tests use a skia gold comparator to compare result images to\nskia gold at runtime. However, no such comparator is available for\ninstrumented tests. `goldctl` isn\u0027t designed to run on android. So we\nneed to get the goldens somehow at APK build time.\n\nIn this PR, the integration tests run in two sequential passes. We first\nexecute a host-driven driver test to save reference screenshots, and\nthen run native JUnit tests to compare against those local files. This\nis the same way we will generate instrumented test APKs to be delivered\nto OEMs. I considered optimizing this by downloading the latest approved\ngoldens directly from the Skia Gold server. It turns out it\u0027s possible\nto calculate the public URLs based on the test name and some other\nparameters. So we could theoretically write a script which cheaply\nfetches all the goldens for the suite. Then we could configure a shard\nwhich runs that script instead of the driver tests, achieving an\nindependent shard for instrumented tests only.\n\nHowever, this downloader approach is incompatible with presubmit. Any PR\nthat introduces a change to the goldens would fail because the script\nwould only return the latest approved goldens. Even after a developer\napproved the change on the dashboard, subsequent presubmit runs would\nstill download the old goldens, deadlocking the PR checks. Technically\nthat\u0027s not a problem for post-submit runs, but I\u0027d rather keep presubmit\nand postsubmit identical instead of optimizing only postsubmit with\nsomething like this.\n\nRunning the driver pass first, as we do in this PR, avoids the problem\nby dynamically capturing reference screenshots generated by the PR\u0027s own\ncode. This keeps the skew at zero and allows tests to pass once\napproved.\n\nAlso, I\u0027m not that comfortable relying on calculating URLs, since there\nwould be no guarantee that the URL scheme will remain stable. The URL\nscheme hasn\u0027t changed since 2020, but compared to relying only on\ngoldctl, it just feels more fragile to me. That being said, I\u0027ve\nimplemented the download script in a personal agent skill instead, where\nI can use it at my own risk, and it isn\u0027t weight bearing in any way.\n\n## Pre-launch Checklist\n\n- [x] I read the [Contributor Guide] and followed the process outlined\nthere for submitting PRs.\n- [x] I read the [AI contribution guidelines] and understand my\nresponsibilities, or I am not using AI tools.\n- [x] I read the [Tree Hygiene] wiki page, which explains my\nresponsibilities.\n- [x] I read and followed the [Flutter Style Guide], including [Features\nwe expect every widget to implement].\n- [x] I signed the [CLA].\n- [x] I listed at least one issue that this PR fixes in the description\nabove.\n- [x] I updated/added relevant documentation (doc comments with `///`).\n- [x] I added new tests to check the change I am making, or this PR is\n[test-exempt].\n- [x] I followed the [breaking change policy] and added [Data Driven\nFixes] where supported.\n- [x] All existing and new tests are passing.\n\n\u003c!-- Links --\u003e\n[Contributor Guide]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview\n[AI contribution guidelines]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines\n[Tree Hygiene]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md\n[test-exempt]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests\n[Flutter Style Guide]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md\n[Features we expect every widget to implement]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement\n[CLA]: https://cla.developers.google.com/\n[flutter/tests]: https://github.com/flutter/tests\n[breaking change policy]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes\n[Discord]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md\n[Data Driven Fixes]:\nhttps://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "ed1857b0c33470ce2154eeed1cf78d64bb136515",
      "old_mode": 33188,
      "old_path": ".ci.yaml",
      "new_id": "a737c5a0b8dbfb875c5f09e8b47b94aeaf701e38",
      "new_mode": 33188,
      "new_path": ".ci.yaml"
    },
    {
      "type": "modify",
      "old_id": "1cf028db17b99eaea8174145f299e619ec3d9c38",
      "old_mode": 33188,
      "old_path": "TESTOWNERS",
      "new_id": "f84904777efe97be41d3a9dab08588bee5641d67",
      "new_mode": 33188,
      "new_path": "TESTOWNERS"
    },
    {
      "type": "modify",
      "old_id": "a1d05cf8a37c32b77b5d40e9765d269095d68f82",
      "old_mode": 33188,
      "old_path": "dev/bots/suite_runners/run_android_hardware_smoke_tests.dart",
      "new_id": "24056c7aefa23f90955f589e3014f848e90a9117",
      "new_mode": 33188,
      "new_path": "dev/bots/suite_runners/run_android_hardware_smoke_tests.dart"
    },
    {
      "type": "modify",
      "old_id": "511ecd3cfd44df1be410a70873f8e76334c76c21",
      "old_mode": 33188,
      "old_path": "dev/bots/test.dart",
      "new_id": "8ab0284582fcbce8c8448a04311c19aa5361cb66",
      "new_mode": 33188,
      "new_path": "dev/bots/test.dart"
    },
    {
      "type": "modify",
      "old_id": "ce6beee5e922b26ada88442dbe0a98413f6c86c5",
      "old_mode": 33188,
      "old_path": "dev/integration_tests/android_hardware_smoke_test/README.md",
      "new_id": "ecc66472e7e8297a48e7c2f7b8bb26f3d47a1a97",
      "new_mode": 33188,
      "new_path": "dev/integration_tests/android_hardware_smoke_test/README.md"
    }
  ]
}