Update Devicelab README with new dashboard information (#48908)
diff --git a/dev/bots/analyze.dart b/dev/bots/analyze.dart
index aa06511..0169470 100644
--- a/dev/bots/analyze.dart
+++ b/dev/bots/analyze.dart
@@ -1023,18 +1023,6 @@
Hash256(0x4C67221E25626CB2, 0x3F94E1F49D34E4CF, 0x3A9787A514924FC5, 0x9EF1E143E5BC5690),
- // HISTORICAL DEVICE LAB IMAGES
-
- // dev/devicelab/images/agent-statuses.png
- Hash256(0x0A43016C84095771, 0x1C610E1C01B1C3B2, 0x681DA0B2062C02AA, 0x31CC7FB3FDC298FD),
-
- // dev/devicelab/images/broken-test.png
- Hash256(0x319459F42967A888, 0x90B20063544D6707, 0x849E1E3447CC56A5, 0xE226C47DE34F13AD),
-
- // dev/devicelab/images/legend.png
- Hash256(0x92A98975AF5F076C, 0xE7BFAB86B0DD7A3D, 0xC45287B706D10456, 0x4E512BA3C41B2749),
-
-
// MISCELLANEOUS
// dev/bots/serviceaccount.enc
@@ -1060,7 +1048,7 @@
assert(
_grandfatheredBinaries
.expand<int>((Hash256 hash) => <int>[hash.a, hash.b, hash.c, hash.d])
- .reduce((int value, int element) => value ^ element) == 0x17138C817203D1A7 // Please do not modify this line.
+ .reduce((int value, int element) => value ^ element) == 0x606B51C908B40BFA // Please do not modify this line.
);
grandfatheredBinaries ??= _grandfatheredBinaries;
if (!Platform.isWindows) { // TODO(ianh): Port this to Windows
diff --git a/dev/devicelab/README.md b/dev/devicelab/README.md
index 1dec92c..6310f3f 100644
--- a/dev/devicelab/README.md
+++ b/dev/devicelab/README.md
@@ -1,41 +1,30 @@
# Flutter devicelab
-"Devicelab" (a.k.a. "cocoon") is a physical lab that tests Flutter on real
-Android and iOS devices.
+"Devicelab" (a.k.a. [Cocoon](https://github.com/flutter/cocoon)) is a physical
+lab that tests Flutter on real Android and iOS devices.
This package contains the code for test framework and the tests. More generally
the tests are referred to as "tasks" in the API, but since we primarily use it
for testing, this document refers to them as "tests".
-Build results are available at https://flutter-dashboard.appspot.com.
+Current statuses for the devicelab are available at
+https://flutter-dashboard.appspot.com.
-# Reading the dashboard
+# Dashboards
-## The build page
+## Build dashboard
-The build page is accessible at https://flutter-dashboard.appspot.com/build.html.
-This page reports the health of build servers, called _agents_, and the statuses
-of build tasks.
-
-### Agents
-
-A green agent is considered healthy and ready to receive new tasks to build. A
-red agent is broken and does not receive new tasks.
-
-In the example below, the dashboard shows that the `linux2` agent is broken and
-requires attention. All other agents are healthy.
-
-
+The build page is accessible at https://flutter-dashboard.appspot.com/#/build.
+This page reports the build statuses of commits to the flutter/flutter repo.
### Tasks
-The table below the agent statuses displays the statuses of build tasks. Task
-statuses are color-coded. The following statuses are available:
+Task statuses are color-coded in the following manner:
-**New task** (light blue): the task is waiting for an agent to pick it up and
+**New task** (blue): the task is waiting for an agent to pick it up and
start the build.
-**Task is running** (spinning blue): an agent is currently building the task.
+**Task is running** (blue with clock): an agent is currently building the task.
**Task succeeded** (green): an agent reported a successful completion of the
task.
@@ -45,13 +34,11 @@
**Task failed** (red): the task failed all of the attempts.
-**Task underperformed** (orange): currently not used.
+**Task is rerunning** (orange): the task is being rerun.
**Task was skipped** (transparent): the task is not scheduled for a build. This
usually happens when a task is removed from `manifest.yaml` file.
-**Task status unknown** (purple): currently not used.
-
In addition to color-coding, a task may display a question mark. This means
that the task was marked as flaky manually. The status of such task is ignored
when considering whether the build is broken or not. For example, if a flaky
@@ -59,18 +46,11 @@
status of a non-flaky task is red, all pending PRs will contain a warning about
the broken build and recommend caution when submitting.
-Legend:
-
-
-
-The example below shows that commit `e122d5d` caused a wide-spread breakage,
-which was fixed by `bdc6f10`. It also shows that Cirrus and Chrome
-Infra (left-most tasks) decided to skip building these commits. Hovering over
-a cell will pop up a tooltip containing the name of the broken task. Clicking
-on the cell will open the log file in a new browser tab (only visible to core
-contributors as of today).
-
-
+Clicking a cell will pop up an overlay with information about that task. It
+includes information such as the task name, number of attempts, run time,
+queue time, whether it is manually marked flaky, and the agent it was run on.
+It has actions to download the log, rerun the task, and view the agent on
+the agent dashboard.
## Why is a task stuck on "new task" status?
@@ -81,7 +61,19 @@
_last_ commit of a PR that's merged into the `master` branch. Chrome Infra may
skip commits when they come in too fast.
-## How the devicelab runs the tasks
+## Agent dashboard
+
+Agent statuses are available at https://flutter-dashboard.appspot.com/#/agents.
+
+A green agent is considered healthy and ready to receive new tasks to build. A
+red agent is broken and does not receive new tasks.
+
+## Performance dashboard
+
+Flutter benchmarks are available at
+https://flutter-dashboard.appspot.com/benchmarks.html.
+
+# How the devicelab runs tasks
The devicelab agents have a small script installed on them that continuously
asks the CI server for tasks to run. When the server finds a suitable task for
@@ -238,5 +230,10 @@
- `{STAGE}` is `devicelab` if you want to run on Android, or `devicelab_ios` if
you want to run on iOS.
- `{CAPABILITIES}` is an array that lists the capabilities required of
- the test agent (the computer that runs the test) to run your test. Available
- capabilities are: `has-android-device`, `has-ios-device`.
+ the test agent (the computer that runs the test) to run your test. As of writing,
+ the available capabilities are: `linux`, `linux/android`, `linux-vm`,
+`mac`, `mac/ios`, `mac/iphonexs`, `mac/ios32`, `mac-catalina/ios`,
+`mac-catalina/android`, `ios/gl-render-image`, `windows`, `windows/android`.
+
+If your test needs to run on multiple operating systems, create a separate test
+for each operating system.
diff --git a/dev/devicelab/images/agent-statuses.png b/dev/devicelab/images/agent-statuses.png
deleted file mode 100644
index ad5eba6..0000000
--- a/dev/devicelab/images/agent-statuses.png
+++ /dev/null
Binary files differ
diff --git a/dev/devicelab/images/broken-test.png b/dev/devicelab/images/broken-test.png
deleted file mode 100644
index fbde629..0000000
--- a/dev/devicelab/images/broken-test.png
+++ /dev/null
Binary files differ
diff --git a/dev/devicelab/images/legend.png b/dev/devicelab/images/legend.png
deleted file mode 100644
index cb94696..0000000
--- a/dev/devicelab/images/legend.png
+++ /dev/null
Binary files differ
