Dart tests are written using the flutter_test package's API, named with the suffix _test.dart, and placed inside the test/ subdirectory of the package under test.
We support several kinds of tests:
Unit tests, e.g. using flutter_test. See below.
Unit tests that use golden-file testing, comparing pixels. See Writing a golden-file test for package:flutter.
End-to-end tests, e.g. using flutter_driver and our device lab.
Our bots run on our test and build infrastructure.
Flutter tests use the flutter_test package (source, API documentation), which provides flutter-specific extensions on top of the Dart test package.
To automatically find all files named *_test.dart inside a package's test/ subdirectory, and run them inside the headless flutter shell as a test, use the flutter test command, e.g:
cd examples/hello_worldflutter testIndividual tests can also be run directly, e.g.: flutter test lib/my_app_test.dart
You can view these tests on a device by running them directly using flutter run. For tests inside the packages/flutter directory, you will need to copy them to (or symlink to them from) the test/ directory of an actual app (e.g. the flutter gallery), since the flutter package itself is not set up to execute as an application (which is necessary to use flutter run with a test).
Unit tests run with flutter test run inside a headless flutter shell on your workstation, you won't see any UI. You can use print to generate console output or you can interact with the Dart VM via the Dart Observatory at http://localhost:8181/.
To debug tests in Observatory, use the --start-paused option to start the test in a paused state and wait for connection from a debugger. This option lets you set breakpoints before the test runs.
To run analysis and all the tests for the entire Flutter repository, the same way that Cirrus runs them, run dart dev/bots/test.dart and dart --enable-asserts dev/bots/analyze.dart.
If you've built your own flutter engine (see Setting up the Engine development environment), you can pass --local-engine to change what flutter shell flutter test uses. For example, if you built an engine in the out/host_debug_unopt directory, you can use:
flutter test \ --local-engine=host_debug_unopt \ --local-engine-host=host_debug_unopt
to run the tests in the locally built engine. Note that in this case you need to specify host_debug_unopt as both arguments.
To learn how to see how well tested the codebase is, see Test coverage for package:flutter.
See also: Flutter Test Fonts
Flutter runs a number of end-to-end tests in a device lab. The Flutter repo contains code for bootstrapping and executing these tests, in addition to the tests themselves.
The code that runs the device lab end-to-end tests can be found here:
dev/devicelab
The tests that run in the device lab can be found here:
dev/integration_tests
When a device lab test fails, it is important to be able to run the test locally to verify the problem and intended solution. To execute a device lab test locally, do the following:
dev/devicelab directory.export LANG=en_US.UTF-8.../../bin/dart bin/run.dart -t [task_name] where [task_name] is replaced by the name of the task you want to run as defined within .ci.yaml.Sometimes a device lab test fails due to engine changes that you‘ve made. In these cases, you’d like to run the impacted device lab tests locally with your local version of the engine. To do this, pass the appropriate flags to run.dart:
../../bin/dart bin/run.dart \ --local-engine-src-path=[path_to_src] \ --local-engine=[engine_build_for_your_device] \ --local-engine-host=[host_engine_build_for_your_device] \ -t [task_name]
If your local Flutter engine is in the same directory as your flutter/ directory then you can omit the --local-engine-src-path parameter because it will be resolved automatically:
../../bin/dart bin/run.dart \ --local-engine=[engine_build_for_your_device] \ --local-engine-host=[host_engine_build_for_your_device] \ -t [task_name]
The following is an example of what running the local engine command might look like:
../../bin/dart bin/run.dart \ --local-engine-src-path=/Users/myname/flutter/engine/src \ --local-engine=android_debug_unopt_x86 \ --local-engine-host=host_debug_unopt_x86 \ -t external_ui_integration_test
The above command would use the local Flutter engine located at /Users/myname/flutter/engine to execute the external_ui_integration_test test on an Android emulator, which is why the android_debug_unopt_x86 version of the engine is used.
Note that some tests may require profile mode instead of debug mode when running with local engine. Make sure to pass in the correct local engine. See Compiling the engine for more details.
See the Testing the engine wiki.