lib/web_ui/README.md - mirrors/engine - Git at Google

 # Flutter Web Engine

 This directory contains the source code for the Web Engine.

 ## Hacking on the Web Engine

 If you are setting up a workspace for the first time, start by following the
 instructions at [Setting up the Engine development environment][1]. In addition,
 it is useful to add the following to your `PATH` environment variable:

 - `ENGINE_ROOT/src/flutter/lib/web_ui/dev`, so you can run the `felt` command
   from anywhere.
 - `FLUTTER_ROOT/bin`, so you can run `dart` and `flutter` commands from
   anywhere.

 ### Using `felt`

 `felt` (stands for "Flutter Engine Local Tester") is a command-line tool that
 aims to make development in the Flutter web engine more productive and pleasant.

 To tell `felt` to do anything you call `felt SUBCOMMAND`, where `SUBCOMMAND` is
 one of the available subcommands, which can be listed by running `felt help`. To
 get help for a specific subcommand, run `felt help SUBCOMMAND`.

 #### `felt build`
 The `build` subcommand builds web engine gn/ninja targets. Targets can be
 individually specified in the command line invocation, or if none are specified,
 all web engine targets are built. Common targets are as follows:
   * `sdk` - The flutter_web_sdk itself.
   * `canvaskit` - Flutter's version of canvakit.
   * `canvaskit_chromium` - A version of canvaskit optimized for use with
     chromium-based browsers.
   * `skwasm` - Builds experimental skia wasm module renderer.
 The output of these steps is used in unit tests, and can be used with the flutter
 command via the `--local-web-sdk=wasm_release` command.

 The `build` command also accepts either the `--profile` or `--debug` flags, which
 can be used to change the build profile of the artifacts.

 ##### Examples
 Builds all web engine targets, then runs a Flutter app using it:
 ```
 felt build
 cd path/to/some/app
 flutter --local-web-sdk=wasm_release run -d chrome
 ```

 Builds only the `sdk` and the `canvaskit` targets:
 ```
 felt build sdk canvaskit
 ```

 #### `felt test`
 The `test` subcommand will compile and/or run web engine unit test suites. For
 information on how test suites are structured, see the test configuration
 [readme][2].

 By default, `felt test` compiles and runs all suites that are compatible with the
 host system. Some useful flags supported by this command:
   * Action flags which say what parts of the test pipeline to perform. More of one
     of these can be specified to run multiple actions. If none are specified, then
     *all* of these actions are performed
     * `--compile` performs compilation of the test bundles.
     * `--run` runs the unit tests
     * `--copy-artifacts` will copy build artifacts needed for the tests to run.
       * The `--profile` or `--debug` flags can be specified to copy over artifacts
         from the profile or debug build folders instead of release.
   * `--list` will list all the test suites and test bundles and exit without
     compiling or running anything.
   * `--verbose` will output some extra information that may be useful for debugging.
   * `--start-paused` will open a browser window and pause the tests before starting
     so that breakpoints can be set before starting the test suites.

 Several other flags can be passed that filter which test suites should be run:
   * `--browser` runs only the test suites that test on the browsers passed. Valid
     values for this are `chrome`, `firefox`, `safari`, or `edge`.
   * `--compiler` runs only the test suites that use a particular compiler. Valid
     values for this are `dart2js` or `dart2wasm`
   * `--renderer` runs only the test suites that use a particular renderer. Valid
     values for this are `html`, `canvakit`, or `skwasm`
   * `--suite` runs a suite by name.
   * `--bundle` runs suites that target a particular test bundle.

 Filters of different types are logically ANDed together, but multiple filter flags
 of the same type are logically ORed together.

 The `test` command will also accept a list of paths to specific test files to be
 compiled and run. If none of these paths are specified, all tests are run, otherwise
 only the tests that are specified will run.

 ##### Examples
 Runs all test suites in all compatible browsers:
 ```
 felt test
 ```
 Runs a specific test on all compatible browsers:
 ```
 felt test test/engine/util_test.dart
 ```
 Runs multiple specific tests on all compatible browsers:
 ```
 felt test test/engine/util_test.dart test/engine/alarm_clock_test.dart
 ```
 Runs only test suites that compile via dart2wasm:
 ```
 felt test --compiler dart2wasm
 ```
 Runs only test suites that run in Chrome and Safari:
 ```
 felt test --browser chrome --browser safari
 ```

 ### Optimizing local builds

 Concurrency of various build steps can be configured via environment variables:

 - `FELT_COMPILE_CONCURRENCY` specifies the number of concurrent compiler
   processes used to compile tests. Default value is 8.

 If you are a Google employee, you can use an internal instance of Goma (go/ma)
 to parallelize your ninja builds. Because Goma compiles code on remote servers,
 this option is particularly effective for building on low-powered laptops.

 ### Test browsers

 Chromium, Firefox, and Safari for iOS are version-locked using the
 [browser_lock.yaml][3] configuration file. Safari for macOS is supplied by the
 computer's operating system. Tests can be run in Edge locally, but Edge is not
 enabled on LUCI. Chromium is used as a proxy for Chrome, Edge, and other
 Chromium-based browsers.

 Changing parameters in the browser lock is effective immediately when running
 tests locally. To make changes effective on LUCI follow instructions in
 [Rolling Browsers][#rolling-browsers].

 ### Rolling browsers

 When running tests on LUCI using Chrome, LUCI uses the version of Chrome for
 Testing fetched from CIPD.

 Since the engine code and infra recipes do not live in the same repository
 there are few steps to follow in order to upgrade a browser's version.

 ### Rolling fallback fonts

 To generate new fallback font data and push the fallback fonts into a CIPD
 package for engine unit tests to consume, run the following felt command:

 ```
 cipd auth-login
 felt roll-fallback-fonts --key=<Google Fonts API key>
 ```

 You can obtain a GoogleFonts API key from here: https://developers.google.com/fonts/docs/developer_api#APIKey

 This will take the following steps:
 * Fetch a list of fonts from the Google Fonts API
 * Download each font we use for fallbacks and calculate its unicode ranges
 * Generate the `font_fallback_data.dart` file that is used in the engine
 * Push the fonts up to a CIPD package called `flutter/flutter_font_fallbacks`
 * Update the `DEPS` file in the engine to use the new version of the package

 To perform all these steps except actually uploading the package to CIPD, pass
 the `--dry-run` flag to the felt command.

 NOTE: Because this script uses `fc-config`, this roll step only actually works
 on Linux, not on macOS or Windows.

 #### Chrome for Testing

 Chrome for Testing is an independent project that gets rolled into Flutter
 manually, and as needed. Flutter consumes a pre-built Chrome for Testing build.
 The available versions of Chrome for Testing available can be found [here](https://googlechromelabs.github.io/chrome-for-testing/). To roll to a newer version:

 - Make sure you have `depot_tools` installed (if you are regularly hacking on
   the engine code, you probably do).
 - If not already authenticated with CIPD, run `cipd auth-login` and follow
   instructions (this step requires sufficient privileges; contact
   #hackers-infra-🌡 on [Flutter's Discord server](https://github.com/flutter/flutter/wiki/Chat)).
 - Edit `dev/browser_lock.yaml` and update the following values under `chrome`:
   - Set `version` to the full four part version number of the build of Chrome
     for Testing you want to roll (for example, `118.0.5993.70`)
 - Run `dart dev/browser_roller.dart` and make sure it completes successfully.
   The script uploads the specified versions of Chromium (and Chromedriver) to the
   right locations in CIPD: [Chrome](https://chrome-infra-packages.appspot.com/p/flutter_internal/browsers/chrome),
   [Chromedriver](https://chrome-infra-packages.appspot.com/p/flutter_internal/browser-drivers/chrome).
 - Send a pull request containing the above file changes. Newer versions of Chromium
   might break some tests or Goldens. Get those fixed too!

 If you have questions, contact the Flutter Web team on Flutter Discord on the
 \#hackers-web-🌍 channel.

 #### Firefox

 We test with Firefox on LUCI in the Linux Web Engine builder. The process for
 rolling Firefox is even easier than Chromium. Simply update `browser_lock.yaml`
 with the latest version of Firefox, and run `browser_roller.dart`.

 #### .ci.yaml

 After rolling Chrome and/or Firefox, also update the CI dependencies in
 `.ci.yaml` to make use of the new versions. The lines look like

 ```yaml
       dependencies: >-
         [
           {"dependency": "chrome_and_driver", "version": "version:107.0"},
           {"dependency": "firefox", "version": "version:83.0"},
           {"dependency": "goldctl", "version": "git_revision:720a542f6fe4f92922c3b8f0fdcc4d2ac6bb83cd"}
         ]
 ```

 ##### **browser_roller.dart**

 The script has the following command-line options:

 - `--dry-run` - The script will stop before uploading artifacts to CIPD. The location of the data will be reported at the end of the script, if the script finishes successfullyThe output of the script will be visible in /tmp/browser-roll-RANDOM_STRING
 - `--verbose` - Greatly increase the amount of information printed to `stdout` by the script.

 > Try the following!
 >
 > ```bash
 > dart ./dev/browser_roller.dart --dry-run --verbose
 > ```

 #### **Other browsers / manual upload**

 In general, the manual process goes like this:

 1. Dowload the binaries for the new browser/driver for each operating system
    (macOS, linux, windows).
 2. Create CIPD packages for these packages (more documentation is available for
    Googlers at go/cipd-flutter-web)
 3. Update the version in this repo. Do this by changing the related fields in
    `browser_lock.yaml` file.

 Resources:

 1. Browser and driver CIPD [packages][4] (requires special access; ping
    hackers-infra on Discord for more information)
 2. LUCI web [recipe][5]
 3. More general reading on CIPD packages [link][6]

 ### Configuration files

 `browser_lock.yaml` contains the version of browsers we use to test Flutter for
 web. Versions are not automatically updated whenever a new release is available.
 Instead, we update this file manually once in a while.

 `canvaskit_lock.yaml` locks the version of CanvasKit for tests and production
 use.

 ## Building CanvasKit

 To build CanvasKit locally, you must first set up your gclient config to
 activate the Emscripten SDK, which is the toolchain used to build CanvasKit.
 To do this, replace the contents of your .gclient file at the root of the
 project (i.e. in the parent directory of the `src` directory) with:

 ```
 solutions = [
   {
     "managed": False,
     "name": "src/flutter",
     "url": "git@github.com:<your_username_here>/engine.git",
     "custom_deps": {},
     "deps_file": "DEPS",
     "safesync_url": "",
     "custom_vars": {
       "download_emsdk": True,
     },
   },
 ]
 ```

 Now run `gclient sync` and it should pull in the Emscripten SDK and activate it.

 To build CanvasKit with `felt`, run:

 ```
 felt build --build-canvaskit
 ```

 This will build CanvasKit in `out/wasm_debug`. If you now run

 ```
 felt test
 ```

 it will detect that you have built CanvasKit and use that instead of the one
 from CIPD to run the tests against.

 ### Upgrading the Emscripten SDK for the CanvasKit build

 The version of the Emscripten SDK should be kept up to date with the version
 used in the Skia build. That version can be found in
 `third_party/skia/bin/activate-emsdk`. It will probably also be necessary to
 roll the dependency on `third_party/emsdk` in DEPS to the same version as in
 `third_party/skia/DEPS`.

 Once you know the version for the Emscripten SDK, change the line in
 `tools/activate_emsdk.py` which defines `EMSDK_VERSION` to match Skia.


 [1]: https://github.com/flutter/flutter/wiki/Setting-up-the-Engine-development-environment
 [2]: https://github.com/flutter/flutter/blob/main/lib/web_ui/test/README
 [3]: https://github.com/flutter/engine/blob/main/lib/web_ui/dev/browser_lock.yaml
 [4]: https://chrome-infra-packages.appspot.com/p/flutter_internal
 [5]: https://cs.opensource.google/flutter/recipes/+/master:recipes/engine/web_engine.py
 [6]: https://chromium.googlesource.com/chromium/src.git/+/main/docs/cipd_and_3pp.md#What-is-CIPD

 ## Unicode properties

 We pull the unicode properties we need from `third_party/web_unicode`. See `third_party/web_unicode/README.md` for more details on how we generate Dart code from unicode properties.
	# Flutter Web Engine

	This directory contains the source code for the Web Engine.

	## Hacking on the Web Engine

	If you are setting up a workspace for the first time, start by following the
	instructions at [Setting up the Engine development environment][1]. In addition,
	it is useful to add the following to your `PATH` environment variable:

	- `ENGINE_ROOT/src/flutter/lib/web_ui/dev`, so you can run the `felt` command
	from anywhere.
	- `FLUTTER_ROOT/bin`, so you can run `dart` and `flutter` commands from
	anywhere.

	### Using `felt`

	`felt` (stands for "Flutter Engine Local Tester") is a command-line tool that
	aims to make development in the Flutter web engine more productive and pleasant.

	To tell `felt` to do anything you call `felt SUBCOMMAND`, where `SUBCOMMAND` is
	one of the available subcommands, which can be listed by running `felt help`. To
	get help for a specific subcommand, run `felt help SUBCOMMAND`.

	#### `felt build`
	The `build` subcommand builds web engine gn/ninja targets. Targets can be
	individually specified in the command line invocation, or if none are specified,
	all web engine targets are built. Common targets are as follows:
	* `sdk` - The flutter_web_sdk itself.
	* `canvaskit` - Flutter's version of canvakit.
	* `canvaskit_chromium` - A version of canvaskit optimized for use with
	chromium-based browsers.
	* `skwasm` - Builds experimental skia wasm module renderer.
	The output of these steps is used in unit tests, and can be used with the flutter
	command via the `--local-web-sdk=wasm_release` command.

	The `build` command also accepts either the `--profile` or `--debug` flags, which
	can be used to change the build profile of the artifacts.

	##### Examples
	Builds all web engine targets, then runs a Flutter app using it:
	```
	felt build
	cd path/to/some/app
	flutter --local-web-sdk=wasm_release run -d chrome
	```

	Builds only the `sdk` and the `canvaskit` targets:
	```
	felt build sdk canvaskit
	```

	#### `felt test`
	The `test` subcommand will compile and/or run web engine unit test suites. For
	information on how test suites are structured, see the test configuration
	[readme][2].

	By default, `felt test` compiles and runs all suites that are compatible with the
	host system. Some useful flags supported by this command:
	* Action flags which say what parts of the test pipeline to perform. More of one
	of these can be specified to run multiple actions. If none are specified, then
	all of these actions are performed
	* `--compile` performs compilation of the test bundles.
	* `--run` runs the unit tests
	* `--copy-artifacts` will copy build artifacts needed for the tests to run.
	* The `--profile` or `--debug` flags can be specified to copy over artifacts
	from the profile or debug build folders instead of release.
	* `--list` will list all the test suites and test bundles and exit without
	compiling or running anything.
	* `--verbose` will output some extra information that may be useful for debugging.
	* `--start-paused` will open a browser window and pause the tests before starting
	so that breakpoints can be set before starting the test suites.

	Several other flags can be passed that filter which test suites should be run:
	* `--browser` runs only the test suites that test on the browsers passed. Valid
	values for this are `chrome`, `firefox`, `safari`, or `edge`.
	* `--compiler` runs only the test suites that use a particular compiler. Valid
	values for this are `dart2js` or `dart2wasm`
	* `--renderer` runs only the test suites that use a particular renderer. Valid
	values for this are `html`, `canvakit`, or `skwasm`
	* `--suite` runs a suite by name.
	* `--bundle` runs suites that target a particular test bundle.

	Filters of different types are logically ANDed together, but multiple filter flags
	of the same type are logically ORed together.

	The `test` command will also accept a list of paths to specific test files to be
	compiled and run. If none of these paths are specified, all tests are run, otherwise
	only the tests that are specified will run.

	##### Examples
	Runs all test suites in all compatible browsers:
	```
	felt test
	```
	Runs a specific test on all compatible browsers:
	```
	felt test test/engine/util_test.dart
	```
	Runs multiple specific tests on all compatible browsers:
	```
	felt test test/engine/util_test.dart test/engine/alarm_clock_test.dart
	```
	Runs only test suites that compile via dart2wasm:
	```
	felt test --compiler dart2wasm
	```
	Runs only test suites that run in Chrome and Safari:
	```
	felt test --browser chrome --browser safari
	```

	### Optimizing local builds

	Concurrency of various build steps can be configured via environment variables:

	- `FELT_COMPILE_CONCURRENCY` specifies the number of concurrent compiler
	processes used to compile tests. Default value is 8.

	If you are a Google employee, you can use an internal instance of Goma (go/ma)
	to parallelize your ninja builds. Because Goma compiles code on remote servers,
	this option is particularly effective for building on low-powered laptops.

	### Test browsers

	Chromium, Firefox, and Safari for iOS are version-locked using the
	[browser_lock.yaml][3] configuration file. Safari for macOS is supplied by the
	computer's operating system. Tests can be run in Edge locally, but Edge is not
	enabled on LUCI. Chromium is used as a proxy for Chrome, Edge, and other
	Chromium-based browsers.

	Changing parameters in the browser lock is effective immediately when running
	tests locally. To make changes effective on LUCI follow instructions in
	[Rolling Browsers][#rolling-browsers].

	### Rolling browsers

	When running tests on LUCI using Chrome, LUCI uses the version of Chrome for
	Testing fetched from CIPD.

	Since the engine code and infra recipes do not live in the same repository
	there are few steps to follow in order to upgrade a browser's version.

	### Rolling fallback fonts

	To generate new fallback font data and push the fallback fonts into a CIPD
	package for engine unit tests to consume, run the following felt command:

	```
	cipd auth-login
	felt roll-fallback-fonts --key=<Google Fonts API key>
	```

	You can obtain a GoogleFonts API key from here: https://developers.google.com/fonts/docs/developer_api#APIKey

	This will take the following steps:
	* Fetch a list of fonts from the Google Fonts API
	* Download each font we use for fallbacks and calculate its unicode ranges
	* Generate the `font_fallback_data.dart` file that is used in the engine
	* Push the fonts up to a CIPD package called `flutter/flutter_font_fallbacks`
	* Update the `DEPS` file in the engine to use the new version of the package

	To perform all these steps except actually uploading the package to CIPD, pass
	the `--dry-run` flag to the felt command.

	NOTE: Because this script uses `fc-config`, this roll step only actually works
	on Linux, not on macOS or Windows.

	#### Chrome for Testing

	Chrome for Testing is an independent project that gets rolled into Flutter
	manually, and as needed. Flutter consumes a pre-built Chrome for Testing build.
	The available versions of Chrome for Testing available can be found [here](https://googlechromelabs.github.io/chrome-for-testing/). To roll to a newer version:

	- Make sure you have `depot_tools` installed (if you are regularly hacking on
	the engine code, you probably do).
	- If not already authenticated with CIPD, run `cipd auth-login` and follow
	instructions (this step requires sufficient privileges; contact
	#hackers-infra-🌡 on [Flutter's Discord server](https://github.com/flutter/flutter/wiki/Chat)).
	- Edit `dev/browser_lock.yaml` and update the following values under `chrome`:
	- Set `version` to the full four part version number of the build of Chrome
	for Testing you want to roll (for example, `118.0.5993.70`)
	- Run `dart dev/browser_roller.dart` and make sure it completes successfully.
	The script uploads the specified versions of Chromium (and Chromedriver) to the
	right locations in CIPD: [Chrome](https://chrome-infra-packages.appspot.com/p/flutter_internal/browsers/chrome),
	[Chromedriver](https://chrome-infra-packages.appspot.com/p/flutter_internal/browser-drivers/chrome).
	- Send a pull request containing the above file changes. Newer versions of Chromium
	might break some tests or Goldens. Get those fixed too!

	If you have questions, contact the Flutter Web team on Flutter Discord on the
	\#hackers-web-🌍 channel.

	#### Firefox

	We test with Firefox on LUCI in the Linux Web Engine builder. The process for
	rolling Firefox is even easier than Chromium. Simply update `browser_lock.yaml`
	with the latest version of Firefox, and run `browser_roller.dart`.

	#### .ci.yaml

	After rolling Chrome and/or Firefox, also update the CI dependencies in
	`.ci.yaml` to make use of the new versions. The lines look like

	```yaml
	dependencies: >-
	[
	{"dependency": "chrome_and_driver", "version": "version:107.0"},
	{"dependency": "firefox", "version": "version:83.0"},
	{"dependency": "goldctl", "version": "git_revision:720a542f6fe4f92922c3b8f0fdcc4d2ac6bb83cd"}
	]
	```

	##### browser_roller.dart

	The script has the following command-line options:

	- `--dry-run` - The script will stop before uploading artifacts to CIPD. The location of the data will be reported at the end of the script, if the script finishes successfullyThe output of the script will be visible in /tmp/browser-roll-RANDOM_STRING
	- `--verbose` - Greatly increase the amount of information printed to `stdout` by the script.

	> Try the following!
	>
	> ```bash
	> dart ./dev/browser_roller.dart --dry-run --verbose
	> ```

	#### Other browsers / manual upload

	In general, the manual process goes like this:

	1. Dowload the binaries for the new browser/driver for each operating system
	(macOS, linux, windows).
	2. Create CIPD packages for these packages (more documentation is available for
	Googlers at go/cipd-flutter-web)
	3. Update the version in this repo. Do this by changing the related fields in
	`browser_lock.yaml` file.

	Resources:

	1. Browser and driver CIPD [packages][4] (requires special access; ping
	hackers-infra on Discord for more information)
	2. LUCI web [recipe][5]
	3. More general reading on CIPD packages [link][6]

	### Configuration files

	`browser_lock.yaml` contains the version of browsers we use to test Flutter for
	web. Versions are not automatically updated whenever a new release is available.
	Instead, we update this file manually once in a while.

	`canvaskit_lock.yaml` locks the version of CanvasKit for tests and production
	use.

	## Building CanvasKit

	To build CanvasKit locally, you must first set up your gclient config to
	activate the Emscripten SDK, which is the toolchain used to build CanvasKit.
	To do this, replace the contents of your .gclient file at the root of the
	project (i.e. in the parent directory of the `src` directory) with:

	```
	solutions = [
	{
	"managed": False,
	"name": "src/flutter",
	"url": "git@github.com:<your_username_here>/engine.git",
	"custom_deps": {},
	"deps_file": "DEPS",
	"safesync_url": "",
	"custom_vars": {
	"download_emsdk": True,
	},
	},
	]
	```

	Now run `gclient sync` and it should pull in the Emscripten SDK and activate it.

	To build CanvasKit with `felt`, run:

	```
	felt build --build-canvaskit
	```

	This will build CanvasKit in `out/wasm_debug`. If you now run

	```
	felt test
	```

	it will detect that you have built CanvasKit and use that instead of the one
	from CIPD to run the tests against.

	### Upgrading the Emscripten SDK for the CanvasKit build

	The version of the Emscripten SDK should be kept up to date with the version
	used in the Skia build. That version can be found in
	`third_party/skia/bin/activate-emsdk`. It will probably also be necessary to
	roll the dependency on `third_party/emsdk` in DEPS to the same version as in
	`third_party/skia/DEPS`.

	Once you know the version for the Emscripten SDK, change the line in
	`tools/activate_emsdk.py` which defines `EMSDK_VERSION` to match Skia.



	[1]: https://github.com/flutter/flutter/wiki/Setting-up-the-Engine-development-environment
	[2]: https://github.com/flutter/flutter/blob/main/lib/web_ui/test/README
	[3]: https://github.com/flutter/engine/blob/main/lib/web_ui/dev/browser_lock.yaml
	[4]: https://chrome-infra-packages.appspot.com/p/flutter_internal
	[5]: https://cs.opensource.google/flutter/recipes/+/master:recipes/engine/web_engine.py
	[6]: https://chromium.googlesource.com/chromium/src.git/+/main/docs/cipd_and_3pp.md#What-is-CIPD

	## Unicode properties

	We pull the unicode properties we need from `third_party/web_unicode`. See `third_party/web_unicode/README.md` for more details on how we generate Dart code from unicode properties.