script/tool/CONTRIBUTING.md

# Tooling Overview

This document explains the high-level structure and philosophy of this
repository tooling. It is intended for developers who want to contribute to
the tooling.

## Guiding Principles

* **Dart tooling is better than shell scripts.** Non-trivial tooling should be
  written and maintained in the language the team is most familiar with. Dart
  is also safer and more testable than shell scripts.
* **Everything should be runnable locally.** It should be trivial to run the
  same tests locally as are run in CI. This makes the development cycle much
  simpler, as contributors don't need to rely on CI cycles to get feedback, and
  also makes investigating failures faster and easier.
* **Auditability is critical.** It should be trivial to determine which tests
  ran on which packages, and what the outcome of each was. When they don't run
  for a given package, it should be clear that they didn't, and why.
* **False negatives are the worst kind of failure.** In general, it's much
  better for the default to be to fail than to silently pass. If most packages
  should run a certain kind of test, then not having that test without explicit
  opt-out should fail.
* **Tooling is better at enforcing things than humans.** If a repo standard or
  best practice can be encoded in the tooling, it should be. Reviewers should
  not need to remember to actively check for every possible violation. When
  there's judgment involved, consider defaulting to failing, and having an
  `override: *` label to allow reviewers to intentionally skip a check in
  presubmit.
* **Collecting failures is better than stopping immediately.** Unless a failure
  would prevent continuing, it is better to add the failure to a list and
  continue, then report all failures. If a package has N unrelated failures,
  it's much better to report all of them in one run, rather than requiring N
  fix/re-run cycles.

## Structure

* Most commands should be subclasses of `PackageLoopingCommand`, which handles
  all of the logic of iterating over packages and summarizing the results.
  Unless a failure is systemic, failures should be returned rather than thrown
  so that the command can continue to run for the remaining packages.
  * Overriding `packageLoopingType` allows controlling which packages are
    included (e.g., top-level only vs all sub-packages).
  * Overriding `initializeRun()` allows for setup that should happen once, such
    as reading/parsing repo-level files needed for all packages.
  * Overriding `completeRun()` allows for final one-time checks based on the
    results of all package runs.
  * Overriding `shouldIgnoreFile()` allows for skipping commands when the only
    files touched cannot possibly impact the result of the command (e.g.,
    `README.md` changes for a native unit test command). `file_filters.dart` has
    several common classes of files that can be useful for filtering.
      * Keep in mind the principle that false negatives are the worst kind of
        failure, so err on the side of *not* skipping unless you're very sure
        it's safe.
* Commands that are not package-level (e.g., the license check should run on
  every file in the repository, whether part of a package or not) should instead
  subclass the more generic `PackageCommand`.
* Very fast and simple checks should be new classes in `lib/src/validators/`
  rather than new commands, and added to `ValidateCommand`. While this is
  slightly worse for looking at failures in CI, it is much easier for
  contributors (and their agents) to run one command to do most of the checks
  rather than remembering the entire suite of commands to run.

## Types of Commands

Commands generally fall into two categories: checks and developer utilities.
* Checks are intended to be run by CI as well as locally. They generally
  validate that some desired condition is met (unit tests pass, pubspecs are
  well structured, etc.) and have a pass/fail result.
* Developer utilities are not intended to be used in CI, but instead to make
  things easier for developers. These should be documented in `README.md` for
  easier discovery.

There is some overlap between these two categories. For example, `format`
is primarily a developer utility, but with the `--fail-on-change` flag
can be used as a check in CI.