Triage Bot 1.0 (#3036)
This is the code that's been running for a few weeks now. It's time to get it onto Appengine.
diff --git a/CODEOWNERS b/CODEOWNERS
index b38f42c..8bfc947 100644
--- a/CODEOWNERS
+++ b/CODEOWNERS
@@ -31,8 +31,9 @@
app_dart/lib/src/request_handlers/update_task_status.dart @keyonghan
app_dart/lib/src/request_handlers/vacuum_github_commits.dart @keyonghan
-## auto_submit app
+## apps
auto_submit @ricardoamador
+triage_bot @Hixie
## cipd packages
cipd_packages/codesign/** @XilaiZhang
diff --git a/triage_bot/.gitignore b/triage_bot/.gitignore
new file mode 100644
index 0000000..9336dae
--- /dev/null
+++ b/triage_bot/.gitignore
@@ -0,0 +1,2 @@
+.dart_tool/
+secrets/
\ No newline at end of file
diff --git a/triage_bot/Dockerfile b/triage_bot/Dockerfile
new file mode 100644
index 0000000..4618011
--- /dev/null
+++ b/triage_bot/Dockerfile
@@ -0,0 +1,16 @@
+# Copyright 2022 The Flutter Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+# Dart Docker official images can be found here: https://hub.docker.com/_/dart
+FROM dart:beta@sha256:d19a5c7453c9ff7f977dfc6956a1a61042386c5572a4dcc249c5b3705b10d41f
+
+WORKDIR /app
+
+# Copy app source code (except anything in .dockerignore).
+COPY . .
+RUN dart pub get
+
+# Start server.
+EXPOSE 8213
+CMD ["/usr/lib/dart/bin/dart", "/app/bin/triage_bot.dart"]
diff --git a/triage_bot/README.md b/triage_bot/README.md
new file mode 100644
index 0000000..4d536eb
--- /dev/null
+++ b/triage_bot/README.md
@@ -0,0 +1,142 @@
+# Triage Automation Bot
+
+This bot implements the automations originally proposed in
+[https://docs.google.com/document/d/1RBvsolWL9nUkcEFhPZV4b-tstVUDHfHKzs-uQ3HiX-w/edit#heading=h.34a91yqebirw](Flutter
+project proposed new triage processes for 2023).
+
+## Implemented automations
+
+The core logic is in `lib/engine.dart`.
+
+There are four components:
+
+- the GitHub webhook
+- background updates
+- the cleanup process
+- the tidy process
+
+In addition, there is an internal model that tracks the state of the
+project. Most notably, it tracks various attributes of the project's
+open issues. This is stored in memory as a Hash table and takes about
+200MB of RAM for about 10,000 issues. It is written to disk
+periodically, taking about 2MB of disk. This is used on startup to
+warm the cache, so that brief interruptions in service do not require
+the multiple days of probing GitHub APIs to fetch all the data.
+
+You can see the current state of this data model by fetching the
+`/debug` HTTP endpoint from a browser.
+
+## The GitHub webhook
+
+The triage bot listens to notifications from GitHub. When it receives
+them, it first updates the issue model accordingly, then (if
+appropriate) sends a message Discord.
+
+The following updates are among those that update the model:
+
+ - new issues, closing issues, reopening issues
+ - issue comments
+ - changes to the assignee
+ - changes to labels
+ - updates to who is a team member
+ - changes to an issue's lock state
+
+Most of these updates are just a matter of tracking whether the issue
+is still open, and whether the update came from a team member, so that
+we can track how long it's been since an issue was updated.
+
+The following updates are among those repeated on Discord:
+
+ - when someone stars a repo
+ - when someone creates a new label
+ - when issues are opened, closed, or reopened
+ - when PRs are filed or closed
+ - when comments are left on issues and PRs
+ - when issues are locked or unlocked
+ - when the wiki is updated
+
+The channels used vary based on the message. Most of them go to
+`#github2`, some go to `#hidden-chat`.
+
+## Background updates
+
+Every few seconds (`backgroundUpdatePeriod`), the bot attempts to
+fetch an issue from GitHub. If the issue is open (and not a pull
+request), the model is updated with the information obtained from
+GitHub.
+
+## The cleanup process
+
+Issues that have recently been examined (either for the webhook or the
+background updates) are added to a cleanup queue.
+
+Roughly every minute (`cleanupUpdatePeriod`), all the issues in the
+queue that were last touched more than 45 minutes ago
+(`cleanupUpdateDelay`) are checked to see if they need cleaning up.
+Cleaning up in this context means making automated changes to the
+issue that enforce invariants. Specifically:
+
+ - If an issue has multiple priorities, all but the highest one are
+ removed.
+ - Issues with multiple `team-*` labels lose all of them (sending the
+ issue back to front-line triage).
+ - `fyi-*` labels are removed if they're redundant with a `team-*`
+ label or acknowledged by a `triaged-*` label.
+ - `triaged-*` labels that don't have a corresponding `team-*` label
+ are removed as redundant.
+ - Issues that have a `triaged-*` label but no priority have their
+ `triaged-*` label removed. This only happens once every two days or
+ so (`refeedDelay`) per team; if more than one issue has this
+ condition at a time, the other issues are left alone until the next
+ time the issue is examined by the background update process.
+ - The thumbs-up label is removed if the issue has been marked as
+ triaged.
+ - The "stale issue" label (the hourglass) is removed if the issue has
+ received an update from a team member since it was added.
+ - Recently re-opened issues are unlocked if necessary.
+
+## The tidy process
+
+Every few hours (`longTermTidyingPeriod`), all the known open issues
+that are _not_ pending a cleanup update are examined, and have
+invariants applied, as follows:
+
+ - If the issue is "stale" (`timeUntilStale`), i.e. is assigned or
+ marked P1 and hasn't received an update from a team member in some
+ time, it is pinged and labeled with the "stale issue" label (the
+ hourglass).
+ - If the issue doesn't receive an update even after getting pinged
+ (`timeUntilReallyStale`), the assignee is removed and the issue is
+ sent back to the team's triage meeting. This process is subject to
+ the same per-team rate-limiting (`refeedDelay`) as the removal of
+ priority labels discussed in the cleanup process section.
+ - Issues that have been locked for a while (`timeUntilUnlock`) are
+ automatically unlocked.
+ - Issues that have gained a lot of thumbs-up recently are flagged for
+ additional triage.
+
+## The self-test issue
+
+Every now and then (`selfTestPeriod`), an issue is filed to test the
+triage process itself. After some additional time (`selfTestWindow`),
+if the issue is open, it is assigned to the critical triage meeting
+for further follow-up.
+
+## Secrets
+
+The following files need to exist in the `secrets` subdirectory to run
+this locally:
+
+* `discord.appid`: The Discord app ID.
+* `discord.token`: The Discord authentication token.
+* `github.app.id`: The GitHub app ID.
+* `github.app.key.pem`: The GitHub application private key.
+* `github.installation.id`: The GitHub application installation ID.
+* `github.webhook.secret`: The GitHub webhook secret password.
+* `server.cert.pem`: The TLS certificate.
+* `server.intermediates.pem`: The TLS intermediate certificates, if
+ any, or else an empty file.
+* `server.key.pem`: The TLS private key.
+
+Alternatively, these files can be provided as secrets in Google
+Cloud's secrets manager.
diff --git a/triage_bot/analysis_options.yaml b/triage_bot/analysis_options.yaml
new file mode 100644
index 0000000..33f9ba4
--- /dev/null
+++ b/triage_bot/analysis_options.yaml
@@ -0,0 +1,211 @@
+linter:
+ rules:
+ - always_declare_return_types
+ - always_put_control_body_on_new_line
+ # - always_put_required_named_parameters_first
+ - always_specify_types
+ # - always_use_package_imports
+ - annotate_overrides
+ # - avoid_annotating_with_dynamic
+ - avoid_bool_literals_in_conditional_expressions
+ # - avoid_catches_without_on_clauses
+ - avoid_catching_errors
+ # - avoid_classes_with_only_static_members
+ - avoid_double_and_int_checks
+ # - avoid_dynamic_calls # we use this for Json
+ - avoid_empty_else
+ - avoid_equals_and_hash_code_on_mutable_classes
+ # - avoid_escaping_inner_quotes
+ - avoid_field_initializers_in_const_classes
+ # - avoid_final_parameters
+ - avoid_function_literals_in_foreach_calls
+ - avoid_implementing_value_types
+ - avoid_init_to_null
+ # - avoid_js_rounded_ints
+ # - avoid_multiple_declarations_per_line
+ - avoid_null_checks_in_equality_operators
+ - avoid_positional_boolean_parameters
+ # - avoid_print
+ - avoid_private_typedef_functions
+ - avoid_redundant_argument_values
+ - avoid_relative_lib_imports
+ - avoid_renaming_method_parameters
+ - avoid_return_types_on_setters
+ - avoid_returning_null_for_void
+ - avoid_returning_this
+ - avoid_setters_without_getters
+ - avoid_shadowing_type_parameters
+ - avoid_single_cascade_in_expression_statements
+ # - avoid_slow_async_io
+ - avoid_type_to_string
+ # - avoid_types_as_parameter_names
+ # - avoid_types_on_closure_parameters
+ - avoid_unnecessary_containers
+ - avoid_unused_constructor_parameters
+ - avoid_void_async
+ - avoid_web_libraries_in_flutter
+ - await_only_futures
+ - camel_case_extensions
+ - camel_case_types
+ - cancel_subscriptions
+ # - cascade_invocations
+ - cast_nullable_to_non_nullable
+ - close_sinks
+ - collection_methods_unrelated_type
+ - combinators_ordering
+ - comment_references
+ - conditional_uri_does_not_exist
+ - constant_identifier_names
+ - control_flow_in_finally
+ - curly_braces_in_flow_control_structures
+ - dangling_library_doc_comments
+ - depend_on_referenced_packages
+ - deprecated_consistency
+ # - diagnostic_describe_all_properties
+ - directives_ordering
+ - discarded_futures
+ - do_not_use_environment
+ - empty_catches
+ - empty_constructor_bodies
+ - empty_statements
+ - eol_at_end_of_file
+ - exhaustive_cases
+ - file_names
+ - flutter_style_todos
+ - hash_and_equals
+ - implementation_imports
+ - implicit_call_tearoffs
+ - invalid_case_patterns
+ - join_return_with_assignment
+ - leading_newlines_in_multiline_strings
+ - library_annotations
+ - library_names
+ - library_prefixes
+ - library_private_types_in_public_api
+ # - lines_longer_than_80_chars
+ - literal_only_boolean_expressions
+ - missing_whitespace_between_adjacent_strings
+ - no_adjacent_strings_in_list
+ - no_default_cases
+ - no_duplicate_case_values
+ - no_leading_underscores_for_library_prefixes
+ - no_leading_underscores_for_local_identifiers
+ - no_logic_in_create_state
+ - no_runtimeType_toString
+ - non_constant_identifier_names
+ - noop_primitive_operations
+ - null_check_on_nullable_type_parameter
+ - null_closures
+ # - omit_local_variable_types
+ - one_member_abstracts
+ - only_throw_errors
+ - overridden_fields
+ - package_api_docs
+ - package_names
+ - package_prefixed_library_names
+ - parameter_assignments
+ - prefer_adjacent_string_concatenation
+ - prefer_asserts_in_initializer_lists
+ # - prefer_asserts_with_message
+ - prefer_collection_literals
+ - prefer_conditional_assignment
+ - prefer_const_constructors
+ - prefer_const_constructors_in_immutables
+ - prefer_const_declarations
+ - prefer_const_literals_to_create_immutables
+ - prefer_constructors_over_static_methods
+ - prefer_contains
+ # - prefer_double_quotes
+ # - prefer_expression_function_bodies
+ - prefer_final_fields
+ - prefer_final_in_for_each
+ - prefer_final_locals
+ # - prefer_final_parameters
+ - prefer_for_elements_to_map_fromIterable
+ - prefer_foreach
+ - prefer_function_declarations_over_variables
+ - prefer_generic_function_type_aliases
+ - prefer_if_elements_to_conditional_expressions
+ - prefer_if_null_operators
+ - prefer_initializing_formals
+ - prefer_inlined_adds
+ # - prefer_int_literals
+ - prefer_interpolation_to_compose_strings
+ - prefer_is_empty
+ - prefer_is_not_empty
+ - prefer_is_not_operator
+ - prefer_iterable_whereType
+ - prefer_mixin
+ - prefer_null_aware_method_calls
+ - prefer_null_aware_operators
+ - prefer_relative_imports
+ - prefer_single_quotes
+ - prefer_spread_collections
+ - prefer_typing_uninitialized_variables
+ - prefer_void_to_null
+ - provide_deprecation_message
+ # - public_member_api_docs
+ - recursive_getters
+ # - require_trailing_commas
+ - secure_pubspec_urls
+ - sized_box_for_whitespace
+ - sized_box_shrink_expand
+ - slash_for_doc_comments
+ - sort_child_properties_last
+ - sort_constructors_first
+ - sort_pub_dependencies
+ - sort_unnamed_constructors_first
+ - test_types_in_equals
+ - throw_in_finally
+ - tighten_type_of_initializing_formals
+ - type_annotate_public_apis
+ - type_init_formals
+ - unawaited_futures
+ - unnecessary_await_in_return
+ - unnecessary_brace_in_string_interps
+ - unnecessary_breaks
+ - unnecessary_const
+ - unnecessary_constructor_name
+ # - unnecessary_final
+ - unnecessary_getters_setters
+ - unnecessary_lambdas
+ - unnecessary_late
+ - unnecessary_library_directive
+ - unnecessary_new
+ - unnecessary_null_aware_assignments
+ - unnecessary_null_aware_operator_on_extension_on_nullable
+ - unnecessary_null_checks
+ - unnecessary_null_in_if_null_operators
+ - unnecessary_nullable_for_final_variable_declarations
+ - unnecessary_overrides
+ - unnecessary_parenthesis
+ # - unnecessary_raw_strings
+ - unnecessary_statements
+ - unnecessary_string_escapes
+ - unnecessary_string_interpolations
+ - unnecessary_this
+ - unnecessary_to_list_in_spreads
+ - unreachable_from_main
+ - unrelated_type_equality_checks
+ - unsafe_html
+ - use_build_context_synchronously
+ - use_colored_box
+ - use_decorated_box
+ - use_enums
+ - use_full_hex_values_for_flutter_colors
+ - use_function_type_syntax_for_parameters
+ - use_if_null_to_convert_nulls_to_bools
+ - use_is_even_rather_than_modulo
+ - use_key_in_widget_constructors
+ - use_late_for_private_fields_and_variables
+ - use_named_constants
+ - use_raw_strings
+ - use_rethrow_when_possible
+ - use_setters_to_change_properties
+ - use_string_buffers
+ - use_string_in_part_of_directives
+ - use_super_parameters
+ - use_test_throws_matchers
+ - use_to_and_as_if_applicable
+ - valid_regexps
+ - void_checks
diff --git a/triage_bot/app.yaml b/triage_bot/app.yaml
new file mode 100644
index 0000000..88f17cc
--- /dev/null
+++ b/triage_bot/app.yaml
@@ -0,0 +1,10 @@
+# Copyright 2019 The Flutter Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+runtime: custom
+env: flex
+service: triage_bot
+
+resources:
+ memory_gb: 2.0
diff --git a/triage_bot/bin/triage_bot.dart b/triage_bot/bin/triage_bot.dart
new file mode 100644
index 0000000..1cfd367
--- /dev/null
+++ b/triage_bot/bin/triage_bot.dart
@@ -0,0 +1,7 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'package:triage_bot/engine.dart';
+
+void main() async => startEngine(null);
diff --git a/triage_bot/cloudbuild_triage_bot.yaml b/triage_bot/cloudbuild_triage_bot.yaml
new file mode 100644
index 0000000..940fb3f
--- /dev/null
+++ b/triage_bot/cloudbuild_triage_bot.yaml
@@ -0,0 +1,27 @@
+# Copyright 2019 The Flutter Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+steps:
+ # Build docker image
+ - name: 'us-docker.pkg.dev/cloud-builders/ga/v1/docker'
+ args: ['build', '-t', 'us-docker.pkg.dev/$PROJECT_ID/appengine/triage_bot.version-$SHORT_SHA', 'triage_bot']
+
+ # Trigger the cloud build that deploys the docker image
+ - name: gcr.io/cloud-builders/gcloud
+ entrypoint: '/bin/bash'
+ args:
+ - '-c'
+ - |-
+ gcloud builds submit \
+ --config triage_bot/cloudbuild_triage_bot_deploy.yaml \
+ --substitutions="SHORT_SHA=$SHORT_SHA" \
+ --async
+
+timeout: 1200s
+
+images: ['us-docker.pkg.dev/$PROJECT_ID/appengine/triage_bot.version-$SHORT_SHA']
+
+# If build provenance is not generated, the docker deployment will fail.
+options:
+ requestedVerifyOption: VERIFIED
diff --git a/triage_bot/cloudbuild_triage_bot_deploy.yaml b/triage_bot/cloudbuild_triage_bot_deploy.yaml
new file mode 100644
index 0000000..0eb2aa3
--- /dev/null
+++ b/triage_bot/cloudbuild_triage_bot_deploy.yaml
@@ -0,0 +1,39 @@
+# Copyright 2019 The Flutter Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style license that can be
+# found in the LICENSE file.
+
+steps:
+ # Get recently pushed docker image and associated provenance, along with the
+ # correct docker digest url, including the hash.
+ - name: gcr.io/cloud-builders/gcloud
+ entrypoint: '/bin/bash'
+ args:
+ - '-c'
+ - |-
+ cloud_build/get_docker_image_provenance.sh \
+ us-docker.pkg.dev/$PROJECT_ID/appengine/triage_bot.version-$SHORT_SHA:latest \
+ unverified_provenance.json
+
+ # Verify provenance is valid before proceeding with deployment.
+ - name: 'golang:1.20'
+ entrypoint: '/bin/bash'
+ args:
+ - '-c'
+ - |-
+ cloud_build/verify_provenance.sh unverified_provenance.json
+
+ # Deploy a new version to google cloud.
+ - name: gcr.io/cloud-builders/gcloud
+ entrypoint: '/bin/bash'
+ args:
+ - '-c'
+ - |-
+ gcloud config set project $PROJECT_ID
+ latest_version=$(gcloud app versions list --hide-no-traffic --format 'value(version.id)')
+ if [ "$latest_version" = "version-$SHORT_SHA" ]; then
+ echo "No updates since last deployment."
+ else
+ bash cloud_build/deploy_triage_bot.sh $PROJECT_ID $SHORT_SHA
+ fi
+
+timeout: 1200s
diff --git a/triage_bot/lib/bytes.dart b/triage_bot/lib/bytes.dart
new file mode 100644
index 0000000..74def6a
--- /dev/null
+++ b/triage_bot/lib/bytes.dart
@@ -0,0 +1,223 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'dart:convert';
+import 'dart:io';
+import 'dart:typed_data';
+
+typedef Reader<T> = T Function();
+typedef CustomReader<T> = T Function(FileReader reader);
+typedef Writer<T> = void Function(T value);
+typedef CustomWriter<T> = void Function(FileWriter reader, T value);
+
+const int _typeNullable = 0x01;
+const int _typeBool = 0x02;
+const int _typeInt = 0x03;
+const int _typeString = 0x04;
+const int _typeDateTime = 0x05;
+const int _typeSet = 0x10;
+const int _typeMap = 0x11;
+const int _typeCustom = 0xFE;
+const int _typeEnd = 0xFF;
+
+class FileReader {
+ FileReader(this._buffer) : _endianness = Endian.host;
+
+ final ByteData _buffer;
+ final Endian _endianness;
+ int _position = 0;
+
+ static Future<FileReader> open(File file) async {
+ final Uint8List bytes = await file.readAsBytes();
+ return FileReader(bytes.buffer.asByteData(bytes.offsetInBytes, bytes.length));
+ }
+
+ void _readType(int expected) {
+ final int type = _buffer.getUint8(_position);
+ _position += 1;
+ if (expected != type) {
+ throw FormatException('expected $expected but got $type at byte ${_position - 1}');
+ }
+ }
+
+ T? readNullOr<T>(Reader<T> reader) {
+ _readType(_typeNullable);
+ final int result = _buffer.getUint8(_position);
+ _position += 1;
+ if (result == 0) {
+ return null;
+ }
+ return reader();
+ }
+
+ bool readBool() {
+ _readType(_typeBool);
+ final int result = _buffer.getUint8(_position);
+ _position += 1;
+ return result != 0x00;
+ }
+
+ int readInt() {
+ _readType(_typeInt);
+ final int result = _buffer.getInt64(_position, _endianness);
+ _position += 8;
+ return result;
+ }
+
+ String readString() {
+ _readType(_typeString);
+ final int length = readInt();
+ final String result = utf8.decode(_buffer.buffer.asUint8List(_buffer.offsetInBytes + _position, length));
+ _position += length;
+ return result;
+ }
+
+ DateTime readDateTime() {
+ _readType(_typeDateTime);
+ return DateTime.fromMicrosecondsSinceEpoch(readInt(), isUtc: true);
+ }
+
+ Reader<Set<T>> readerForSet<T>(Reader<T> reader) {
+ return () {
+ _readType(_typeSet);
+ final int count = readInt();
+ final Set<T> result = <T>{};
+ for (int index = 0; index < count; index += 1) {
+ result.add(reader());
+ }
+ return result;
+ };
+ }
+
+ Set<T> readSet<T>(Reader<T> reader) {
+ return readerForSet<T>(reader)();
+ }
+
+ Reader<Map<K, V>> readerForMap<K, V>(Reader<K> keyReader, Reader<V> valueReader) {
+ return () {
+ _readType(_typeMap);
+ final int count = readInt();
+ final Map<K, V> result = <K, V>{};
+ for (int index = 0; index < count; index += 1) {
+ result[keyReader()] = valueReader();
+ }
+ return result;
+ };
+ }
+
+ Map<K, V> readMap<K, V>(Reader<K> keyReader, Reader<V> valueReader) {
+ return readerForMap<K, V>(keyReader, valueReader)();
+ }
+
+ Reader<T> readerForCustom<T>(CustomReader<T> reader) {
+ return () {
+ _readType(_typeCustom);
+ return reader(this);
+ };
+ }
+
+ void close() {
+ _readType(_typeEnd);
+ if (_position != _buffer.lengthInBytes) {
+ throw StateError('read failed; position=$_position, expected ${_buffer.lengthInBytes}');
+ }
+ }
+}
+
+class FileWriter {
+ FileWriter() : _endianness = Endian.host;
+
+ final BytesBuilder _buffer = BytesBuilder();
+ final Endian _endianness;
+
+ void _writeType(int type) {
+ _buffer.addByte(type);
+ }
+
+ void writeNullOr<T>(T? value, Writer<T> writer) {
+ _writeType(_typeNullable);
+ if (value == null) {
+ _buffer.addByte(0x00);
+ } else {
+ _buffer.addByte(0xFF);
+ writer(value);
+ }
+ }
+
+ void writeBool(bool value) { // ignore: avoid_positional_boolean_parameters
+ _writeType(_typeBool);
+ _buffer.addByte(value ? 0x01 : 0x00);
+ }
+
+ final ByteData _intBuffer = ByteData(8);
+ late final Uint8List _intBytes = _intBuffer.buffer.asUint8List();
+
+ void writeInt(int value) {
+ _writeType(_typeInt);
+ _intBuffer.setInt64(0, value, _endianness);
+ _buffer.add(_intBytes);
+ }
+
+ void writeString(String value) {
+ _writeType(_typeString);
+ final List<int> stringBuffer = utf8.encode(value);
+ writeInt(stringBuffer.length);
+ _buffer.add(stringBuffer);
+ }
+
+ void writeDateTime(DateTime value) {
+ _writeType(_typeDateTime);
+ writeInt(value.microsecondsSinceEpoch);
+ }
+
+ Writer<Set<T>> writerForSet<T>(Writer<T> writer) {
+ return (Set<T> value) {
+ _writeType(_typeSet);
+ writeInt(value.length);
+ value.forEach(writer);
+ };
+ }
+
+ void writeSet<T>(Writer<T> writer, Set<T> value) {
+ writerForSet<T>(writer)(value);
+ }
+
+ Writer<Map<K, V>> writerForMap<K, V>(Writer<K> keyWriter, Writer<V> valueWriter) {
+ return (Map<K, V> value) {
+ _writeType(_typeMap);
+ writeInt(value.length);
+ value.forEach((K key, V value) {
+ keyWriter(key);
+ valueWriter(value);
+ });
+ };
+ }
+
+ void writeMap<K, V>(Writer<K> keyWriter, Writer<V> valueWriter, Map<K, V> value) {
+ writerForMap<K, V>(keyWriter, valueWriter)(value);
+ }
+
+ Writer<T> writerForCustom<T>(CustomWriter<T> writer) {
+ return (T value) {
+ _writeType(_typeCustom);
+ writer(this, value);
+ };
+ }
+
+ Future<void> write(File file) async {
+ _writeType(_typeEnd);
+ final File temp = File('${file.path}.\$\$\$');
+ await temp.writeAsBytes(_buffer.takeBytes());
+ if (file.existsSync()) {
+ await file.delete();
+ }
+ await temp.rename(file.path);
+ }
+
+ ByteData serialize() {
+ _writeType(_typeEnd);
+ final int length = _buffer.length;
+ return _buffer.takeBytes().buffer.asByteData(0, length);
+ }
+}
diff --git a/triage_bot/lib/discord.dart b/triage_bot/lib/discord.dart
new file mode 100644
index 0000000..3c780f7
--- /dev/null
+++ b/triage_bot/lib/discord.dart
@@ -0,0 +1,135 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'package:nyxx/nyxx.dart';
+
+sealed class DiscordChannels {
+ static const Snowflake botTest = Snowflake.value(945411053179764736);
+ static const Snowflake hiddenChat = Snowflake.value(610574672865656952);
+ static const Snowflake github2 = Snowflake.value(1116095786657267722); // this value is >2^53 and thus cannot be used in JS mode
+}
+
+List<Pattern> get boilerplates => <Pattern>[
+ '\r',
+ r'### Is there an existing issue for this?',
+ RegExp(r'- \[[ xX]] I have searched the \[existing issues]\(https://github\.com/flutter/flutter/issues\)'),
+ RegExp(r'- \[[ xX]] I have read the \[guide to filing a bug]\(https://flutter\.dev/docs/resources/bug-reports\)'),
+ r'*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*',
+ r'*List which issues are fixed by this PR. You must list at least one issue.*',
+ r'*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*',
+ r'## Pre-launch Checklist',
+ RegExp(r'- \[[ xX]] I read the \[Contributor Guide] and followed the process outlined there for submitting PRs\.'),
+ RegExp(r'- \[[ xX]] I read the \[Tree Hygiene] wiki page, which explains my responsibilities\.'),
+ RegExp(r'- \[[ xX]] I read and followed the \[Flutter Style Guide], including \[Features we expect every widget to implement]\.'),
+ RegExp(r'- \[[ xX]] I read the \[Flutter Style Guide] _recently_, and have followed its advice\.'),
+ RegExp(r'- \[[ xX]] I read and followed the \[Flutter Style Guide] and the \[C\+\+, Objective-C, Java style guides]\.'),
+ RegExp(r'- \[[ xX]] I read and followed the \[relevant style guides] and ran the auto-formatter\. \(Unlike the flutter/flutter repo, the flutter/packages repo does use `dart format`\.\)'),
+ RegExp(r'- \[[ xX]] I signed the \[CLA]\.'),
+ RegExp(r'- \[[ xX]] I listed at least one issue that this PR fixes in the description above\.'),
+ RegExp(r'- \[[ xX]] I updated/added relevant documentation \(doc comments with `///`\)\.'),
+ RegExp(r'- \[[ xX]] I added new tests to check the change I am making, or this PR is \[test-exempt]\.'),
+ RegExp(r'- \[[ xX]] I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt\. See \[testing the engine] for instructions on writing and running engine tests\.'),
+ RegExp(r'- \[[ xX]] All existing and new tests are passing\.'),
+ RegExp(r'- \[[ xX]] The title of the PR starts with the name of the package surrounded by square brackets, e\.g\. `\[shared_preferences]`'),
+ RegExp(r'- \[[ xX]] I updated `pubspec.yaml` with an appropriate new version according to the \[pub versioning philosophy], or this PR is \[exempt from version changes]\.'),
+ RegExp(r'- \[[ xX]] I updated `CHANGELOG.md` to add a description of the change, \[following repository CHANGELOG style]\.'),
+ r'If you need help, consider asking for advice on the #hackers-new channel on [Discord].',
+ r'<!-- Links -->',
+ r'[Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview',
+ r'[Contributor Guide]: https://github.com/flutter/packages/blob/main/CONTRIBUTING.md',
+ r'[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene',
+ r'[test-exempt]: https://github.com/flutter/flutter/wiki/Tree-hygiene#tests',
+ r'[Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo',
+ r'[Features we expect every widget to implement]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement',
+ r'[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style',
+ r'[relevant style guides]: https://github.com/flutter/packages/blob/main/CONTRIBUTING.md#style',
+ r'[testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine',
+ r'[CLA]: https://cla.developers.google.com/',
+ r'[flutter/tests]: https://github.com/flutter/tests',
+ r'[breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes',
+ r'[Discord]: https://github.com/flutter/flutter/wiki/Chat',
+ r'[pub versioning philosophy]: https://dart.dev/tools/pub/versioning',
+ r'[exempt from version changes]: https://github.com/flutter/flutter/wiki/Contributing-to-Plugins-and-Packages#version-and-changelog-updates',
+ '### Screenshots or Video\n\n<details>\n<summary>Screenshots / Video demonstration</summary>\n\n[Upload media here]\n\n</details>',
+ '### Logs\n\n<details><summary>Logs</summary>\n\n```console\n[Paste your logs here]\n```\n\n</details>',
+ '### Code sample\n\n<details><summary>Code sample</summary>\n\n```dart\n[Paste your code here]\n```\n\n</details>',
+ '<!-- Thank you for contributing to Flutter!\n\n If you are filing a bug, please add the steps to reproduce, expected and actual results.\n\n If you are filing a feature request, please describe the use case and a proposal.\n\n If you are requesting a small infra task with P0 or P1 priority, please add it to the\n "Infra Ticket Queue" project with "New" column, explain why the task is needed and what\n actions need to perform (if you happen to know). No need to set an assignee; the infra oncall\n will triage and process the infra ticket queue.\n-->',
+ r'<details>',
+ r'<summary>',
+ r'</details>',
+ r'</summary>',
+];
+
+String stripBoilerplate(String message, { bool inline = false }) {
+ String current = message;
+ for (final Pattern candidate in boilerplates) {
+ current = current.replaceAll(candidate, '');
+ }
+ current = current
+ .replaceAll(RegExp(r'\n( *\n)+'), '\n\n')
+ .trim();
+ if (current.isEmpty) {
+ return '<blank>';
+ }
+ if (current.contains('\n') && inline) {
+ return '\n$current';
+ }
+ return current;
+}
+
+const int _maxLength = 2000;
+const String _truncationMarker = '\n**[...truncated]**';
+const String _padding = '\n╰╴ ';
+final RegExp _imagePattern = RegExp(r'!\[[^\]]*]\(([^)]+)\)$');
+
+Future<void> sendDiscordMessage({
+ required INyxx discord,
+ required String body,
+ String suffix = '',
+ required Snowflake channel,
+ IEmoji? emoji,
+ String? embedTitle,
+ String? embedDescription,
+ String? embedColor,
+ required void Function(String) log,
+}) async {
+ assert(_maxLength > _truncationMarker.length + _padding.length + suffix.length);
+ assert((embedTitle == null) == (embedDescription == null) && (embedDescription == null) == (embedColor == null));
+ final String content;
+ final List<String> embeds = <String>[];
+ body = body.replaceAllMapped(_imagePattern, (Match match) { // this replaces a trailing markdown image with actually showing that image in discord
+ embeds.add(match.group(1)!);
+ return '';
+ });
+ if (body.length + _padding.length + suffix.length > _maxLength) {
+ content = body.substring(0, _maxLength - _truncationMarker.length - _padding.length - suffix.length) + _truncationMarker + _padding + suffix;
+ } else if (suffix.isNotEmpty) {
+ content = body + _padding + suffix;
+ } else {
+ content = body;
+ }
+ final MessageBuilder messageBuilder = MessageBuilder(
+ content: content,
+ embeds: <EmbedBuilder>[
+ for (final String url in embeds)
+ EmbedBuilder(
+ imageUrl: url,
+ ),
+ if (embedDescription != null)
+ EmbedBuilder(
+ title: embedTitle,
+ description: embedDescription,
+ color: DiscordColor.fromHexString(embedColor!),
+ ),
+ ],
+ )..flags = (MessageFlagBuilder()..suppressEmbeds = embeds.isEmpty && embedDescription == null);
+ try {
+ final IMessage message = await discord.httpEndpoints.sendMessage(channel, messageBuilder);
+ if (emoji != null) {
+ await message.createReaction(emoji);
+ }
+ } catch (e) {
+ log('Discord error: $e (${e.runtimeType})');
+ }
+}
diff --git a/triage_bot/lib/engine.dart b/triage_bot/lib/engine.dart
new file mode 100644
index 0000000..2d5d030
--- /dev/null
+++ b/triage_bot/lib/engine.dart
@@ -0,0 +1,1510 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'dart:async';
+import 'dart:convert';
+import 'dart:io';
+import 'dart:typed_data';
+
+import 'package:appengine/appengine.dart' show authClientService, runAppEngine, withAppEngineServices;
+import 'package:crypto/crypto.dart';
+import 'package:dart_jsonwebtoken/dart_jsonwebtoken.dart';
+import 'package:github/github.dart';
+import 'package:googleapis/secretmanager/v1.dart';
+import 'package:http/http.dart' as http show Client;
+import 'package:nyxx/nyxx.dart';
+
+import 'bytes.dart';
+import 'discord.dart';
+import 'json.dart';
+import 'utils.dart';
+
+const int port = 8213; // used only when not using appengine
+const int maxLogLength = 1024;
+
+sealed class GitHubSettings {
+ static const String organization = 'flutter';
+ static const String teamName = 'flutter-hackers';
+ static final RepositorySlug primaryRepository = RepositorySlug(organization, 'flutter');
+ static const String teamPrefix = 'team-';
+ static const String triagedPrefix = 'triaged-';
+ static const String fyiPrefix = 'fyi-';
+ static const String designDoc = 'design doc';
+ static const String permanentlyLocked = 'permanently locked';
+ static const String thumbsUpLabel = ':+1:';
+ static const String staleIssueLabel = ':hourglass_flowing_sand:';
+ static const Set<String> priorities = <String>{ 'P0', 'P1', 'P2', 'P3' };
+ static const String staleP1Message = 'This issue is marked P1 but has had no recent status updates.\n'
+ '\n'
+ 'The P1 label indicates high-priority issues that are at the top of the work list. '
+ 'This is the highest priority level a bug can have '
+ 'if it isn\'t affecting a top-tier customer or breaking the build. '
+ 'Bugs marked P1 are generally actively being worked on '
+ 'unless the assignee is dealing with a P0 bug (or another P1 bug). '
+ 'Issues at this level should be resolved in a matter of months and should have monthly updates on GitHub.\n'
+ '\n'
+ 'Please consider where this bug really falls in our current priorities, and label it or assign it accordingly. '
+ 'This allows people to have a clearer picture of what work is actually planned. Thanks!';
+ static const String willNeedAdditionalTriage = 'will need additional triage';
+ static const Set<String> teams = <String>{
+ // these are the teams that the self-test issue is assigned to
+ 'android',
+ 'codelabs',
+ 'design',
+ 'desktop',
+ 'ecosystem',
+ 'engine',
+ 'framework',
+ 'games',
+ 'google-testing',
+ 'go_router',
+ 'infra',
+ 'ios',
+ 'news',
+ 'release',
+ 'tool',
+ 'web',
+ };
+ static const int thumbsMinimum = 100; // an issue needs at least this many thumbs up to trigger retriage
+ static const double thumbsThreshold = 2.0; // and the count must have increased by this factor since last triage
+ static const Set<String> knownBots = <String>{ // we don't report events from bots to Discord
+ 'auto-submit[bot]',
+ 'DartDevtoolWorkflowBot',
+ 'dependabot[bot]',
+ 'engine-flutter-autoroll',
+ 'flutter-dashboard[bot]',
+ 'flutter-triage-bot[bot]', // that's us!
+ 'fluttergithubbot',
+ 'github-actions[bot]',
+ 'google-cla[bot]',
+ 'google-ospo-administrator[bot]',
+ 'skia-flutter-autoroll',
+ };
+
+ static bool isRelevantLabel(String label, { bool ignorePriorities = false }) {
+ return label.startsWith(GitHubSettings.teamPrefix)
+ || label.startsWith(GitHubSettings.triagedPrefix)
+ || label.startsWith(GitHubSettings.fyiPrefix)
+ || label == GitHubSettings.designDoc
+ || label == GitHubSettings.permanentlyLocked
+ || label == GitHubSettings.thumbsUpLabel
+ || label == GitHubSettings.staleIssueLabel
+ || (!ignorePriorities && GitHubSettings.priorities.contains(label));
+ }
+}
+
+sealed class Timings {
+ static const Duration backgroundUpdatePeriod = Duration(seconds: 1); // how long to wait between issues when scanning in the background
+ static const Duration cleanupUpdateDelay = Duration(minutes: 45); // how long to wait for an issue to be idle before cleaning it up
+ static const Duration cleanupUpdatePeriod = Duration(seconds: 60); // time between attempting to clean up the pending cleanup issues
+ static const Duration longTermTidyingPeriod = Duration(hours: 5); // time between attempting to run long-term tidying of all issues
+ static const Duration credentialsUpdatePeriod = Duration(minutes: 45); // how often to update GitHub credentials
+ static const Duration timeUntilStale = Duration(days: 20 * 7); // how long since the last team interaction before considering an issue stale
+ static const Duration timeUntilReallyStale = Duration(days: 30 * 7); // how long since the last team interaction before unassigning an issue
+ static const Duration timeUntilUnlock = Duration(days: 28); // how long to leave open issues locked
+ static const Duration selfTestPeriod = Duration(days: 4 * 7); // how often to file an issue to test the triage process
+ static const Duration selfTestWindow = Duration(days: 14); // how long to leave the self-test issue open before assigning it to critical triage
+ static const Duration refeedDelay = Duration(hours: 48); // how long between times we mark an issue as needing retriage (~3 a week)
+}
+
+final class Secrets {
+ Future<List<int>> get serverCertificate => _getSecret('server.cert.pem');
+ Future<DateTime> get serverCertificateModificationDate => _getSecretModificationDate('server.cert.pem');
+ Future<List<int>> get serverIntermediateCertificates => _getSecret('server.intermediates.pem');
+ Future<DateTime> get serverIntermediateCertificatesModificationDate => _getSecretModificationDate('server.intermediates.pem');
+ Future<List<int>> get serverKey => _getSecret('server.key.pem');
+ Future<DateTime> get serverKeyModificationDate => _getSecretModificationDate('server.key.pem');
+ Future<String> get discordToken async => utf8.decode(await _getSecret('discord.token'));
+ Future<int> get discordAppId async => int.parse(utf8.decode(await _getSecret('discord.appid')));
+ Future<List<int>> get githubWebhookSecret => _getSecret('github.webhook.secret');
+ Future<String> get githubAppKey async => utf8.decode(await _getSecret('github.app.key.pem'));
+ Future<String> get githubAppId async => utf8.decode(await _getSecret('github.app.id'));
+ Future<String> get githubInstallationId async => utf8.decode(await _getSecret('github.installation.id'));
+ final File store = File('store.db');
+
+ static const String _projectId = 'xxxxx?????xxxxx'; // TODO(ianh): we should update this appropriately
+
+ static File asFile(String name) => File('secrets/$name');
+
+ static String asKey(String name) => 'projects/$_projectId/secrets/$name/versions/latest';
+
+ static Future<List<int>> _getSecret(String name) async {
+ final File file = asFile(name);
+ if (await file.exists()) {
+ return file.readAsBytes();
+ }
+ // authClientService is https://pub.dev/documentation/gcloud/latest/http/authClientService.html
+ final SecretManagerApi secretManager = SecretManagerApi(authClientService);
+ final String key = asKey(name);
+ final AccessSecretVersionResponse response = await secretManager.projects.secrets.versions.access(key);
+ return response.payload!.dataAsBytes;
+ }
+
+ static Future<DateTime> _getSecretModificationDate(String name) async {
+ final File file = asFile(name);
+ if (await file.exists()) {
+ return file.lastModified();
+ }
+ // authClientService is https://pub.dev/documentation/gcloud/latest/http/authClientService.html
+ final SecretManagerApi secretManager = SecretManagerApi(authClientService);
+ final String key = asKey(name);
+ final SecretVersion response = await secretManager.projects.secrets.versions.get(key);
+ return DateTime.parse(response.createTime!);
+ }
+}
+
+class IssueStats {
+ IssueStats({
+ this.lastContributorTouch,
+ this.lastAssigneeTouch,
+ Set<String>? labels,
+ this.openedAt,
+ this.lockedAt,
+ this.assignedAt,
+ this.assignedToTeamMemberReporter = false,
+ this.triagedAt,
+ this.thumbsAtTriageTime,
+ this.thumbs = 0,
+ }) : labels = labels ?? <String>{};
+
+ factory IssueStats.read(FileReader reader) {
+ return IssueStats(
+ lastContributorTouch: reader.readNullOr<DateTime>(reader.readDateTime),
+ lastAssigneeTouch: reader.readNullOr<DateTime>(reader.readDateTime),
+ labels: reader.readSet<String>(reader.readString),
+ openedAt: reader.readNullOr<DateTime>(reader.readDateTime),
+ lockedAt: reader.readNullOr<DateTime>(reader.readDateTime),
+ assignedAt: reader.readNullOr<DateTime>(reader.readDateTime),
+ assignedToTeamMemberReporter: reader.readBool(),
+ triagedAt: reader.readNullOr<DateTime>(reader.readDateTime),
+ thumbsAtTriageTime: reader.readNullOr<int>(reader.readInt),
+ thumbs: reader.readInt(),
+ );
+ }
+
+ static void write(FileWriter writer, IssueStats value) {
+ writer.writeNullOr<DateTime>(value.lastContributorTouch, writer.writeDateTime);
+ writer.writeNullOr<DateTime>(value.lastAssigneeTouch, writer.writeDateTime);
+ writer.writeSet<String>(writer.writeString, value.labels);
+ writer.writeNullOr<DateTime>(value.openedAt, writer.writeDateTime);
+ writer.writeNullOr<DateTime>(value.lockedAt, writer.writeDateTime);
+ writer.writeNullOr<DateTime>(value.assignedAt, writer.writeDateTime);
+ writer.writeBool(value.assignedToTeamMemberReporter);
+ writer.writeNullOr<DateTime>(value.triagedAt, writer.writeDateTime);
+ writer.writeNullOr<int>(value.thumbsAtTriageTime, writer.writeInt);
+ writer.writeInt(value.thumbs);
+ }
+
+ DateTime? lastContributorTouch;
+ DateTime? lastAssigneeTouch;
+ Set<String> labels;
+ DateTime? openedAt;
+ DateTime? lockedAt;
+ DateTime? assignedAt;
+ bool assignedToTeamMemberReporter = false;
+ DateTime? triagedAt;
+ int? thumbsAtTriageTime;
+ int thumbs;
+
+ @override
+ String toString() {
+ final StringBuffer buffer = StringBuffer();
+ buffer.write('{${(labels.toList()..sort()).join(', ')}} and $thumbs 👍');
+ if (openedAt != null) {
+ buffer.write('; openedAt: $openedAt');
+ }
+ if (lastContributorTouch != null) {
+ buffer.write('; lastContributorTouch: $lastContributorTouch');
+ } else {
+ buffer.write('; lastContributorTouch: never');
+ }
+ if (assignedAt != null) {
+ buffer.write('; assignedAt: $assignedAt');
+ if (assignedToTeamMemberReporter) {
+ buffer.write(' (to team-member reporter)');
+ }
+ if (lastAssigneeTouch != null) {
+ buffer.write('; lastAssigneeTouch: $lastAssigneeTouch');
+ } else {
+ buffer.write('; lastAssigneeTouch: never');
+ }
+ } else {
+ if (lastAssigneeTouch != null) {
+ buffer.write('; lastAssigneeTouch: $lastAssigneeTouch (?!)');
+ }
+ if (assignedToTeamMemberReporter) {
+ buffer.write('; assigned to team-member reporter (?!)');
+ }
+ }
+ if (lockedAt != null) {
+ buffer.write('; lockedAt: $lockedAt');
+ }
+ if (triagedAt != null) {
+ buffer.write('; triagedAt: $triagedAt');
+ if (thumbsAtTriageTime != null) {
+ buffer.write(' with $thumbsAtTriageTime 👍');
+ }
+ } else {
+ if (thumbsAtTriageTime != null) {
+ buffer.write('; not triaged with $thumbsAtTriageTime 👍 when triaged (?!)');
+ }
+ }
+ return buffer.toString();
+ }
+}
+
+typedef StoreFields = ({
+ Map<int, IssueStats> issues,
+ Map<int, DateTime> pendingCleanupIssues,
+ int? selfTestIssue,
+ DateTime? selfTestClosedDate,
+ int currentBackgroundIssue,
+ int highestKnownIssue,
+ Map<String, DateTime> lastRefeedByTime,
+ DateTime? lastCleanupStart,
+ DateTime? lastCleanupEnd,
+ DateTime? lastTidyStart,
+ DateTime? lastTidyEnd,
+});
+
+class Engine {
+ Engine._({
+ required this.webhookSecret,
+ required this.discord,
+ required this.github,
+ required this.secrets,
+ required Set<String> contributors,
+ required StoreFields? store,
+ }) : _contributors = contributors,
+ _issues = store?.issues ?? <int, IssueStats>{},
+ _pendingCleanupIssues = store?.pendingCleanupIssues ?? <int, DateTime>{},
+ _selfTestIssue = store?.selfTestIssue,
+ _selfTestClosedDate = store?.selfTestClosedDate,
+ _currentBackgroundIssue = store?.currentBackgroundIssue ?? 1,
+ _highestKnownIssue = store?.highestKnownIssue ?? 1,
+ _lastRefeedByTime = store?.lastRefeedByTime ?? <String, DateTime>{},
+ _lastCleanupStart = store?.lastCleanupStart,
+ _lastCleanupEnd = store?.lastCleanupEnd,
+ _lastTidyStart = store?.lastTidyStart,
+ _lastTidyEnd = store?.lastTidyEnd {
+ _startup = DateTime.timestamp();
+ scheduleMicrotask(_updateStoreInBackground);
+ _nextCleanup = (_lastCleanupEnd ?? _startup).add(Timings.cleanupUpdatePeriod);
+ _cleanupTimer = Timer(_nextCleanup.difference(_startup), _performCleanups);
+ _nextTidy = (_lastTidyEnd ?? _startup).add(Timings.longTermTidyingPeriod);
+ _tidyTimer = Timer(_nextTidy.difference(_startup), _performLongTermTidying);
+ log('Startup');
+ }
+
+ static Future<Engine> initialize({
+ required List<int> webhookSecret,
+ required INyxx discord,
+ required GitHub github,
+ void Function()? onChange,
+ required Secrets secrets,
+ }) async {
+ return Engine._(
+ webhookSecret: webhookSecret,
+ discord: discord,
+ github: github,
+ secrets: secrets,
+ contributors: await _loadContributors(github),
+ store: await _read(secrets),
+ );
+ }
+
+ final List<int> webhookSecret;
+ final INyxx discord;
+ final GitHub github;
+ final Secrets secrets;
+
+ // data this is stored on local disk
+ final Set<String> _contributors;
+ final Map<int, IssueStats> _issues;
+ final Map<int, DateTime> _pendingCleanupIssues;
+ int? _selfTestIssue;
+ DateTime? _selfTestClosedDate;
+ int _currentBackgroundIssue;
+ int _highestKnownIssue;
+ final Map<String, DateTime> _lastRefeedByTime; // last time we forced an otherwise normal issue to get retriaged by each team
+
+ final Set<String> _recentIds = <String>{}; // used to detect duplicate messages and discard them
+ final List<String> _log = <String>[];
+ late final DateTime _startup;
+ DateTime? _lastCleanupStart;
+ DateTime? _lastCleanupEnd;
+ late DateTime _nextCleanup;
+ Timer? _cleanupTimer;
+ DateTime? _lastTidyStart;
+ DateTime? _lastTidyEnd;
+ late DateTime _nextTidy;
+ Timer? _tidyTimer;
+
+ void log(String message) {
+ stderr.writeln(message);
+ _log.add('${DateTime.timestamp().toIso8601String()} $message');
+ while (_log.length > maxLogLength) {
+ _log.removeAt(0);
+ }
+ }
+
+ int _actives = 0;
+ bool _shuttingDown = false;
+ Completer<void> _pendingIdle = Completer<void>();
+ Future<void> shutdown(Future<void> Function() shutdownCallback) async {
+ assert(!_shuttingDown, 'shutdown called reentrantly');
+ _shuttingDown = true;
+ while (_actives > 0) {
+ await _pendingIdle.future;
+ }
+ return shutdownCallback();
+ }
+
+ static Future<Set<String>> _loadContributors(GitHub github) async {
+ final int teamId = (await github.organizations.getTeamByName(GitHubSettings.organization, GitHubSettings.teamName)).id!;
+ return github.organizations.listTeamMembers(teamId).map((TeamMember member) => member.login!).toSet();
+ }
+
+ static Future<StoreFields?> _read(Secrets secrets) async {
+ if (await secrets.store.exists()) {
+ try {
+ final FileReader reader = FileReader((await secrets.store.readAsBytes()).buffer.asByteData());
+ return (
+ issues: reader.readMap<int, IssueStats>(reader.readInt, reader.readerForCustom<IssueStats>(IssueStats.read)),
+ pendingCleanupIssues: reader.readMap<int, DateTime>(reader.readInt, reader.readDateTime),
+ selfTestIssue: reader.readNullOr<int>(reader.readInt),
+ selfTestClosedDate: reader.readNullOr<DateTime>(reader.readDateTime),
+ currentBackgroundIssue: reader.readInt(),
+ highestKnownIssue: reader.readInt(),
+ lastRefeedByTime: reader.readMap<String, DateTime>(reader.readString, reader.readDateTime),
+ lastCleanupStart: reader.readNullOr<DateTime>(reader.readDateTime),
+ lastCleanupEnd: reader.readNullOr<DateTime>(reader.readDateTime),
+ lastTidyStart: reader.readNullOr<DateTime>(reader.readDateTime),
+ lastTidyEnd: reader.readNullOr<DateTime>(reader.readDateTime),
+ );
+ } catch (e) {
+ print('Error loading issue store, consider deleting ${secrets.store.path} file.');
+ rethrow;
+ }
+ }
+ return null;
+ }
+
+ bool _writing = false;
+ bool _dirty = false;
+ Future<void> _write() async {
+ if (_writing) {
+ _dirty = true;
+ return;
+ }
+ try {
+ _writing = true;
+ final FileWriter writer = FileWriter();
+ writer.writeMap<int, IssueStats>(writer.writeInt, writer.writerForCustom<IssueStats>(IssueStats.write), _issues);
+ writer.writeMap<int, DateTime>(writer.writeInt, writer.writeDateTime, _pendingCleanupIssues);
+ writer.writeNullOr<int>(_selfTestIssue, writer.writeInt);
+ writer.writeNullOr<DateTime>(_selfTestClosedDate, writer.writeDateTime);
+ writer.writeInt(_currentBackgroundIssue);
+ writer.writeInt(_highestKnownIssue);
+ writer.writeMap<String, DateTime>(writer.writeString, writer.writeDateTime, _lastRefeedByTime);
+ writer.writeNullOr<DateTime>(_lastCleanupStart, writer.writeDateTime);
+ writer.writeNullOr<DateTime>(_lastCleanupEnd, writer.writeDateTime);
+ writer.writeNullOr<DateTime>(_lastTidyStart, writer.writeDateTime);
+ writer.writeNullOr<DateTime>(_lastTidyEnd, writer.writeDateTime);
+ await writer.write(secrets.store);
+ } finally {
+ _writing = false;
+ }
+ if (_dirty) {
+ _dirty = false;
+ return _write();
+ }
+ }
+
+ // the maxFraction argument represents the fraction of the total rate limit that is allowed to be
+ // used before waiting.
+ //
+ // the background update code sets it to 0.5 so that there is still a buffer for the other calls,
+ // otherwise the background update code could just use it all up and then stall everything else.
+ Future<void> _githubReady([double maxFraction = 0.95]) async {
+ if (github.rateLimitRemaining != null && github.rateLimitRemaining! < (github.rateLimitLimit! * (1.0 - maxFraction)).round()) {
+ assert(github.rateLimitReset != null);
+ await _until(github.rateLimitReset!);
+ }
+ }
+
+ static Future<void> _until(DateTime target) {
+ final DateTime now = DateTime.timestamp();
+ if (!now.isBefore(target)) {
+ return Future<void>.value();
+ }
+ final Duration delta = target.difference(now);
+ return Future<void>.delayed(delta);
+ }
+
+ Future<void> handleRequest(HttpRequest request) async {
+ _actives += 1;
+ try {
+ try {
+ if (await _handleDebugRequests(request)) {
+ return;
+ }
+ final List<int> bytes = await request.expand((Uint8List sublist) => sublist).toList();
+ final String expectedSignature = 'sha256=${Hmac(sha256, webhookSecret).convert(bytes).bytes.map(hex).join()}';
+ final List<String> actualSignatures = request.headers['X-Hub-Signature-256'] ?? const <String>[];
+ final List<String> eventKind = request.headers['X-GitHub-Event'] ?? const <String>[];
+ final List<String> eventId = request.headers['X-GitHub-Delivery'] ?? const <String>[];
+ if (actualSignatures.length != 1 || expectedSignature != actualSignatures.single ||
+ eventKind.length != 1 || eventId.length != 1) {
+ request.response.writeln('Invalid metadata.');
+ return;
+ }
+ if (_recentIds.contains(eventId.single)) {
+ request.response.writeln('I got that one already.');
+ return;
+ }
+ _recentIds.add(eventId.single);
+ while (_recentIds.length > 50) {
+ _recentIds.remove(_recentIds.first);
+ }
+ final dynamic payload = Json.parse(utf8.decode(bytes));
+ await _updateModelFromWebhook(eventKind.single, payload);
+ await _updateDiscordFromWebhook(eventKind.single, payload);
+ request.response.writeln('Acknowledged.');
+ } catch (e, s) {
+ log('Failed to handle ${request.uri}: $e (${e.runtimeType})\n$s');
+ } finally {
+ await request.response.close();
+ }
+ } finally {
+ _actives -= 1;
+ if (_shuttingDown && _actives == 0) {
+ _pendingIdle.complete();
+ _pendingIdle = Completer<void>();
+ }
+ }
+ }
+
+ Future<bool> _handleDebugRequests(HttpRequest request) async {
+ if (request.uri.path == '/debug') {
+ final DateTime now = DateTime.timestamp();
+ request.response.writeln('FLUTTER TRIAGE BOT');
+ request.response.writeln('==================');
+ request.response.writeln();
+ request.response.writeln('Current time: $now');
+ request.response.writeln('Uptime: ${now.difference(_startup)} (startup at $_startup).');
+ request.response.writeln('Cleaning: ${_cleaning ? "active" : "pending"} (${_pendingCleanupIssues.length} issue${s(_pendingCleanupIssues.length)}); last started $_lastCleanupStart, last ended $_lastCleanupEnd, next in ${_nextCleanup.difference(now)}.');
+ request.response.writeln('Tidying: ${_tidying ? "active" : "pending"}; last started $_lastTidyStart, last ended $_lastTidyEnd, next in ${_nextTidy.difference(now)}.');
+ request.response.writeln('Background scan: currently fetching issue #$_currentBackgroundIssue, highest known issue #$_highestKnownIssue.');
+ request.response.writeln('${_contributors.length} known contributor${s(_contributors.length)}.');
+ request.response.writeln('GitHub Rate limit status: ${github.rateLimitRemaining}/${github.rateLimitLimit} (reset at ${github.rateLimitReset})');
+ if (_selfTestIssue != null) {
+ request.response.writeln('Current self test issue: #$_selfTestIssue');
+ }
+ if (_selfTestClosedDate != null) {
+ request.response.writeln('Self test last closed on: $_selfTestClosedDate (${now.difference(_selfTestClosedDate!)} ago, next in ${_selfTestClosedDate!.add(Timings.selfTestPeriod).difference(now)})');
+ }
+ request.response.writeln();
+ request.response.writeln('Last refeeds (refeed delay: ${Timings.refeedDelay}):');
+ for (final String team in _lastRefeedByTime.keys.toList()..sort((String a, String b) => _lastRefeedByTime[a]!.compareTo(_lastRefeedByTime[b]!))) {
+ final Duration delta = now.difference(_lastRefeedByTime[team]!);
+ final String annotation = delta > Timings.refeedDelay
+ ? ''
+ : '; blocking immediate refeeds';
+ request.response.writeln('${team.padRight(30, '.')}.${_lastRefeedByTime[team]} ($delta ago$annotation)');
+ }
+ request.response.writeln();
+ request.response.writeln('Tracking ${_issues.length} issue${s(_issues.length)}:');
+ for (final int number in _issues.keys.toList()..sort()) {
+ String cleanup = '';
+ if (_pendingCleanupIssues.containsKey(number)) {
+ final Duration delta = Timings.cleanupUpdateDelay - now.difference(_pendingCleanupIssues[number]!);
+ if (delta < Duration.zero) {
+ cleanup = ' [cleanup pending]';
+ } else if (delta.inMinutes <= 1) {
+ cleanup = ' [cleanup soon]';
+ } else {
+ cleanup = ' [cleanup in ${delta.inMinutes} minute${s(delta.inMinutes)}]';
+ }
+ }
+ request.response.writeln(' #${number.toString().padLeft(6, "0")}: ${_issues[number]}$cleanup');
+ }
+ request.response.writeln();
+ request.response.writeln('LOG');
+ _log.forEach(request.response.writeln);
+ return true;
+ }
+ if (request.uri.path == '/force-update') {
+ final int number = int.parse(request.uri.query); // if input is not an integer, this'll throw
+ await _updateStoreInBackgroundForIssue(number);
+ request.response.writeln('${_issues[number]}');
+ return true;
+ }
+ if (request.uri.path == '/force-cleanup') {
+ log('User-triggered forced cleanup');
+ await _performCleanups();
+ _log.forEach(request.response.writeln);
+ return true;
+ }
+ if (request.uri.path == '/force-tidy') {
+ log('User-triggered forced tidy');
+ await _performLongTermTidying();
+ _log.forEach(request.response.writeln);
+ return true;
+ }
+ return false;
+ }
+
+ // Called when we get a webhook message.
+ Future<void> _updateModelFromWebhook(String event, dynamic payload) async {
+ final DateTime now = DateTime.timestamp();
+ switch (event) {
+ case 'issue_comment':
+ if (!payload.issue.hasKey('pull_request') && payload.repository.full_name.toString() == GitHubSettings.primaryRepository.fullName) {
+ _updateIssueFromWebhook(payload.sender.login.toString(), payload.issue, now);
+ }
+ case 'issues':
+ if (payload.repository.full_name.toString() != GitHubSettings.primaryRepository.fullName) {
+ return;
+ }
+ if (payload.action.toString() == 'closed') {
+ final int number = payload.issue.number.toInt();
+ _issues.remove(number);
+ _pendingCleanupIssues.remove(number);
+ if (number == _selfTestIssue) {
+ _selfTestIssue = null;
+ _selfTestClosedDate = now;
+ }
+ } else {
+ final IssueStats? issue = _updateIssueFromWebhook(payload.sender.login.toString(), payload.issue, now);
+ if (issue != null) {
+ if (payload.action.toString() == 'assigned') {
+ // if we are adding a second assignee, _updateIssueFromWebhook won't update the assignedAt timestamp
+ _issues[payload.issue.number.toInt()]!.assignedAt = now;
+ } else if (payload.action.toString() == 'opened' || payload.action.toString() == 'reopened') {
+ _issues[payload.issue.number.toInt()]!.openedAt = now;
+ } else if (payload.action.toString() == 'labeled') {
+ final String label = payload.label.name.toString();
+ final String? team = getTeamFor(GitHubSettings.triagedPrefix, label);
+ if (team != null) {
+ final Set<String> teams = getTeamsFor(GitHubSettings.teamPrefix, issue.labels);
+ if (teams.length == 1) {
+ if (teams.single == team) {
+ issue.triagedAt = now;
+ }
+ }
+ }
+ }
+ }
+ }
+ case 'membership':
+ if (payload.team.slug.toString() == '${GitHubSettings.organization}/${GitHubSettings.teamName}') {
+ switch (payload.action.toString()) {
+ case 'added':
+ _contributors.add(payload.member.login.toString());
+ case 'removed':
+ _contributors.remove(payload.member.login.toString());
+ }
+ }
+ }
+ await _write();
+ }
+
+ // Called when we get a webhook message that we've established is an
+ // interesting update to an issue.
+ // Attempts to build up and/or update the data for an issue based on
+ // the data in a change event. This will be approximate until we can actually
+ // scan the issue properly in _updateStoreInBackground.
+ IssueStats? _updateIssueFromWebhook(String user, dynamic data, DateTime now) {
+ final int number = data.number.toInt();
+ if (number > _highestKnownIssue) {
+ _highestKnownIssue = number;
+ }
+ if (data.state.toString() == 'closed') {
+ _issues.remove(number);
+ _pendingCleanupIssues.remove(number);
+ if (number == _selfTestIssue) {
+ _selfTestIssue = null;
+ _selfTestClosedDate = now;
+ }
+ return null;
+ }
+ final IssueStats issue = _issues.putIfAbsent(number, IssueStats.new);
+ final Set<String> newLabels = <String>{};
+ for (final dynamic label in data.labels.asIterable()) {
+ final String name = label.name.toString();
+ if (GitHubSettings.isRelevantLabel(name)) {
+ newLabels.add(name);
+ }
+ }
+ issue.labels = newLabels;
+ final Set<String> assignees = <String>{};
+ for (final dynamic assignee in data.assignees.asIterable()) {
+ assignees.add(assignee.login.toString());
+ }
+ final String reporter = data.user.login.toString();
+ if (assignees.isEmpty) {
+ issue.lastAssigneeTouch = null;
+ issue.assignedAt = null;
+ issue.assignedToTeamMemberReporter = false;
+ } else {
+ issue.assignedAt ??= now;
+ if (assignees.contains(user)) {
+ issue.lastAssigneeTouch = now;
+ }
+ issue.assignedToTeamMemberReporter = assignees.contains(reporter) && _contributors.contains(reporter);
+ }
+ if (_contributors.contains(user)) {
+ issue.lastContributorTouch = now;
+ }
+ if (!data.locked.toBoolean()) {
+ issue.lockedAt = null;
+ } else {
+ issue.lockedAt ??= now;
+ }
+ final Set<String> teams = getTeamsFor(GitHubSettings.triagedPrefix, newLabels);
+ if (teams.isEmpty) {
+ issue.thumbsAtTriageTime = null;
+ issue.triagedAt = null;
+ }
+ _pendingCleanupIssues[number] = now;
+ return issue;
+ }
+
+ Future<void> _updateDiscordFromWebhook(String event, dynamic payload) async {
+ if (GitHubSettings.knownBots.contains(payload.sender.login.toString())) {
+ return;
+ }
+ switch (event) {
+ case 'star':
+ switch (payload.action.toString()) {
+ case 'created':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** starred ${payload.repository.full_name}',
+ channel: DiscordChannels.github2,
+ emoji: UnicodeEmoji('🌟'),
+ log: log,
+ );
+ }
+ case 'label':
+ switch (payload.action.toString()) {
+ case 'created':
+ String message;
+ if (payload.label.description.toString().isEmpty) {
+ message = '**@${payload.sender.login}** created a new label in ${payload.repository.full_name}, `${payload.label.name}`, but did not give it a description!';
+ } else {
+ message = '**@${payload.sender.login}** created a new label in ${payload.repository.full_name}, `${payload.label.name}`, with the description "${payload.label.description}".';
+ }
+ await sendDiscordMessage(
+ discord: discord,
+ body: message,
+ channel: DiscordChannels.hiddenChat,
+ embedTitle: '${payload.label.name}',
+ embedDescription: '${payload.label.description}',
+ embedColor: '${payload.label.color}',
+ log: log,
+ );
+ }
+ case 'pull_request':
+ switch (payload.action.toString()) {
+ case 'closed':
+ final bool merged = payload.pull_request.merged_at.toScalar() != null;
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** ${ merged ? "merged" : "closed" } *${payload.pull_request.title}* (${payload.pull_request.html_url})',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ case 'opened':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** submitted a new pull request: **${payload.pull_request.title}** (${payload.repository.full_name} #${payload.pull_request.number.toInt()})\n${stripBoilerplate(payload.pull_request.body.toString())}',
+ suffix: '*${payload.pull_request.html_url}*',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ }
+ case 'pull_request_review':
+ switch (payload.action.toString()) {
+ case 'submitted':
+ switch (payload.review.state.toString()) {
+ case 'approved':
+ await sendDiscordMessage(
+ discord: discord,
+ body: payload.review.body.toString().isEmpty ?
+ '**@${payload.sender.login}** gave **LGTM** for *${payload.pull_request.title}* (${payload.pull_request.html_url})' :
+ '**@${payload.sender.login}** gave **LGTM** for *${payload.pull_request.title}* (${payload.pull_request.html_url}): ${stripBoilerplate(payload.review.body.toString(), inline: true)}',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ }
+ }
+ case 'pull_request_review_comment':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** wrote: ${stripBoilerplate(payload.comment.body.toString(), inline: true)}',
+ suffix: '*${payload.comment.html_url} ${payload.pull_request.title}*',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ case 'issue_comment':
+ switch (payload.action.toString()) {
+ case 'created':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** wrote: ${stripBoilerplate(payload.comment.body.toString(), inline: true)}',
+ suffix: '*${payload.comment.html_url} ${payload.issue.title}*',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ }
+ case 'issues':
+ switch (payload.action.toString()) {
+ case 'closed':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** closed *${payload.issue.title}* (${payload.issue.html_url})',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ case 'reopened':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** reopened *${payload.issue.title}* (${payload.issue.html_url})',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ case 'opened':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** filed a new issue: **${payload.issue.title}** (${payload.repository.full_name} #${payload.issue.number.toInt()})\n${stripBoilerplate(payload.issue.body.toString())}',
+ suffix: '*${payload.issue.html_url}*',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ bool isDesignDoc = false;
+ for (final dynamic label in payload.issue.labels.asIterable()) {
+ final String name = label.name.toString();
+ if (name == GitHubSettings.designDoc) {
+ isDesignDoc = true;
+ break;
+ }
+ }
+ if (isDesignDoc) {
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** wrote a new design doc: **${payload.issue.title}**\n${stripBoilerplate(payload.issue.body.toString())}',
+ suffix: '*${payload.issue.html_url}*',
+ channel: DiscordChannels.hiddenChat,
+ log: log,
+ );
+ }
+ case 'locked':
+ case 'unlocked':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** ${payload.action} ${payload.issue.html_url} - ${payload.issue.title}',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ }
+ case 'membership':
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** ${payload.action} user **@${payload.member.login}** (${payload.team.name})',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ case 'gollum':
+ for (final dynamic page in payload.pages.asIterable()) {
+ // sadly the commit message doesn't get put into the event payload
+ await sendDiscordMessage(
+ discord: discord,
+ body: '**@${payload.sender.login}** ${page.action} the **${page.title}** wiki page',
+ suffix: '*${page.html_url}*',
+ channel: DiscordChannels.github2,
+ log: log,
+ );
+ }
+ }
+ }
+
+ // This is called every few seconds to update one issue in our store.
+ // We do this because (a) initially, we don't have any data so we need
+ // to fill our database somehow, and (b) thereafter, we might go out of
+ // sync if we miss an event, e.g. due to network issues.
+ Future<void> _updateStoreInBackground() async {
+ await _updateStoreInBackgroundForIssue(_currentBackgroundIssue);
+ _currentBackgroundIssue -= 1;
+ if (_currentBackgroundIssue <= 0) {
+ _currentBackgroundIssue = _highestKnownIssue;
+ }
+ await _write();
+ await Future<void>.delayed(Timings.backgroundUpdatePeriod);
+ scheduleMicrotask(_updateStoreInBackground);
+ }
+
+ Future<void> _updateStoreInBackgroundForIssue(int number) async {
+ try {
+ await _githubReady(0.5);
+ final Issue githubIssue = await github.issues.get(GitHubSettings.primaryRepository, number);
+ if (githubIssue.pullRequest == null && githubIssue.isOpen) {
+ final String? reporter = githubIssue.user?.login;
+ bool open = true;
+ final Set<String> assignees = <String>{};
+ final Set<String> labels = <String>{};
+ DateTime? lastContributorTouch;
+ DateTime? lastAssigneeTouch;
+ DateTime? openedAt = githubIssue.createdAt;
+ DateTime? lockedAt;
+ DateTime? assignedAt;
+ DateTime? triagedAt;
+ DateTime? lastChange;
+ await _githubReady();
+ await for (final TimelineEvent event in github.issues.listTimeline(GitHubSettings.primaryRepository, number)) {
+ String? user;
+ DateTime? time;
+ // event.actor could be null if the original user was deleted (shows as "ghost" in GitHub's web UI)
+ // see e.g. https://github.com/flutter/flutter/issues/93070
+ switch (event.event) {
+ case 'renamed': // The issue or pull request title was changed.
+ case 'commented':
+ user = event.actor?.login;
+ time = event.createdAt;
+ case 'locked': // The issue or pull request was locked.
+ user = event.actor?.login;
+ time = event.createdAt;
+ lockedAt = time;
+ case 'unlocked': // The issue was unlocked.
+ user = event.actor?.login;
+ time = event.createdAt;
+ lockedAt = null;
+ case 'assigned':
+ event as AssigneeEvent;
+ if (event.assignee != null && event.assignee!.login != null) {
+ user = event.actor?.login;
+ time = event.createdAt;
+ assignees.add(event.assignee!.login!);
+ assignedAt = time;
+ }
+ case 'unassigned':
+ event as AssigneeEvent;
+ user = event.actor?.login;
+ time = event.createdAt;
+ if (event.assignee != null) {
+ assignees.remove(event.assignee!.login);
+ if (assignees.isEmpty) {
+ assignedAt = null;
+ lastAssigneeTouch = null;
+ }
+ }
+ case 'labeled':
+ event as LabelEvent;
+ user = event.actor?.login;
+ time = event.createdAt;
+ final String label = event.label!.name;
+ if (GitHubSettings.isRelevantLabel(label, ignorePriorities: true)) {
+ // we add the priority labels later to avoid confusion from the renames
+ labels.add(label);
+ }
+ final String? triagedTeam = getTeamFor(GitHubSettings.triagedPrefix, label);
+ if (triagedTeam != null) {
+ final Set<String> teams = getTeamsFor(GitHubSettings.teamPrefix, labels);
+ if (teams.length == 1 && teams.single == triagedTeam) {
+ triagedAt = event.createdAt;
+ }
+ }
+ case 'unlabeled':
+ event as LabelEvent;
+ user = event.actor?.login;
+ time = event.createdAt;
+ final String label = event.label!.name;
+ labels.remove(label);
+ final Set<String> teams = getTeamsFor(GitHubSettings.teamPrefix, labels);
+ final Set<String> triagedTeams = getTeamsFor(GitHubSettings.triagedPrefix, labels);
+ if (teams.intersection(triagedTeams).isEmpty) {
+ triagedAt = null;
+ }
+ case 'closed':
+ user = event.actor?.login;
+ time = event.createdAt;
+ open = false;
+ case 'reopened':
+ user = event.actor?.login;
+ time = event.createdAt;
+ openedAt = event.createdAt;
+ open = true;
+ }
+ if (user != null) {
+ assert(time != null);
+ if (_contributors.contains(user)) {
+ lastContributorTouch = time;
+ }
+ if (assignees.contains(user)) {
+ lastAssigneeTouch = time;
+ }
+ }
+ if (lastChange == null || (time != null && time.isAfter(lastChange))) {
+ lastChange = time;
+ }
+ await _githubReady();
+ }
+ if (open) {
+ // Because we renamed some of the labels, we can't trust the
+ // historical names we get from the timeline. We have to use the
+ // actual current labels from the githubIssue.
+ // Also, there might be missing labels because the timeline doesn't
+ // include the issue's original labels from when the issue was filed.
+ final Set<String> actualLabels = githubIssue.labels
+ .map<String>((IssueLabel label) => label.name)
+ .where(GitHubSettings.isRelevantLabel)
+ .toSet();
+ for (final String label in actualLabels.difference(labels)) {
+ // could have been renamed, but let's assume it was added when the issue was created (and never removed).
+ final String? triagedTeam = getTeamFor(GitHubSettings.triagedPrefix, label);
+ if (triagedTeam != null) {
+ final Set<String> teams = getTeamsFor(GitHubSettings.teamPrefix, actualLabels);
+ if (teams.length == 1 && teams.single == triagedTeam) {
+ triagedAt = openedAt;
+ }
+ }
+ }
+ final IssueStats issue = _issues.putIfAbsent(number, IssueStats.new);
+ issue.lastContributorTouch = lastContributorTouch;
+ issue.lastAssigneeTouch = lastAssigneeTouch;
+ issue.labels = actualLabels;
+ issue.openedAt = openedAt;
+ issue.lockedAt = lockedAt;
+ assert((assignedAt != null) == (assignees.isNotEmpty));
+ issue.assignedAt = assignedAt;
+ issue.assignedToTeamMemberReporter = reporter != null && assignees.contains(reporter) && _contributors.contains(reporter);
+ issue.thumbs = githubIssue.reactions?.plusOne ?? 0;
+ issue.triagedAt = triagedAt;
+ if (triagedAt != null) {
+ if (issue.thumbsAtTriageTime == null) {
+ int thumbsAtTriageTime = 0;
+ await _githubReady();
+ await for (final Reaction reaction in github.issues.listReactions(GitHubSettings.primaryRepository, number)) {
+ if (reaction.createdAt != null && reaction.createdAt!.isAfter(triagedAt)) {
+ break;
+ }
+ if (reaction.content == '+1') {
+ thumbsAtTriageTime += 1;
+ }
+ await _githubReady();
+ }
+ issue.thumbsAtTriageTime = thumbsAtTriageTime;
+ }
+ } else {
+ issue.thumbsAtTriageTime = null;
+ }
+ } else {
+ _issues.remove(number);
+ _pendingCleanupIssues.remove(number);
+ }
+ if (!_pendingCleanupIssues.containsKey(number)) {
+ _pendingCleanupIssues[number] = lastChange ?? DateTime.timestamp();
+ }
+ } else {
+ if (_selfTestIssue == number) {
+ _selfTestIssue = null;
+ _selfTestClosedDate = githubIssue.closedAt;
+ }
+ }
+ } on NotFound {
+ _issues.remove(number);
+ _pendingCleanupIssues.remove(number);
+ } catch (e, s) {
+ log('Failed to perform background update of issue #$number: $e (${e.runtimeType})\n$s');
+ }
+ }
+
+ bool _cleaning = false;
+ // This is called periodically to look at recently-updated issues.
+ // This lets us enforce invariants but only after humans have had a chance
+ // to do whatever it is they are doing on the issue.
+ Future<void> _performCleanups([Timer? timer]) async {
+ final DateTime now = DateTime.timestamp();
+ _cleanupTimer?.cancel();
+ _nextCleanup = now.add(Timings.cleanupUpdatePeriod);
+ _cleanupTimer = Timer(Timings.cleanupUpdatePeriod, _performCleanups);
+ if (_cleaning) {
+ return;
+ }
+ try {
+ _cleaning = true;
+ _lastCleanupStart = now;
+ final DateTime refeedThreshold = now.subtract(Timings.refeedDelay);
+ final DateTime cleanupThreshold = now.subtract(Timings.cleanupUpdateDelay);
+ final DateTime staleThreshold = now.subtract(Timings.timeUntilStale);
+ final List<int> issues = _pendingCleanupIssues.keys.toList();
+ for (final int number in issues) {
+ try {
+ if (_pendingCleanupIssues.containsKey(number) && _pendingCleanupIssues[number]!.isBefore(cleanupThreshold)) {
+ assert(_issues.containsKey(number));
+ final IssueStats issue = _issues[number]!;
+ final Set<String> labelsToRemove = <String>{};
+ final List<String> messages = <String>[];
+ // PRIORITY LABELS
+ final Set<String> priorities = issue.labels.intersection(GitHubSettings.priorities);
+ if (priorities.length > 1) {
+ // When an issue has multiple priorities, remove all but the highest.
+ for (final String priority in GitHubSettings.priorities.toList().reversed) {
+ if (priorities.contains(priority)) {
+ labelsToRemove.add(priority);
+ priorities.remove(priority);
+ }
+ if (priorities.length == 1) {
+ break;
+ }
+ }
+ }
+ // TEAM LABELS
+ final Set<String> teams = getTeamsFor(GitHubSettings.teamPrefix, issue.labels);
+ final Set<String> triaged = getTeamsFor(GitHubSettings.triagedPrefix, issue.labels);
+ final Set<String> fyi = getTeamsFor(GitHubSettings.fyiPrefix, issue.labels);
+ if (teams.length > 1 && number != _selfTestIssue) {
+ // Issues should only have a single "team-foo" label.
+ // When this is violated, we remove all of them to send the issue back to front-line triage.
+ messages.add(
+ 'Issue is assigned to multiple teams (${teams.join(", ")}). '
+ 'Please ensure the issue has only one `${GitHubSettings.teamPrefix}*` label at a time. '
+ 'Use `${GitHubSettings.fyiPrefix}*` labels to have another team look at the issue without reassigning it.'
+ );
+ for (final String team in teams) {
+ labelsToRemove.add('${GitHubSettings.teamPrefix}$team');
+ // Also remove the labels we'd end up removing below, to avoid having confusing messages.
+ if (triaged.contains(team)) {
+ labelsToRemove.add('${GitHubSettings.triagedPrefix}$team');
+ triaged.remove(team);
+ }
+ if (fyi.contains(team)) {
+ labelsToRemove.add('${GitHubSettings.fyiPrefix}$team');
+ fyi.remove(team);
+ }
+ }
+ teams.clear();
+ }
+ for (final String team in fyi.toList()) {
+ if (teams.contains(team)) {
+ // Remove redundant fyi-* labels.
+ messages.add('The `${GitHubSettings.fyiPrefix}$team` label is redundant with the `${GitHubSettings.teamPrefix}$team` label.');
+ labelsToRemove.add('${GitHubSettings.fyiPrefix}$team');
+ fyi.remove(team);
+ } else if (triaged.contains(team)) {
+ // If an fyi-* label has been acknowledged by a triaged-* label, we can remove them both.
+ labelsToRemove.add('${GitHubSettings.fyiPrefix}$team');
+ labelsToRemove.add('${GitHubSettings.triagedPrefix}$team');
+ fyi.remove(team);
+ triaged.remove(team);
+ }
+ }
+ for (final String team in triaged.toList()) {
+ // Remove redundant triaged-* labels.
+ if (!teams.contains(team)) {
+ messages.add(
+ 'The `${GitHubSettings.triagedPrefix}$team` label is irrelevant if '
+ 'there is no `${GitHubSettings.teamPrefix}$team` label or `${GitHubSettings.fyiPrefix}$team` label.'
+ );
+ labelsToRemove.add('${GitHubSettings.triagedPrefix}$team');
+ triaged.remove(team);
+ }
+ }
+ assert(teams.length <= 1 || number == _selfTestIssue);
+ assert(triaged.length <= teams.length);
+ if (triaged.isNotEmpty && priorities.isEmpty && number != _selfTestIssue) {
+ assert(triaged.length == 1);
+ final String team = triaged.single;
+ if (!_lastRefeedByTime.containsKey(team) || _lastRefeedByTime[team]!.isBefore(refeedThreshold)) {
+ messages.add(
+ 'This issue is missing a priority label. '
+ 'Please set a priority label when adding the `${GitHubSettings.triagedPrefix}$team` label.'
+ );
+ _lastRefeedByTime[team] = now;
+ labelsToRemove.add('${GitHubSettings.triagedPrefix}$team');
+ triaged.remove(team);
+ assert(triaged.isEmpty);
+ }
+ }
+ // STALE THUMBS UP LABEL
+ if (triaged.isNotEmpty && issue.labels.contains(GitHubSettings.thumbsUpLabel)) {
+ labelsToRemove.add(GitHubSettings.thumbsUpLabel);
+ }
+ // STALE STALE ISSUE LABEL
+ if (issue.labels.contains(GitHubSettings.staleIssueLabel) &&
+ ((issue.lastContributorTouch != null && issue.lastContributorTouch!.isAfter(staleThreshold)) ||
+ (issue.assignedAt == null))) {
+ labelsToRemove.add(GitHubSettings.staleIssueLabel);
+ }
+ // LOCKED STATUS
+ final bool shouldUnlock = issue.openedAt != null && issue.lockedAt != null && issue.lockedAt!.isBefore(issue.openedAt!);
+ // APPLY PENDING CHANGES
+ if ((labelsToRemove.isNotEmpty || messages.isNotEmpty || shouldUnlock) && await isActuallyOpen(number)) {
+ for (final String label in labelsToRemove) {
+ log('Removing label "$label" on issue #$number');
+ await _githubReady();
+ await github.issues.removeLabelForIssue(GitHubSettings.primaryRepository, number, label);
+ issue.labels.remove(label);
+ }
+ if (messages.isNotEmpty) {
+ log('Posting message on issue #$number:\n ${messages.join("\n ")}');
+ await _githubReady();
+ await github.issues.createComment(GitHubSettings.primaryRepository, number, messages.join('\n'));
+ }
+ if (shouldUnlock) {
+ log('Unlocking issue #$number (reopened after being locked)');
+ await _githubReady();
+ await github.issues.unlock(GitHubSettings.primaryRepository, number);
+ }
+ }
+ _pendingCleanupIssues.remove(number);
+ }
+ } catch (e, s) {
+ log('Failure in cleanup for #$number: $e (${e.runtimeType})\n$s');
+ }
+ }
+ } finally {
+ _cleaning = false;
+ _lastCleanupEnd = DateTime.timestamp();
+ }
+ }
+
+ bool _tidying = false;
+ // This is called periodically to enforce long-term policies (things that
+ // only apply after an issue has been in a particular state for weeks).
+ Future<void> _performLongTermTidying([Timer? timer]) async {
+ _tidyTimer?.cancel();
+ final DateTime now = DateTime.timestamp();
+ _nextTidy = now.add(Timings.longTermTidyingPeriod);
+ _tidyTimer = Timer(Timings.longTermTidyingPeriod, _performLongTermTidying);
+ if (_tidying) {
+ return;
+ }
+ try {
+ _tidying = true;
+ _lastTidyStart = now;
+ final DateTime staleThreshold = now.subtract(Timings.timeUntilStale);
+ final DateTime reallyStaleThreshold = now.subtract(Timings.timeUntilReallyStale);
+ final DateTime unlockThreshold = now.subtract(Timings.timeUntilUnlock);
+ final DateTime refeedThreshold = now.subtract(Timings.refeedDelay);
+ int number = 1;
+ while (number < _highestKnownIssue) {
+ try {
+ if (_issues.containsKey(number) && !_pendingCleanupIssues.containsKey(number) && number != _selfTestIssue) {
+ // Tidy the issue.
+ final IssueStats issue = _issues[number]!;
+ final Set<String> triagedTeams = getTeamsFor(GitHubSettings.triagedPrefix, issue.labels);
+ final Set<String> assignedTeams = getTeamsFor(GitHubSettings.teamPrefix, issue.labels);
+ // Check for assigned issues that aren't making progress.
+ if (issue.assignedAt != null &&
+ issue.lastContributorTouch != null &&
+ issue.lastContributorTouch!.isBefore(staleThreshold) &&
+ (!issue.assignedToTeamMemberReporter || issue.labels.contains(GitHubSettings.designDoc))) {
+ await _githubReady();
+ final Issue actualIssue = await github.issues.get(GitHubSettings.primaryRepository, number);
+ if (actualIssue.assignees != null && actualIssue.assignees!.isNotEmpty && isActuallyOpenFromRawIssue(actualIssue)) {
+ final String assignee = actualIssue.assignees!.map((User user) => '@${user.login}').join(' and ');
+ if (!issue.labels.contains(GitHubSettings.staleIssueLabel)) {
+ log('Issue #$number is assigned to $assignee but not making progress; adding comment.');
+ await _githubReady();
+ await github.issues.addLabelsToIssue(GitHubSettings.primaryRepository, number, <String>[GitHubSettings.staleIssueLabel]);
+ issue.labels.add(GitHubSettings.staleIssueLabel);
+ await _githubReady();
+ await github.issues.createComment(GitHubSettings.primaryRepository, number,
+ 'This issue is assigned to $assignee but has had no recent status updates. '
+ 'Please consider unassigning this issue if it is not going to be addressed in the near future. '
+ 'This allows people to have a clearer picture of what work is actually planned. Thanks!',
+ );
+ } else if (issue.lastContributorTouch!.isBefore(reallyStaleThreshold)) {
+ bool skip = false;
+ String team = 'primary triage';
+ if (assignedTeams.length == 1) { // if it's more, then cleanup will take care of it
+ team = assignedTeams.single;
+ if (!_lastRefeedByTime.containsKey(team) || _lastRefeedByTime[team]!.isBefore(refeedThreshold)) {
+ _lastRefeedByTime[team] = now;
+ } else {
+ skip = true;
+ }
+ }
+ if (!skip) {
+ log('Issue #$number is assigned to $assignee but still not making progress (for ${now.difference(issue.lastContributorTouch!)}); sending back to triage (for $team team).');
+ for (final String triagedTeam in triagedTeams) {
+ await _githubReady();
+ await github.issues.removeLabelForIssue(GitHubSettings.primaryRepository, number, '${GitHubSettings.triagedPrefix}$triagedTeam');
+ issue.labels.remove('${GitHubSettings.triagedPrefix}$triagedTeam');
+ }
+ await _githubReady();
+ await github.issues.edit(GitHubSettings.primaryRepository, number, IssueRequest(assignees: const <String>[]));
+ await _githubReady();
+ await github.issues.createComment(GitHubSettings.primaryRepository, number,
+ 'This issue was assigned to $assignee but has had no status updates in a long time. '
+ 'To remove any ambiguity about whether the issue is being worked on, the assignee was removed.',
+ );
+ await _githubReady();
+ await github.issues.removeLabelForIssue(GitHubSettings.primaryRepository, number, GitHubSettings.staleIssueLabel);
+ issue.labels.remove(GitHubSettings.staleIssueLabel);
+ }
+ }
+ }
+ }
+ // Check for P1 issues that aren't making progress.
+ // We currently rate-limit this to only a few per week so that teams don't get overwhelmed.
+ if (issue.assignedAt == null &&
+ issue.labels.contains('P1') &&
+ issue.lastContributorTouch != null &&
+ issue.lastContributorTouch!.isBefore(staleThreshold) &&
+ triagedTeams.length == 1) {
+ final String team = triagedTeams.single;
+ if ((!_lastRefeedByTime.containsKey(team) || _lastRefeedByTime[team]!.isBefore(refeedThreshold)) && await isActuallyOpen(number)) {
+ log('Issue #$number is P1 but not assigned and not making progress; removing triage label and adding comment.');
+ await _githubReady();
+ await github.issues.removeLabelForIssue(GitHubSettings.primaryRepository, number, '${GitHubSettings.triagedPrefix}${triagedTeams.single}');
+ issue.labels.remove('${GitHubSettings.triagedPrefix}${triagedTeams.single}');
+ await _githubReady();
+ await github.issues.createComment(GitHubSettings.primaryRepository, number, GitHubSettings.staleP1Message);
+ _lastRefeedByTime[team] = now;
+ }
+ }
+ // Unlock issues after a timeout.
+ if (issue.lockedAt != null && issue.lockedAt!.isBefore(unlockThreshold) &&
+ !issue.labels.contains(GitHubSettings.permanentlyLocked) &&
+ await isActuallyOpen(number)) {
+ log('Issue #$number has been locked for too long, unlocking.');
+ await _githubReady();
+ await github.issues.unlock(GitHubSettings.primaryRepository, number);
+ }
+ // Flag issues that have gained a lot of thumbs-up.
+ // We don't consider refeedThreshold for this because it should be relatively rare and
+ // is always noteworthy when it happens.
+ if (issue.thumbsAtTriageTime != null && triagedTeams.length == 1 &&
+ issue.thumbs >= issue.thumbsAtTriageTime! * GitHubSettings.thumbsThreshold &&
+ issue.thumbs >= GitHubSettings.thumbsMinimum &&
+ await isActuallyOpen(number)) {
+ log('Issue #$number has gained a lot of thumbs-up, flagging for retriage.');
+ await _githubReady();
+ await github.issues.removeLabelForIssue(GitHubSettings.primaryRepository, number, '${GitHubSettings.triagedPrefix}${triagedTeams.single}');
+ issue.labels.remove('${GitHubSettings.triagedPrefix}${triagedTeams.single}');
+ await _githubReady();
+ await github.issues.addLabelsToIssue(GitHubSettings.primaryRepository, number, <String>[GitHubSettings.thumbsUpLabel]);
+ issue.labels.add(GitHubSettings.thumbsUpLabel);
+ }
+ }
+ } catch (e, s) {
+ log('Failure in tidying for #$number: $e (${e.runtimeType})\n$s');
+ }
+ number += 1;
+ }
+ try {
+ if (_selfTestIssue == null) {
+ if (_selfTestClosedDate == null || _selfTestClosedDate!.isBefore(now.subtract(Timings.selfTestPeriod))) {
+ await _githubReady();
+ final Issue issue = await github.issues.create(GitHubSettings.primaryRepository, IssueRequest(
+ title: 'Triage process self-test',
+ body: 'This is a test of our triage processes.\n'
+ '\n'
+ 'Please handle this issue the same way you would a normal valid but low-priority issue.\n'
+ '\n'
+ 'For more details see https://github.com/flutter/flutter/wiki/Triage',
+ labels: <String>[
+ ...GitHubSettings.teams.map((String team) => '${GitHubSettings.teamPrefix}$team'),
+ 'P2',
+ ],
+ ));
+ _selfTestIssue = issue.number;
+ _selfTestClosedDate = null;
+ log('Filed self-test issue #$_selfTestIssue.');
+ }
+ } else if (_issues.containsKey(_selfTestIssue)) {
+ final IssueStats issue = _issues[_selfTestIssue]!;
+ if (!issue.labels.contains(GitHubSettings.willNeedAdditionalTriage) &&
+ issue.lastContributorTouch!.isBefore(now.subtract(Timings.selfTestWindow)) &&
+ await isActuallyOpen(_selfTestIssue!)) {
+ log('Flagging self-test issue #$_selfTestIssue for critical triage.');
+ for (final String team in getTeamsFor(GitHubSettings.triagedPrefix, issue.labels)) {
+ await _githubReady();
+ await github.issues.removeLabelForIssue(GitHubSettings.primaryRepository, _selfTestIssue!, '${GitHubSettings.teamPrefix}$team');
+ issue.labels.remove('${GitHubSettings.teamPrefix}$team');
+ await _githubReady();
+ await github.issues.removeLabelForIssue(GitHubSettings.primaryRepository, _selfTestIssue!, '${GitHubSettings.triagedPrefix}$team');
+ issue.labels.remove('${GitHubSettings.triagedPrefix}$team');
+ }
+ await _githubReady();
+ await github.issues.addLabelsToIssue(GitHubSettings.primaryRepository, _selfTestIssue!, <String>[GitHubSettings.willNeedAdditionalTriage]);
+ issue.labels.add(GitHubSettings.willNeedAdditionalTriage);
+ }
+ }
+ } catch (e, s) {
+ log('Failure in self-test logic: $e (${e.runtimeType})\n$s');
+ }
+ } finally {
+ _tidying = false;
+ _lastTidyEnd = DateTime.timestamp();
+ }
+ }
+
+ Future<bool> isActuallyOpen(int number) async {
+ if (!_issues.containsKey(number)) {
+ return false;
+ }
+ await _githubReady();
+ final Issue rawIssue = await github.issues.get(GitHubSettings.primaryRepository, number);
+ return isActuallyOpenFromRawIssue(rawIssue);
+ }
+
+ bool isActuallyOpenFromRawIssue(Issue rawIssue) {
+ if (rawIssue.isClosed) {
+ log('Issue #${rawIssue.number} was unexpectedly found to be closed when doing cleanup.');
+ _issues.remove(rawIssue.number);
+ _pendingCleanupIssues.remove(rawIssue.number);
+ if (rawIssue.number == _selfTestIssue) {
+ _selfTestIssue = null;
+ }
+ return false;
+ }
+ return true;
+ }
+
+ static String? getTeamFor(String prefix, String label) {
+ if (label.startsWith(prefix)) {
+ return label.substring(prefix.length);
+ }
+ return null;
+ }
+
+ static Set<String> getTeamsFor(String prefix, Set<String> labels) {
+ if (labels.isEmpty) {
+ return const <String>{};
+ }
+ Set<String>? result;
+ for (final String label in labels) {
+ final String? team = getTeamFor(prefix, label);
+ if (team != null) {
+ result ??= <String>{};
+ result.add(team);
+ }
+ }
+ return result ?? const <String>{};
+ }
+}
+
+int secondsSinceEpoch(DateTime time) => time.millisecondsSinceEpoch ~/ 1000;
+
+Future<String> obtainGitHubCredentials(Secrets secrets, http.Client client) async {
+ // https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-json-web-token-jwt-for-a-github-app
+ final DateTime now = DateTime.timestamp();
+ final String jwt = JWT(<String, dynamic>{
+ 'iat': secondsSinceEpoch(now.subtract(const Duration(seconds: 60))),
+ 'exp': secondsSinceEpoch(now.add(const Duration(minutes: 10))),
+ 'iss': await secrets.githubAppId,
+ }).sign(
+ RSAPrivateKey(await secrets.githubAppKey),
+ algorithm: JWTAlgorithm.RS256,
+ noIssueAt: true,
+ );
+ final String installation = await secrets.githubInstallationId;
+ final dynamic response = Json.parse((await client.post(
+ Uri.parse('https://api.github.com/app/installations/$installation/access_tokens'),
+ body: '{}',
+ headers: <String, String>{
+ 'Accept': 'application/vnd.github+json',
+ 'Authorization': 'Bearer $jwt', // should not need escaping, base64 is safe in a header value
+ 'X-GitHub-Api-Version': '2022-11-28',
+ },
+ )).body);
+ return response.token.toString();
+}
+
+Future<void> maintainGitHubCredentials(GitHub github, Secrets secrets, Engine engine, http.Client client) async {
+ try {
+ await Future<void>.delayed(Timings.credentialsUpdatePeriod);
+ github.auth = Authentication.withToken(await obtainGitHubCredentials(secrets, client));
+ } catch (e, s) {
+ engine.log('Failed to maintain GitHub credentials: $e (${e.runtimeType})\n$s');
+ }
+}
+
+DateTime _laterOf(DateTime a, DateTime b) {
+ if (a.isAfter(b)) {
+ return a;
+ }
+ return b;
+}
+
+Future<DateTime> getCertificateTimestamp(Secrets secrets) async {
+ return _laterOf(
+ _laterOf(
+ await secrets.serverCertificateModificationDate,
+ await secrets.serverIntermediateCertificatesModificationDate,
+ ),
+ await secrets.serverKeyModificationDate,
+ );
+}
+
+Future<SecurityContext> loadCertificates(Secrets secrets) async {
+ return SecurityContext()
+ ..useCertificateChainBytes(
+ await secrets.serverCertificate +
+ await secrets.serverIntermediateCertificates,
+ )
+ ..usePrivateKeyBytes(await secrets.serverKey);
+}
+
+final bool usingAppEngine = Platform.environment.containsKey('APPENGINE_RUNTIME');
+
+Future<Engine> startEngine(void Function()? onChange) async {
+ final Secrets secrets = Secrets();
+
+ final INyxx discord = NyxxFactory.createNyxxRest(
+ await secrets.discordToken,
+ GatewayIntents.none,
+ Snowflake.value(await secrets.discordAppId),
+ );
+ await discord.connect();
+
+ final http.Client httpClient = http.Client();
+
+ final GitHub github = GitHub(
+ client: httpClient,
+ auth: Authentication.withToken(await obtainGitHubCredentials(secrets, httpClient)),
+ );
+
+ final Engine engine = await Engine.initialize(
+ webhookSecret: await secrets.githubWebhookSecret,
+ discord: discord,
+ github: github,
+ onChange: onChange,
+ secrets: secrets,
+ );
+
+ if (usingAppEngine) {
+ await withAppEngineServices(() async {
+ runAppEngine(engine.handleRequest); // ignore: unawaited_futures
+ while (true) {
+ await maintainGitHubCredentials(github, secrets, engine, httpClient);
+ }
+ });
+ } else {
+ scheduleMicrotask(() async {
+ DateTime activeCertificateTimestamp = await getCertificateTimestamp(secrets);
+ SecurityContext securityContext = await loadCertificates(secrets);
+ while (true) {
+ final HttpServer server = await HttpServer.bindSecure(InternetAddress.anyIPv4, port, securityContext);
+ server.listen(engine.handleRequest);
+ DateTime pendingCertificateTimestamp = activeCertificateTimestamp;
+ do {
+ await maintainGitHubCredentials(github, secrets, engine, httpClient);
+ pendingCertificateTimestamp = await getCertificateTimestamp(secrets);
+ } while (pendingCertificateTimestamp == activeCertificateTimestamp);
+ activeCertificateTimestamp = pendingCertificateTimestamp;
+ engine.log('Updating TLS credentials...');
+ securityContext = await loadCertificates(secrets);
+ await engine.shutdown(server.close);
+ // There's a race condition here where we might miss messages because the server is down.
+ }
+ });
+ }
+
+ return engine;
+}
diff --git a/triage_bot/lib/json.dart b/triage_bot/lib/json.dart
new file mode 100644
index 0000000..60c7d56
--- /dev/null
+++ b/triage_bot/lib/json.dart
@@ -0,0 +1,257 @@
+// Copyright 2017 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+// The whole point of this file is to wrap dynamic calls in a pretense of type safety, so we use dynamic calls a lot.
+// Also the code uses a lot of one-line flow control so we don't bother wrapping them all in blocks.
+// ignore_for_file: avoid_dynamic_calls, curly_braces_in_flow_control_structures
+
+import 'dart:convert' as dart show json;
+import 'package:meta/meta.dart';
+
+@immutable
+class Json {
+ factory Json(dynamic input) {
+ if (input is Json)
+ return Json._wrap(input._value);
+ return Json._wrap(input);
+ }
+
+ factory Json.list(List<dynamic> input) {
+ return Json._raw(input.map<Json>(Json._wrap).toList());
+ }
+
+ // (This differs from "real" JSON in that we don't allow duplicate keys.)
+ factory Json.map(Map<dynamic, dynamic> input) {
+ final Map<String, Json> values = <String, Json>{};
+ input.forEach((dynamic key, dynamic value) {
+ final String name = key.toString();
+ assert(!values.containsKey(name), 'Json.map keys must be unique strings');
+ values[name] = Json._wrap(value);
+ });
+ return Json._raw(values);
+ }
+
+ factory Json.parse(String value) {
+ return Json(dart.json.decode(value));
+ }
+
+ const Json._raw(this._value);
+
+ factory Json._wrap(dynamic value) {
+ if (value == null)
+ return const Json._raw(null);
+ if (value is num)
+ return Json._raw(value.toDouble());
+ if (value is List)
+ return Json.list(value);
+ if (value is Map)
+ return Json.map(value);
+ if (value == true)
+ return const Json._raw(true);
+ if (value == false)
+ return const Json._raw(false);
+ if (value is Json)
+ return value;
+ return Json._raw(value.toString());
+ }
+
+ final dynamic _value;
+
+ dynamic unwrap() {
+ if (_value is Map)
+ return toMap();
+ if (_value is List)
+ return toList();
+ return _value;
+ }
+
+ bool get isMap => _value is Map;
+ bool get isList => _value is List;
+ bool get isScalar => _value == null || _value is num || _value is bool || _value is String;
+ Type get valueType => _value.runtimeType;
+
+ Map<String, dynamic> toMap() {
+ final Map<String, dynamic> values = <String, dynamic>{};
+ if (_value is Map) {
+ _value.forEach((String key, Json value) {
+ values[key] = value.unwrap();
+ });
+ } else if (_value is List) {
+ for (int index = 0; index < (_value as List<dynamic>).length; index += 1)
+ values[index.toString()] = _value[index].unwrap();
+ } else {
+ values['0'] = unwrap();
+ }
+ return values;
+ }
+
+ List<dynamic> toList() {
+ if (_value is Map)
+ return (_value as Map<String, Json>).values.map<dynamic>((Json value) => value.unwrap()).toList();
+ if (_value is List)
+ return (_value as List<Json>).map<dynamic>((Json value) => value.unwrap()).toList();
+ return <dynamic>[unwrap()];
+ }
+
+ dynamic toScalar() {
+ assert(isScalar, 'toScalar called on non-scalar. Check "isScalar" first.');
+ return _value;
+ }
+
+ Iterable<Json> asIterable() {
+ if (_value is Map)
+ return (_value as Map<String, Json>).values.toList();
+ if (_value is List)
+ return _value as List<Json>;
+ return const <Json>[];
+ }
+
+ double toDouble() => _value as double;
+
+ int toInt() => (_value as double).toInt();
+
+ bool toBoolean() => _value as bool;
+
+ @override
+ String toString() => _value.toString();
+
+ String toJson() {
+ return dart.json.encode(unwrap());
+ }
+
+ dynamic operator [](dynamic key) {
+ return _value[key];
+ }
+
+ void operator []=(dynamic key, dynamic value) {
+ _value[key] = Json._wrap(value);
+ }
+
+ bool hasKey(String key) {
+ return _value is Map && (_value as Map<String, Json>).containsKey(key);
+ }
+
+ @override
+ dynamic noSuchMethod(Invocation invocation) {
+ if (invocation.isGetter) {
+ final String name = _symbolName(invocation.memberName);
+ if (_value is Map) {
+ if ((_value as Map<String, Json>).containsKey(name))
+ return this[name];
+ return const Json._raw(null);
+ }
+ }
+ if (invocation.isSetter)
+ return this[_symbolName(invocation.memberName, stripEquals: true)] = invocation.positionalArguments[0];
+ return super.noSuchMethod(invocation);
+ }
+
+ // Workaround for https://github.com/dart-lang/sdk/issues/28372
+ String _symbolName(Symbol symbol, { bool stripEquals = false }) {
+ // WARNING: Assumes a fixed format for Symbol.toString which is *not*
+ // guaranteed anywhere.
+ final String s = '$symbol';
+ return s.substring(8, s.length - (2 + (stripEquals ? 1 : 0)));
+ }
+
+ bool operator <(Object other) {
+ if (other is Json)
+ return _value < other._value as bool;
+ return _value < other as bool;
+ }
+
+ bool operator <=(Object other) {
+ if (other is Json)
+ return _value <= other._value as bool;
+ return _value <= other as bool;
+ }
+
+ bool operator >(Object other) {
+ if (other is Json)
+ return _value > other._value as bool;
+ return _value > other as bool;
+ }
+
+ bool operator >=(Object other) {
+ if (other is Json)
+ return _value >= other._value as bool;
+ return _value >= other as bool;
+ }
+
+ dynamic operator -(Object other) {
+ if (other is Json)
+ return _value - other._value;
+ return _value - other;
+ }
+
+ dynamic operator +(Object other) {
+ if (other is Json)
+ return _value + other._value;
+ return _value + other;
+ }
+
+ dynamic operator /(Object other) {
+ if (other is Json)
+ return _value / other._value;
+ return _value / other;
+ }
+
+ dynamic operator ~/(Object other) {
+ if (other is Json)
+ return _value ~/ other._value;
+ return _value ~/ other;
+ }
+
+ dynamic operator *(Object other) {
+ if (other is Json)
+ return _value * other._value;
+ return _value * other;
+ }
+
+ dynamic operator %(Object other) {
+ if (other is Json)
+ return _value % other._value;
+ return _value % other;
+ }
+
+ dynamic operator |(Object other) {
+ if (other is Json)
+ return _value.toInt() | other._value.toInt();
+ return _value.toInt() | other;
+ }
+
+ dynamic operator ^(Object other) {
+ if (other is Json)
+ return _value.toInt() ^ other._value.toInt();
+ return _value.toInt() ^ other;
+ }
+
+ dynamic operator &(Object other) {
+ if (other is Json)
+ return _value.toInt() & other._value.toInt();
+ return _value.toInt() & other;
+ }
+
+ dynamic operator <<(Object other) {
+ if (other is Json)
+ return _value.toInt() << other._value.toInt();
+ return _value.toInt() << other;
+ }
+
+ dynamic operator >>(Object other) {
+ if (other is Json)
+ return _value.toInt() >> other._value.toInt();
+ return _value.toInt() >> other;
+ }
+
+ @override
+ bool operator ==(Object other) {
+ if (other is Json)
+ return _value == other._value;
+ return _value == other;
+ }
+
+ @override
+ int get hashCode => _value.hashCode;
+}
diff --git a/triage_bot/lib/utils.dart b/triage_bot/lib/utils.dart
new file mode 100644
index 0000000..cc439c1
--- /dev/null
+++ b/triage_bot/lib/utils.dart
@@ -0,0 +1,7 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+String hex(int byte) => byte.toRadixString(16).padLeft(2, '0');
+
+String s(int n) => n == 1 ? '' : 's';
diff --git a/triage_bot/pubspec.yaml b/triage_bot/pubspec.yaml
new file mode 100644
index 0000000..bf64364
--- /dev/null
+++ b/triage_bot/pubspec.yaml
@@ -0,0 +1,21 @@
+name: triage_bot
+description: Flutter's triage bot.
+version: 1.0.0
+repository: https://github.com/flutter/cocoon/
+publish_to: 'none'
+
+environment:
+ sdk: ^3.0.0
+
+dependencies:
+ appengine: ^0.13.5
+ crypto: ^3.0.3
+ dart_jsonwebtoken: ^2.8.1
+ github: ^9.14.0
+ googleapis: ^11.2.0
+ http: ^0.13.6
+ meta: ^1.9.1
+ nyxx: ^5.0.4
+
+dev_dependencies:
+ test: ^1.21.0
diff --git a/triage_bot/test/bytes_test.dart b/triage_bot/test/bytes_test.dart
new file mode 100644
index 0000000..fd5a9c2
--- /dev/null
+++ b/triage_bot/test/bytes_test.dart
@@ -0,0 +1,78 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'package:test/test.dart';
+import 'package:triage_bot/bytes.dart';
+
+void main() {
+ test('roundtrip null', () {
+ final FileWriter writer = FileWriter();
+ writer.writeNullOr<bool>(null, writer.writeBool);
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readNullOr<bool>(reader.readBool), isNull);
+ reader.close();
+ });
+
+ test('roundtrip true', () {
+ final FileWriter writer = FileWriter();
+ writer.writeNullOr<bool>(true, writer.writeBool);
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readNullOr<bool>(reader.readBool), isTrue);
+ reader.close();
+ });
+
+ test('roundtrip false', () {
+ final FileWriter writer = FileWriter();
+ writer.writeNullOr<bool>(false, writer.writeBool);
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readNullOr<bool>(reader.readBool), isFalse);
+ reader.close();
+ });
+
+ test('roundtrip integer', () {
+ final FileWriter writer = FileWriter();
+ writer.writeInt(12345);
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readInt(), 12345);
+ reader.close();
+ });
+
+ test('roundtrip String', () {
+ final FileWriter writer = FileWriter();
+ writer.writeString('');
+ writer.writeString('abc');
+ writer.writeString(String.fromCharCode(0));
+ writer.writeString('🤷🏿');
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readString(), '');
+ expect(reader.readString(), 'abc');
+ expect(reader.readString(), '\x00');
+ expect(reader.readString(), '🤷🏿');
+ reader.close();
+ });
+
+ test('roundtrip DateTime', () {
+ final FileWriter writer = FileWriter();
+ writer.writeDateTime(DateTime.utc(2023, 6, 23, 15, 45));
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readDateTime().toIso8601String(), '2023-06-23T15:45:00.000Z');
+ reader.close();
+ });
+
+ test('roundtrip Set', () {
+ final FileWriter writer = FileWriter();
+ writer.writeSet(writer.writeString, <String>{'a', 'b', 'c'});
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readSet(reader.readString), <String>{'c', 'b', 'a'});
+ reader.close();
+ });
+
+ test('roundtrip Map', () {
+ final FileWriter writer = FileWriter();
+ writer.writeMap(writer.writeString, writer.writeInt, <String, int>{'a': 1, 'b': 2, 'c': 3});
+ final FileReader reader = FileReader(writer.serialize());
+ expect(reader.readMap(reader.readString, reader.readInt), <String, int>{'c': 3, 'b': 2, 'a': 1});
+ reader.close();
+ });
+}
diff --git a/triage_bot/test/discord_test.dart b/triage_bot/test/discord_test.dart
new file mode 100644
index 0000000..f8a15ef
--- /dev/null
+++ b/triage_bot/test/discord_test.dart
@@ -0,0 +1,49 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'package:test/test.dart';
+import 'package:triage_bot/discord.dart';
+
+void main() {
+ // copied from https://github.com/flutter/engine/pull/43163
+ const String prText = '''
+*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*
+
+*List which issues are fixed by this PR. You must list at least one issue.*
+
+*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
+
+## Pre-launch Checklist
+
+- [ ] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
+- [ ] I read the [Tree Hygiene] wiki page, which explains my responsibilities.
+- [ ] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides].
+- [ ] I listed at least one issue that this PR fixes in the description above.
+- [ ] I added new tests to check the change I am making or feature I am adding, or Hixie said the PR is test-exempt. See [testing the engine] for instructions on writing and running engine tests.
+- [ ] I updated/added relevant documentation (doc comments with `///`).
+- [ ] I signed the [CLA].
+- [ ] All existing and new tests are passing.
+
+If you need help, consider asking for advice on the #hackers-new channel on [Discord].
+
+<!-- Links -->
+[Contributor Guide]: https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
+[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
+[Flutter Style Guide]: https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
+[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
+[testing the engine]: https://github.com/flutter/flutter/wiki/Testing-the-engine
+[CLA]: https://cla.developers.google.com/
+[flutter/tests]: https://github.com/flutter/tests
+[breaking change policy]: https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
+[Discord]: https://github.com/flutter/flutter/wiki/Chat
+''';
+
+ test('hex', () {
+ expect(stripBoilerplate(prText), '<blank>');
+ });
+
+ test('hex', () {
+ expect(stripBoilerplate('hello $prText'), 'hello');
+ });
+}
diff --git a/triage_bot/test/json_test.dart b/triage_bot/test/json_test.dart
new file mode 100644
index 0000000..98e726c
--- /dev/null
+++ b/triage_bot/test/json_test.dart
@@ -0,0 +1,21 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'package:test/test.dart';
+import 'package:triage_bot/json.dart';
+
+void main() {
+ test('json', () {
+ expect(Json.parse('null').toScalar(), null);
+ expect(Json.parse('1.0').toScalar() is double, isTrue);
+ expect(Json.parse('1.0').toScalar(), 1.0);
+ expect(Json.parse('2').toScalar() is double, isTrue);
+ expect(Json.parse('2').toScalar(), 2.0);
+ expect(Json.parse('true').toScalar(), true);
+ expect(Json.parse('false').toScalar(), false);
+ expect(Json.parse('[1, 2, 3]').toList(), <double>[1.0, 2.0, 3.0]);
+ expect(Json.parse('{"1": 2, "3": 4}').toMap(), <String, double>{'1': 2.0, '3': 4.0});
+ expect((Json.parse('{"a": {"b": {"c": "d"}}}') as dynamic).a.b.c.toString(), 'd');
+ });
+}
diff --git a/triage_bot/test/utils_test.dart b/triage_bot/test/utils_test.dart
new file mode 100644
index 0000000..b6856d6
--- /dev/null
+++ b/triage_bot/test/utils_test.dart
@@ -0,0 +1,26 @@
+// Copyright 2019 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+import 'package:test/test.dart';
+import 'package:triage_bot/utils.dart';
+
+void main() {
+ test('hex', () {
+ expect(hex(-1), '-1');
+ expect(hex(0), '00');
+ expect(hex(1), '01');
+ expect(hex(15), '0f');
+ expect(hex(16), '10');
+ expect(hex(256), '100');
+ });
+
+ test('s', () {
+ expect(s(-1), 's');
+ expect(s(0), 's');
+ expect(s(1), '');
+ expect(s(15), 's');
+ expect(s(16), 's');
+ expect(s(256), 's');
+ });
+}
