Latest Attempt:

The main change here versus the other attempts is to remove the use of the m.path objects. These objects are creating issues when attempting to compare objects with what should be strings and when converting to and from JSON. They cannot be converted directly to json.

Original Description:
https://flutter-review.googlesource.com/c/recipes/+/53221

This cl introduces a cache micro manager that will run cleanup primarily for osx_sdk directories but could be used elsewhere to manage clean up on the cache subdirectory level.

Addresses: https://github.com/flutter/flutter/issues/137951 in which it was found that having multiple versions of Xcode was causing significant overhead to runtimes on mac os tests.

The change here will track packages in a cache for example osx_sdk and keep track of the time in which packages will be removed with a metadata file in the directory.
* If a package is updated and used the removal date will always be x days (currently 30 days) from the last day it was touched.
* If a package is not updated it will be determined if it needs to be removed and if so will remove it.
* Packages FOUND ON disk but NOT IN the file will be added automatically for tracking and removal.
* Packages NOT FOUND on disk BUT IN the metadata file will be removed from the metadata file.

One things to note is that we may want to tweak the time that we leave the packages around to something shorter or make it configurable via the ci.yaml. Though I am not sure how often multiple packages are used.

This tool is unobtrusive and does not aggressively remove files. If you want to reset all package expirations you can simple remove the file from the machine to force it to recalculate the removal dates.

This does not run as a service and can be added as a step to existing recipes if we need to remove packages causing overhead to other test runtimes.

https://ci.chromium.org/ui/p/flutter/builders/try.shadow/Mac%20plugin_test_ios/4/overview

Bug: https://github.com/flutter/flutter/issues/137951
Change-Id: I89e013ae869713f763292c9b4bd1dfb962c8323d
Reviewed-on: https://flutter-review.googlesource.com/c/recipes/+/53801
Reviewed-by: Keyong Han <keyonghan@google.com>
Commit-Queue: Ricardo Amador <ricardoamador@google.com>
39 files changed
tree: 3b9aca5ed653798ea4248b72e35ce1ec144c298c
  1. infra/
  2. recipe_modules/
  3. recipes/
  4. .gitignore
  5. .style.yapf
  6. AUTHORS
  7. CONTRIBUTING.md
  8. LICENSE
  9. OWNERS
  10. README.md
  11. recipes.py
README.md

Flutter LUCI Recipes

This repository contains Flutter's LUCI recipes. For the LUCI infrastructure config, see flutter/infra. Builds can be found in the Flutter Dashboard.

Supported repositories roll their .ci.yaml into flutter/infra, which updates what properties builds have. For example, flutter config specifies various dependencies the different tests require, which are then used by the flutter_deps recipe_module No modifications to flutter/infra are required to work on the recipes.

Contributing

Please follow instructions on Contributing docs to set up your development environment.