Add codesign keychain to search list.
update 04/18:
Another day of research and I finally tracked down EVERY single bit of every root causes of flakes on codesign runs. One of the major problems identified and solved was that, we now add our keychain to the search list.
The latest version has been thoroughly tested on 10+ led runs at different times, (>60 subbuilds in total?) and they all passed. I have 99.9 percent confidence that we will not see a codesign flake anymore. led runs are triggered with `led get-build 8784446248215539137 | led edit-recipe-bundle | led edit-system -p 10 | led launch`. Some example successful runs are:
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/ba79a610f55ba0d9afd53534d72bc02cd90bd8416a7f7ed7589242039a6361f2/+/build.proto?server=chromium-swarm.appspot.com
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/771e1a09798e2abfa9b548a9b1ee899573129ea67f7cbc6b0e379f2e31ea578a/+/build.proto?server=chromium-swarm.appspot.com
update 04/17:
Move all keychain setup commands into bash script to run in the same ssh session.
The passes were green in the morning, but started to turn red later in the day. One example failure was
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/b789c3878bda578f21b55983d5ec9dd6d7029e0182e60aafdfd83a385535a9a6/+/build.proto?server=chromium-swarm.appspot.com
side note, sometimes I see
"WARNING: This build ran on a canary version of LUCI. If you suspect it failed due to infra, retry the build. Next time it may use the non-canary version." but i think this might be unrelated to our use case.
Update 04/16:
I am starting to reproduce the errors in importing certificates. It appears to me that for all the successful runs, the only present certificate should be the flutter certificate. On the other hand, if the device already have two certificates
1) 5BB6BDEBAA24047669E1A93BCF5CBD2B1E41BB67 "MDM Client Identity Certificate"
2) CC21BF11AB9925313E54CD4FCCFEB82347E75AFA "Apple Development: Flutter Devicelab (24273VPZX7)"
then the import will be guaranteed to fail. It looks like the success depends on which bot the task is run on.
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/f8de2c9b8bc1ba038fb34202c7d4557ac5f41ceb1d1075b139f5c50cf14c0e5a/+/build.proto?server=chromium-swarm.appspot.com
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/fe251512a89251551e70c3e54ba2993e9e7eb27a874c0b806b6ebc9315fd6f6d/+/build.proto?server=chromium-swarm.appspot.com
Update 04/15:
Tuned bash script and test coverage to make them eventually pass. Learnt about set -e with if, local variable, bash expressions etc.
Was trying to test flakiness with 100 led runs and used a script to pipe luci output (not stdout or stderr) of 100 led runs using script command to google sheet. However, the 100 runs were cancelled due to contention for resources. a sample cancelled run looks like:
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/59b48935638603ac39c73aa8e8ccb69943f10423f9799026e47daae83d98736c/+/build.proto?server=chromium-swarm.appspot.com
update 4/14:
wasn't able to reproduce the flake, successful led runs are:
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/d57db265bec15c048ed82199f0d8f77a0b9e67ba109577c02d57dcd747401227/+/build.proto?server=chromium-swarm.appspot.com
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/20efcb900545754ea92434f7ba64339f3571104089dadb51e564a0a7721d7768/+/build.proto?server=chromium-swarm.appspot.com
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/xilaizhang_google.com/4e8f7c2696f7a39ac4721198d83896f00dea821814cd3cd3918ca7895a39817b/+/build.proto?server=chromium-swarm.appspot.com
Change-Id: Ic40e62032886ca9cbe77679239fd48ba4744da04
Reviewed-on: https://flutter-review.googlesource.com/c/recipes/+/41900
Reviewed-by: Godofredo Contreras <godofredoc@google.com>
Commit-Queue: Xilai Zhang <xilaizhang@google.com>
diff --git a/recipe_modules/signing/api.py b/recipe_modules/signing/api.py
index 0f30a5b..5e80642 100644
--- a/recipe_modules/signing/api.py
+++ b/recipe_modules/signing/api.py
@@ -5,7 +5,6 @@
#import zipfile
from recipe_engine import recipe_api
-
# File name inside artifacts that require signing with entitlements.
ENTITLEMENTS_FILENAME = 'entitlements.txt'
# File name inside artifacts that require signing without entitlements.
@@ -20,7 +19,6 @@
self._initialized = False
self._codesign_binary_path = None
-
def requires_signing(self, artifact_path):
"""Validates if a file needs to be codesigned.
@@ -36,7 +34,8 @@
file_list = self.m.zip.namelist('namelist', artifact_path)
return (
ENTITLEMENTS_FILENAME in file_list or
- WITHOUT_ENTITLEMENTS_FILENAME in file_list)
+ WITHOUT_ENTITLEMENTS_FILENAME in file_list
+ )
@property
def codesign_binary(self):
@@ -49,15 +48,13 @@
self._codesign_environment(env, env_prefixes)
self._keychain_setup(env, env_prefixes)
-
def _stop(self):
self._keychain_cleanup()
-
def _ensure(self):
if not self._codesign_binary_path:
with self.m.step.nest('Codesign Dependencies'):
- self._codesign_binary_path = self.m.flutter_deps.codesign({},{})
+ self._codesign_binary_path = self.m.flutter_deps.codesign({}, {})
def code_sign(self, files_to_sign):
if not self.m.platform.is_mac:
@@ -71,7 +68,6 @@
if not self.m.runtime.in_global_shutdown:
self._stop()
-
def _codesign_environment(self, env, env_prefixes):
with self.m.step.nest('Setup codesign environment'):
secrets_dict = {
@@ -85,65 +81,13 @@
self.m.kms.decrypt_secrets(env, secrets_dict)
env['CODESIGN_PATH'] = self.codesign_binary
-
-
def _keychain_setup(self, env, env_prefixes):
- """KeychainSetup adds flutter .p12 to a temporary keychain named 'build'.
-
- Args:
- env (dict): environment variables.
- env_prefixes (dict) : environment paths.
- """
- with self.m.step.nest('Setup keychain'):
- # Delete build.keychain if exists.
- self.m.step(
- 'delete previous keychain',
- ['security', 'delete-keychain', 'build.keychain'],
- ok_ret='any'
- )
- # Create build.keychain.
- self.m.step(
- 'create keychain',
- ['security', 'create-keychain', '-p', '', 'build.keychain']
- )
- # Set build.keychain as default.
- self.m.step(
- 'default keychain',
- ['security', 'default-keychain', '-s', 'build.keychain']
- )
- # Unlock build.keychain to allow sign commands to use its secrets.
- self.m.step(
- 'unlock build keychain',
- ['security', 'unlock-keychain', '-p', '', 'build.keychain']
- )
- # Import flutter's certificate to the keychain.
- self._import_certificate(env, env_prefixes)
- # Sets a partition list to identify the app signatures allowed to use the key.
- self.m.step(
- 'set key partition list', [
- 'security', 'set-key-partition-list', '-S',
- 'apple-tool:,apple:,codesign:', '-s', '-k', '', 'build.keychain'
- ]
- )
- # Grabs existing identities to find out if we are ready to sign with flutter's
- # identity.
- show_identities_step = self.m.step(
- 'show-identities', ['security', 'find-identity', '-v'],
- ok_ret='any',
- stdout=self.m.raw_io.output_text(),
- stderr=self.m.raw_io.output_text()
- )
- flutter_identity_name = 'FLUTTER.IO LLC'
- if flutter_identity_name not in show_identities_step.stdout:
- raise ValueError(
- 'identities are %s, does not include flutter identity' %
- (show_identities_step.stdout)
- )
-
- def _import_certificate(self, env, env_prefixes):
- """Import flutter codesign identity into keychain.
-
- This function triggers a shell script that supplies p12 password,
+ """KeychainSetup sets up keychain for codesign.
+
+ This function triggers a shell script that creates a keychain named
+ build.keychain. It unlocks the keychain,
+ adds keychain to codesign search list, and adds flutter .p12
+ to this keychain. This script also supplies p12 password,
and grants codesign cipd and system codesign the correct access controls.
The p12 password is hidden from stdout.
@@ -151,18 +95,21 @@
env (dict): environment variables.
env_prefixes (dict) : environment paths.
"""
- resource_name = self.resource('import_certificate.sh')
- self.m.step(
- 'Set execute permission',
- ['chmod', '755', resource_name],
- infra_step=True,
- )
+ with self.m.step.nest('Setup keychain'):
+ resource_name = self.resource('setup_keychain.sh')
+ self.m.step(
+ 'Set execute permission',
+ ['chmod', '755', resource_name],
+ infra_step=True,
+ )
# Only filepath with a .p12 suffix will be recognized.
p12_suffix_filepath = self.m.path['cleanup'].join('flutter.p12')
env['P12_SUFFIX_FILEPATH'] = p12_suffix_filepath
with self.m.context(env=env, env_prefixes=env_prefixes):
- self.m.step('import certificate', [resource_name])
-
+ self.m.step(
+ 'run keychain setup script', [resource_name],
+ stdout=self.m.raw_io.output_text()
+ )
def _signer_tasks(self, env, env_prefixes, files_to_sign):
"""Concurrently creates jobs to codesign each binary.
@@ -174,12 +121,12 @@
signer_builds = []
for source_path in files_to_sign:
signer_builds.append(
- self.m.futures.spawn(
- self._run_signer_tool_command,
- env,
- env_prefixes,
- source_path,
- )
+ self.m.futures.spawn(
+ self._run_signer_tool_command,
+ env,
+ env_prefixes,
+ source_path,
+ )
)
futures = self.m.futures.wait(signer_builds)
@@ -204,13 +151,9 @@
team_id_filepath = env['CODESIGN_TEAM_ID']
path, base_name = self.m.path.split(source_path)
unsigned_path = self.m.path.join(path, 'unsigned_%s' % base_name)
- self.m.file.move(
- 'Move %s' % str(source_path),
- source_path,
- unsigned_path
- )
+ self.m.file.move('Move %s' % str(source_path), source_path, unsigned_path)
with self.m.step.nest('Codesign %s' % str(unsigned_path)):
- flutter_certificate_name = 'FLUTTER.IO LLC'
+ flutter_certificate_name = 'Developer ID Application: FLUTTER.IO LLC (S8QB4VV633)'
self.m.step(
'unlock build keychain',
['security', 'unlock-keychain', '-p', '', 'build.keychain']
@@ -239,9 +182,10 @@
def _keychain_cleanup(self):
"""Clean up temporary keychain used in codesign process."""
with self.m.step.nest('Keychain cleanup'):
- self.m.step('delete keychain', ['security', 'delete-keychain', 'build.keychain'])
self.m.step(
- 'Cleanup keychain.restore default keychain',
- ['security', 'default-keychain', '-s', 'login.keychain']
+ 'delete keychain', ['security', 'delete-keychain', 'build.keychain']
)
-
+ self.m.step(
+ 'Cleanup keychain.restore default keychain',
+ ['security', 'default-keychain', '-s', 'login.keychain']
+ )
diff --git a/recipe_modules/signing/examples/code_sign.expected/mac_require_signing.json b/recipe_modules/signing/examples/code_sign.expected/mac_require_signing.json
index a8b9aac..0da9b06 100644
--- a/recipe_modules/signing/examples/code_sign.expected/mac_require_signing.json
+++ b/recipe_modules/signing/examples/code_sign.expected/mac_require_signing.json
@@ -359,58 +359,9 @@
},
{
"cmd": [
- "security",
- "delete-keychain",
- "build.keychain"
- ],
- "name": "Setup keychain.delete previous keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "create-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "name": "Setup keychain.create keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "default-keychain",
- "-s",
- "build.keychain"
- ],
- "name": "Setup keychain.default keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "unlock-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "name": "Setup keychain.unlock build keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
"chmod",
"755",
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"infra_step": true,
"name": "Setup keychain.Set execute permission",
@@ -420,7 +371,7 @@
},
{
"cmd": [
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"env": {
"CODESIGN_APP_SPECIFIC_PASSWORD": "[CLEANUP]/CODESIGN_APP_SPECIFIC_PASSWORD",
@@ -431,37 +382,7 @@
"FLUTTER_P12_PASSWORD": "[CLEANUP]/FLUTTER_P12_PASSWORD",
"P12_SUFFIX_FILEPATH": "[CLEANUP]/flutter.p12"
},
- "name": "Setup keychain.import certificate",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "set-key-partition-list",
- "-S",
- "apple-tool:,apple:,codesign:",
- "-s",
- "-k",
- "",
- "build.keychain"
- ],
- "name": "Setup keychain.set key partition list",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "find-identity",
- "-v"
- ],
- "name": "Setup keychain.show-identities",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
+ "name": "run keychain setup script"
},
{
"cmd": [
@@ -498,7 +419,7 @@
"cmd": [
"[CLEANUP]/tmp_tmp_1/codesign",
"--codesign-cert-name",
- "FLUTTER.IO LLC",
+ "Developer ID Application: FLUTTER.IO LLC (S8QB4VV633)",
"--no-dryrun",
"--app-specific-password-file-path",
"[CLEANUP]/CODESIGN_APP_SPECIFIC_PASSWORD",
diff --git a/recipe_modules/signing/examples/code_sign.expected/no_signing_identity.json b/recipe_modules/signing/examples/code_sign.expected/no_signing_identity.json
index 9ae929d..0da9b06 100644
--- a/recipe_modules/signing/examples/code_sign.expected/no_signing_identity.json
+++ b/recipe_modules/signing/examples/code_sign.expected/no_signing_identity.json
@@ -355,65 +355,13 @@
},
{
"cmd": [],
- "name": "Setup keychain",
- "~followup_annotations": [
- "@@@STEP_EXCEPTION@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "delete-keychain",
- "build.keychain"
- ],
- "name": "Setup keychain.delete previous keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "create-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "name": "Setup keychain.create keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "default-keychain",
- "-s",
- "build.keychain"
- ],
- "name": "Setup keychain.default keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "unlock-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "name": "Setup keychain.unlock build keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
+ "name": "Setup keychain"
},
{
"cmd": [
"chmod",
"755",
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"infra_step": true,
"name": "Setup keychain.Set execute permission",
@@ -423,7 +371,7 @@
},
{
"cmd": [
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"env": {
"CODESIGN_APP_SPECIFIC_PASSWORD": "[CLEANUP]/CODESIGN_APP_SPECIFIC_PASSWORD",
@@ -434,23 +382,81 @@
"FLUTTER_P12_PASSWORD": "[CLEANUP]/FLUTTER_P12_PASSWORD",
"P12_SUFFIX_FILEPATH": "[CLEANUP]/flutter.p12"
},
- "name": "Setup keychain.import certificate",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
+ "name": "run keychain setup script"
+ },
+ {
+ "cmd": [
+ "vpython3",
+ "-u",
+ "RECIPE_MODULE[recipe_engine::file]/resources/fileutil.py",
+ "--json-output",
+ "/path/to/tmp/json",
+ "move",
+ "file1.zip",
+ "unsigned_file1.zip"
+ ],
+ "infra_step": true,
+ "name": "Move file1.zip"
+ },
+ {
+ "cmd": [],
+ "name": "Codesign unsigned_file1.zip"
},
{
"cmd": [
"security",
- "set-key-partition-list",
- "-S",
- "apple-tool:,apple:,codesign:",
- "-s",
- "-k",
+ "unlock-keychain",
+ "-p",
"",
"build.keychain"
],
- "name": "Setup keychain.set key partition list",
+ "name": "Codesign unsigned_file1.zip.unlock build keychain",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@1@@@"
+ ]
+ },
+ {
+ "cmd": [
+ "[CLEANUP]/tmp_tmp_1/codesign",
+ "--codesign-cert-name",
+ "Developer ID Application: FLUTTER.IO LLC (S8QB4VV633)",
+ "--no-dryrun",
+ "--app-specific-password-file-path",
+ "[CLEANUP]/CODESIGN_APP_SPECIFIC_PASSWORD",
+ "--codesign-appstore-id-file-path",
+ "[CLEANUP]/CODESIGN_APP_STORE_ID",
+ "--codesign-team-id-file-path",
+ "[CLEANUP]/CODESIGN_TEAM_ID",
+ "--input-zip-file-path",
+ "unsigned_file1.zip",
+ "--output-zip-file-path",
+ "file1.zip"
+ ],
+ "env": {
+ "CODESIGN_APP_SPECIFIC_PASSWORD": "[CLEANUP]/CODESIGN_APP_SPECIFIC_PASSWORD",
+ "CODESIGN_APP_STORE_ID": "[CLEANUP]/CODESIGN_APP_STORE_ID",
+ "CODESIGN_PATH": "[CLEANUP]/tmp_tmp_1/codesign",
+ "CODESIGN_TEAM_ID": "[CLEANUP]/CODESIGN_TEAM_ID",
+ "FLUTTER_P12": "[CLEANUP]/FLUTTER_P12",
+ "FLUTTER_P12_PASSWORD": "[CLEANUP]/FLUTTER_P12_PASSWORD",
+ "P12_SUFFIX_FILEPATH": "[CLEANUP]/flutter.p12"
+ },
+ "name": "Codesign unsigned_file1.zip.codesign Apple engine binaries",
+ "~followup_annotations": [
+ "@@@STEP_NEST_LEVEL@1@@@"
+ ]
+ },
+ {
+ "cmd": [],
+ "name": "Keychain cleanup"
+ },
+ {
+ "cmd": [
+ "security",
+ "delete-keychain",
+ "build.keychain"
+ ],
+ "name": "Keychain cleanup.delete keychain",
"~followup_annotations": [
"@@@STEP_NEST_LEVEL@1@@@"
]
@@ -458,10 +464,11 @@
{
"cmd": [
"security",
- "find-identity",
- "-v"
+ "default-keychain",
+ "-s",
+ "login.keychain"
],
- "name": "Setup keychain.show-identities",
+ "name": "Keychain cleanup.Cleanup keychain.restore default keychain",
"~followup_annotations": [
"@@@STEP_NEST_LEVEL@1@@@"
]
diff --git a/recipe_modules/signing/examples/code_sign.py b/recipe_modules/signing/examples/code_sign.py
index 8b4370c..f540951 100644
--- a/recipe_modules/signing/examples/code_sign.py
+++ b/recipe_modules/signing/examples/code_sign.py
@@ -15,18 +15,8 @@
def RunSteps(api):
- env = {}
- env_prefixes = {}
files_to_sign = ['file1.zip']
- if api.properties.get('raises'):
- with api.assertions.assertRaises(ValueError):
- api.signing.code_sign(
- files_to_sign=files_to_sign,
- )
- else:
- api.signing.code_sign(
- files_to_sign=files_to_sign,
- )
+ api.signing.code_sign(files_to_sign=files_to_sign)
def GenTests(api):
@@ -38,11 +28,10 @@
yield api.test(
'mac_require_signing',
api.platform.name('mac'),
- api.properties(expected_result=True),
- api.signing.flutter_signing_identity(),
+ api.properties(expected_result=True)
)
yield api.test(
'no_signing_identity',
api.platform.name('mac'),
- api.properties(expected_result=False, raises=True),
+ api.properties(expected_result=False)
)
diff --git a/recipe_modules/signing/resources/import_certificate.sh b/recipe_modules/signing/resources/import_certificate.sh
deleted file mode 100644
index f4151e1..0000000
--- a/recipe_modules/signing/resources/import_certificate.sh
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-
-# Helper script to import a flutter p12 identity.
-# Note: do not enable -x to display expanded values of the variables, as this will leak the passwords.
-set -e
-
-RAW_PASSWORD=$(cat $FLUTTER_P12_PASSWORD)
-# Only filepath with a .p12 suffix will be recognized
-mv $FLUTTER_P12 $P12_SUFFIX_FILEPATH
-/usr/bin/security import $P12_SUFFIX_FILEPATH -k build.keychain -P $RAW_PASSWORD -T $CODESIGN_PATH -T /usr/bin/codesign
\ No newline at end of file
diff --git a/recipe_modules/signing/resources/runner.sh b/recipe_modules/signing/resources/runner.sh
deleted file mode 100644
index a35286d..0000000
--- a/recipe_modules/signing/resources/runner.sh
+++ /dev/null
@@ -1,12 +0,0 @@
-#!/bin/bash
-
-# Helper script to unlock the keychain in the same session
-# as the test runner script.
-set -e
-
-if [ -f /usr/local/bin/unlock_login_keychain.sh ]
-then
- /usr/local/bin/unlock_login_keychain.sh
-else
- echo "This bot does not support codesigning"
-fi
\ No newline at end of file
diff --git a/recipe_modules/signing/resources/setup_keychain.sh b/recipe_modules/signing/resources/setup_keychain.sh
new file mode 100644
index 0000000..52a36c8
--- /dev/null
+++ b/recipe_modules/signing/resources/setup_keychain.sh
@@ -0,0 +1,53 @@
+#!/bin/bash
+
+# Helper script to import a flutter p12 identity.
+# Note: do not enable -x to display expanded values of the variables, as this will leak the passwords.
+set -e
+
+RAW_PASSWORD=$(cat $FLUTTER_P12_PASSWORD)
+# Only filepath with a .p12 suffix will be recognized
+mv $FLUTTER_P12 $P12_SUFFIX_FILEPATH
+
+# Delete build.keychain if it exists, do no-op if not exist.
+if /usr/bin/security delete-keychain build.keychain; then
+ :
+fi
+# Create build.keychain.
+/usr/bin/security create-keychain -p '' build.keychain
+
+# Retrieve current list of keychains on the search list of current machine.
+keychains=$(security list-keychains -d user)
+
+keychainNames=();
+
+for keychain in $keychains
+do
+ basename=$(basename "$keychain")
+ keychainName=${basename::${#basename}-4}
+ keychainNames+=("$keychainName")
+done
+
+echo "User keychains on this machine: ${keychainNames[@]}";
+
+# Add keychain name to search list. (FML, took me 5 days to hunt this down)
+/usr/bin/security -v list-keychains -s "${keychainNames[@]}" build.keychain
+
+# Set build.keychain as default.
+/usr/bin/security default-keychain -s build.keychain
+
+# Unlock build.keychain to allow sign commands to use its certs.
+/usr/bin/security unlock-keychain -p '' build.keychain
+
+attempt=0
+sleep_time=2
+while [ $attempt -lt 3 ]; do
+ /usr/bin/security import $P12_SUFFIX_FILEPATH -k build.keychain -P $RAW_PASSWORD -T $CODESIGN_PATH -T /usr/bin/codesign
+ /usr/bin/security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k '' build.keychain
+ if /usr/bin/security find-identity -v build.keychain | grep 'FLUTTER.IO LLC'; then
+ exit 0
+ fi
+ sleep $sleep_time
+ attempt=$(( attempt + 1 ))
+ sleep_time=$(( sleep_time * sleep_time ))
+done
+exit 1
\ No newline at end of file
diff --git a/recipe_modules/signing/test_api.py b/recipe_modules/signing/test_api.py
deleted file mode 100644
index b0a21d4..0000000
--- a/recipe_modules/signing/test_api.py
+++ /dev/null
@@ -1,16 +0,0 @@
-import os
-import tempfile
-import zipfile
-
-from recipe_engine import recipe_test_api
-
-class RecipeTestingTestApi(recipe_test_api.RecipeTestApi):
-
- def flutter_signing_identity(self, step_name=None):
- step_name = step_name or 'Setup keychain.show-identities'
- return self.step_data(
- step_name,
- stdout=self.m.raw_io.output_text(
- '1) ABCD "Developer ID Application: FLUTTER.IO LLC (ABCD)"'
- )
- )
diff --git a/recipes/engine_v2/builder.expected/mac.json b/recipes/engine_v2/builder.expected/mac.json
index d466b47..0e3b51b 100644
--- a/recipes/engine_v2/builder.expected/mac.json
+++ b/recipes/engine_v2/builder.expected/mac.json
@@ -2829,182 +2829,9 @@
},
{
"cmd": [
- "security",
- "delete-keychain",
- "build.keychain"
- ],
- "cwd": "[CACHE]/builder/src/flutter",
- "env": {
- "ANDROID_HOME": "[CACHE]/builder/src/third_party/android_tools/sdk",
- "DEPOT_TOOLS": "RECIPE_REPO[depot_tools]",
- "ENGINE_CHECKOUT_PATH": "[CACHE]/builder",
- "ENGINE_PATH": "[CACHE]/builder",
- "GIT_BRANCH": "",
- "LUCI_BRANCH": "",
- "LUCI_CI": "True",
- "LUCI_PR": "",
- "LUCI_WORKDIR": "[START_DIR]",
- "OS": "darwin",
- "REVISION": "abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd"
- },
- "env_prefixes": {
- "PATH": [
- "[CACHE]/builder/src/third_party/dart/tools/sdks/dart-sdk/bin"
- ]
- },
- "luci_context": {
- "realm": {
- "name": "flutter:prod"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Setup keychain.delete previous keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "create-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "cwd": "[CACHE]/builder/src/flutter",
- "env": {
- "ANDROID_HOME": "[CACHE]/builder/src/third_party/android_tools/sdk",
- "DEPOT_TOOLS": "RECIPE_REPO[depot_tools]",
- "ENGINE_CHECKOUT_PATH": "[CACHE]/builder",
- "ENGINE_PATH": "[CACHE]/builder",
- "GIT_BRANCH": "",
- "LUCI_BRANCH": "",
- "LUCI_CI": "True",
- "LUCI_PR": "",
- "LUCI_WORKDIR": "[START_DIR]",
- "OS": "darwin",
- "REVISION": "abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd"
- },
- "env_prefixes": {
- "PATH": [
- "[CACHE]/builder/src/third_party/dart/tools/sdks/dart-sdk/bin"
- ]
- },
- "luci_context": {
- "realm": {
- "name": "flutter:prod"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Setup keychain.create keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "default-keychain",
- "-s",
- "build.keychain"
- ],
- "cwd": "[CACHE]/builder/src/flutter",
- "env": {
- "ANDROID_HOME": "[CACHE]/builder/src/third_party/android_tools/sdk",
- "DEPOT_TOOLS": "RECIPE_REPO[depot_tools]",
- "ENGINE_CHECKOUT_PATH": "[CACHE]/builder",
- "ENGINE_PATH": "[CACHE]/builder",
- "GIT_BRANCH": "",
- "LUCI_BRANCH": "",
- "LUCI_CI": "True",
- "LUCI_PR": "",
- "LUCI_WORKDIR": "[START_DIR]",
- "OS": "darwin",
- "REVISION": "abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd"
- },
- "env_prefixes": {
- "PATH": [
- "[CACHE]/builder/src/third_party/dart/tools/sdks/dart-sdk/bin"
- ]
- },
- "luci_context": {
- "realm": {
- "name": "flutter:prod"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Setup keychain.default keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "unlock-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "cwd": "[CACHE]/builder/src/flutter",
- "env": {
- "ANDROID_HOME": "[CACHE]/builder/src/third_party/android_tools/sdk",
- "DEPOT_TOOLS": "RECIPE_REPO[depot_tools]",
- "ENGINE_CHECKOUT_PATH": "[CACHE]/builder",
- "ENGINE_PATH": "[CACHE]/builder",
- "GIT_BRANCH": "",
- "LUCI_BRANCH": "",
- "LUCI_CI": "True",
- "LUCI_PR": "",
- "LUCI_WORKDIR": "[START_DIR]",
- "OS": "darwin",
- "REVISION": "abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd"
- },
- "env_prefixes": {
- "PATH": [
- "[CACHE]/builder/src/third_party/dart/tools/sdks/dart-sdk/bin"
- ]
- },
- "luci_context": {
- "realm": {
- "name": "flutter:prod"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Setup keychain.unlock build keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
"chmod",
"755",
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"cwd": "[CACHE]/builder/src/flutter",
"env": {
@@ -3045,7 +2872,7 @@
},
{
"cmd": [
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"cwd": "[CACHE]/builder/src/flutter",
"env": {
@@ -3085,99 +2912,7 @@
"hostname": "rdbhost"
}
},
- "name": "Setup keychain.import certificate",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "set-key-partition-list",
- "-S",
- "apple-tool:,apple:,codesign:",
- "-s",
- "-k",
- "",
- "build.keychain"
- ],
- "cwd": "[CACHE]/builder/src/flutter",
- "env": {
- "ANDROID_HOME": "[CACHE]/builder/src/third_party/android_tools/sdk",
- "DEPOT_TOOLS": "RECIPE_REPO[depot_tools]",
- "ENGINE_CHECKOUT_PATH": "[CACHE]/builder",
- "ENGINE_PATH": "[CACHE]/builder",
- "GIT_BRANCH": "",
- "LUCI_BRANCH": "",
- "LUCI_CI": "True",
- "LUCI_PR": "",
- "LUCI_WORKDIR": "[START_DIR]",
- "OS": "darwin",
- "REVISION": "abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd"
- },
- "env_prefixes": {
- "PATH": [
- "[CACHE]/builder/src/third_party/dart/tools/sdks/dart-sdk/bin"
- ]
- },
- "luci_context": {
- "realm": {
- "name": "flutter:prod"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Setup keychain.set key partition list",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "find-identity",
- "-v"
- ],
- "cwd": "[CACHE]/builder/src/flutter",
- "env": {
- "ANDROID_HOME": "[CACHE]/builder/src/third_party/android_tools/sdk",
- "DEPOT_TOOLS": "RECIPE_REPO[depot_tools]",
- "ENGINE_CHECKOUT_PATH": "[CACHE]/builder",
- "ENGINE_PATH": "[CACHE]/builder",
- "GIT_BRANCH": "",
- "LUCI_BRANCH": "",
- "LUCI_CI": "True",
- "LUCI_PR": "",
- "LUCI_WORKDIR": "[START_DIR]",
- "OS": "darwin",
- "REVISION": "abcdabcdabcdabcdabcdabcdabcdabcdabcdabcd"
- },
- "env_prefixes": {
- "PATH": [
- "[CACHE]/builder/src/third_party/dart/tools/sdks/dart-sdk/bin"
- ]
- },
- "luci_context": {
- "realm": {
- "name": "flutter:prod"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Setup keychain.show-identities",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@1@@@"
- ]
+ "name": "run keychain setup script"
},
{
"cmd": [],
diff --git a/recipes/engine_v2/builder.py b/recipes/engine_v2/builder.py
index 187e9fa..138694a 100644
--- a/recipes/engine_v2/builder.py
+++ b/recipes/engine_v2/builder.py
@@ -287,7 +287,6 @@
revision='abcd' * 10,
build_number=123,
),
- api.signing.flutter_signing_identity(),
api.step_data(
'Identify branches.git branch',
stdout=api.raw_io
diff --git a/recipes/engine_v2/engine_v2.expected/codesign_release_branch.json b/recipes/engine_v2/engine_v2.expected/codesign_release_branch.json
index dd046df..62c5a67 100644
--- a/recipes/engine_v2/engine_v2.expected/codesign_release_branch.json
+++ b/recipes/engine_v2/engine_v2.expected/codesign_release_branch.json
@@ -1932,106 +1932,9 @@
},
{
"cmd": [
- "security",
- "delete-keychain",
- "build.keychain"
- ],
- "luci_context": {
- "realm": {
- "name": "proj:try"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Global generators.Setup keychain.delete previous keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@2@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "create-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "luci_context": {
- "realm": {
- "name": "proj:try"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Global generators.Setup keychain.create keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@2@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "default-keychain",
- "-s",
- "build.keychain"
- ],
- "luci_context": {
- "realm": {
- "name": "proj:try"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Global generators.Setup keychain.default keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@2@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "unlock-keychain",
- "-p",
- "",
- "build.keychain"
- ],
- "luci_context": {
- "realm": {
- "name": "proj:try"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Global generators.Setup keychain.unlock build keychain",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@2@@@"
- ]
- },
- {
- "cmd": [
"chmod",
"755",
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"infra_step": true,
"luci_context": {
@@ -2053,7 +1956,7 @@
},
{
"cmd": [
- "RECIPE_MODULE[flutter::signing]/resources/import_certificate.sh"
+ "RECIPE_MODULE[flutter::signing]/resources/setup_keychain.sh"
],
"env": {
"CODESIGN_APP_SPECIFIC_PASSWORD": "[CLEANUP]/CODESIGN_APP_SPECIFIC_PASSWORD",
@@ -2076,60 +1979,9 @@
"hostname": "rdbhost"
}
},
- "name": "Global generators.Setup keychain.import certificate",
+ "name": "Global generators.run keychain setup script",
"~followup_annotations": [
- "@@@STEP_NEST_LEVEL@2@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "set-key-partition-list",
- "-S",
- "apple-tool:,apple:,codesign:",
- "-s",
- "-k",
- "",
- "build.keychain"
- ],
- "luci_context": {
- "realm": {
- "name": "proj:try"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Global generators.Setup keychain.set key partition list",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@2@@@"
- ]
- },
- {
- "cmd": [
- "security",
- "find-identity",
- "-v"
- ],
- "luci_context": {
- "realm": {
- "name": "proj:try"
- },
- "resultdb": {
- "current_invocation": {
- "name": "invocations/build:8945511751514863184",
- "update_token": "token"
- },
- "hostname": "rdbhost"
- }
- },
- "name": "Global generators.Setup keychain.show-identities",
- "~followup_annotations": [
- "@@@STEP_NEST_LEVEL@2@@@"
+ "@@@STEP_NEST_LEVEL@1@@@"
]
},
{
diff --git a/recipes/engine_v2/engine_v2.py b/recipes/engine_v2/engine_v2.py
index 7ee5566..91171e8 100644
--- a/recipes/engine_v2/engine_v2.py
+++ b/recipes/engine_v2/engine_v2.py
@@ -442,7 +442,6 @@
stdout=api.raw_io
.output_text('branch1\nbranch2\nflutter-3.2-candidate.5')
),
- api.signing.flutter_signing_identity('Global generators.Setup keychain.show-identities'),
)
tests = [{
