Describe the enhancement requested
Summary
Several crossbow workflows use sccache w/ S3 for build caching. They are currently configured to use long-lived access key / secret key pairs stored as github secrets.
An alternative approach exists w/ temporary credentials: by registering GitHub's OIDC provider and assuming an AWS_ROLE_ARN that's been properly trusted.
Note: if the proposal is adopted (to standardize on ccache) in #50722, this is moot.
Note 2: I have this working on a fork (I use this setup elsewhere), and can share a draft PR if wanted.
References:
Current State
compose.yaml's x-sccache anchor passes AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY through to the build containers, inherited from the runner environment:
|
AWS_ACCESS_KEY_ID: |
|
AWS_SECRET_ACCESS_KEY: |
In GitHub Actions, these come from repository secrets: long-term IAM user credentials / secret-access key pairs:
|
"AWS_ACCESS_KEY_ID": '${{ secrets.AWS_ACCESS_KEY_ID }}', |
|
"AWS_SECRET_ACCESS_KEY": '${{ secrets.AWS_SECRET_ACCESS_KEY }}', |
Proposal: AssumeRoleWithWebIdentity w/ AWS_ROLE_ARN
Use AssumeRoleWithWebIdentity for sccache S3 credentials.
By assuming the AWS_ROLE_ARN, the sccache session relies on credentials that expire*, issued only to workflows in the trusted repository, and scoped to the sccache bucket.
This requires a few things:
- create a role in IAM with the bucket and object level permissions,
- trust the github oidc-provider,
- define AWS_ROLE_ARN and set id-token permission
- grant sts:AssumeRoleWithWebIdentity for the specific repositories or organization.
- pass AWS_SESSION_TOKEN
- add a
aws-actions/configure-aws-credentials@v4 step to the workflow.
* default is 1 hour, configurable
Component(s)
Continuous Integration
Describe the enhancement requested
Summary
Several crossbow workflows use sccache w/ S3 for build caching. They are currently configured to use long-lived access key / secret key pairs stored as github secrets.
An alternative approach exists w/ temporary credentials: by registering GitHub's OIDC provider and assuming an AWS_ROLE_ARN that's been properly trusted.
Note: if the proposal is adopted (to standardize on ccache) in #50722, this is moot.Note 2: I have this working on a fork (I use this setup elsewhere), and can share a draft PR if wanted.
References:
Current State
compose.yaml'sx-sccacheanchor passesAWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEYthrough to the build containers, inherited from the runner environment:arrow/compose.yaml
Lines 74 to 75 in 18146fc
In GitHub Actions, these come from repository secrets: long-term IAM user credentials / secret-access key pairs:
arrow/dev/tasks/macros.jinja
Lines 292 to 293 in 18146fc
Proposal: AssumeRoleWithWebIdentity w/ AWS_ROLE_ARN
Use AssumeRoleWithWebIdentity for sccache S3 credentials.
By assuming the AWS_ROLE_ARN, the sccache session relies on credentials that expire*, issued only to workflows in the trusted repository, and scoped to the sccache bucket.
This requires a few things:
aws-actions/configure-aws-credentials@v4step to the workflow.* default is 1 hour, configurable
Component(s)
Continuous Integration