Skip to content

Promotion 2026-08-25 anvilprod (#8245, #8255, #8282, DataBiosphere/azul-private#377, DataBiosphere/azul-private#414) - #8262

Merged
hannes-ucsc merged 56 commits into
anvilprodfrom
promotions/2026-08-25-anvilprod
Sep 5, 2026
Merged

Promotion 2026-08-25 anvilprod (#8245, #8255, #8282, DataBiosphere/azul-private#377, DataBiosphere/azul-private#414)#8262
hannes-ucsc merged 56 commits into
anvilprodfrom
promotions/2026-08-25-anvilprod

Conversation

@hannes-ucsc

@hannes-ucsc hannes-ucsc commented Aug 27, 2026

Copy link
Copy Markdown
Member

Linked issues: #8245, #8255, #8282, DataBiosphere/azul-private#377, DataBiosphere/azul-private#414

Checklist

Author

  • PR is assigned to the author
  • Status of PR is In progress
  • Target branch is anvilprod
  • Name of PR branch matches promotions/yyyy-mm-dd-anvilprod
  • PR is linked to the promotion issues it resolves
  • Status of linked issue is In progress
  • PR description links to linked issues
  • Title of linked issue matches Promotion yyyy-mm-dd
  • PR title starts with title of linked issue followed by anvilprod
  • PR title references the linked issues

Author (reindex)

  • This PR is labeled reindex:anvilprod or the changes introduced by it will not require reindexing of anvilprod
  • This PR is labeled reindex:partial and its description documents the specific reindexing procedure for anvilprod or requires a full reindex or is not labeledreindex:anvilprod

Author (mirror)

  • This PR is labeled mirror:anvilprod or the changes introduced by it will not require mirroring of anvilprod
  • This PR is labeled mirror:partial and its description documents the specific mirroring procedure for anvilprod or requires a full mirroring or is not labeledmirror:anvilprod

Author (upgrading deployments)

  • This PR is labeled upgrade or does not require upgrading deployments
  • This PR is labeled deploy:shared or does not modify docker_images.json, and does not require deploying the shared component for any other reason
  • This PR is labeled deploy:gitlab or does not require deploying the gitlab component
  • This PR is labeled deploy:runner or does not require deploying the runner image

Author (before every review)

  • PR branch is up to date (if not, merge anvilprod into PR branch to integrate upstream changes)
  • PR is not a draft
  • PR is awaiting requested review from system administrator
  • Status of PR is Review requested
  • PR is assigned to only the system administrator and the author

System administrator (after approval)

  • Actually approved the PR
  • Decided if PR can be labeled no sandbox
  • N reviews label is accurate
  • Status of PR is Approved
  • PR is assigned to only the operator and the author

Operator

  • Pushed PR branch to GitHub

Operator (deploy .shared and .gitlab components)

  • Ran _select anvilprod.shared && CI_COMMIT_REF_NAME=anvilprod make -C terraform/shared apply_keep_unused or this PR is not labeled deploy:shared
  • Ran _select anvilprod.gitlab && python scripts/create_gitlab_snapshot.py --no-restart (see operator manual for details) or this PR is not labeled backup:gitlab
  • Ran _select anvilprod.gitlab && CI_COMMIT_REF_NAME=anvilprod make -C terraform/gitlab apply(an error from _login_docker_gitlab is benign if the instance was stopped for backup) or this PR is not labeled deploy:gitlab
  • Checked the items in the next section or this PR is labeled deploy:gitlab
  • PR is assigned to only the system administrator and the author or this PR is not labeled deploy:gitlab

System administrator (post-deploy of .gitlab component)

  • Background migrations for anvilprod.gitlab are complete or this PR is not labeled deploy:gitlab
  • PR is assigned to only the operator and the author

Operator (deploy runner image)

  • Ran _select anvilprod.gitlab && make -C terraform/gitlab/runner or this PR is not labeled deploy:runner

Operator (sandbox build)

  • Added sandbox label or PR is labeled no sandbox
  • Pushed PR branch to GitLab anvilprod or PR is labeled no sandbox
  • Build passes in hammerbox deployment or PR is labeled no sandbox
  • Reviewed build logs for anomalies in hammerbox deployment or PR is labeled no sandbox
  • Applied upgrade instructions from UPGRADING.rst to hammerbox or this PR is not labeled upgrade, or upgrade instructions do not apply to hammerbox
  • In hammerbox, deleted the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • In hammerbox, deindexed the sources sepcified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • In hammerbox, indexed the sources specified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • In hammerbox, indexed the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • Started full reindex in hammerbox or this PR is not labeled reindex:anvilprod or it is labeled reindex:partial
  • Checked for failures in hammerbox or this PR is not labeled reindex:anvilprod
  • Started mirroring in hammerbox or this PR is not labeled mirror:anvilprod
  • Checked for failures in hammerbox or this PR is not labeled mirror:anvilprod

Operator (merge the branch)

  • All status checks passed and the PR is mergeable
  • The title of the merge commit starts with the title of this PR
  • Added PR # reference to merge commit title
  • Collected commit title tags in merge commit title but excluded any p tags
  • Pushed merge commit to GitHub
  • Status of PR is Merged stable

Operator (main build)

  • Pushed merge commit to GitLab anvilprod
  • Build passes on GitLab anvilprod
  • Reviewed build logs for anomalies on GitLab anvilprod
  • Applied upgrade instructions from UPGRADING.rst to anvilprod or this PR is not labeled upgrade, or upgrade instructions do not apply to anvilprod
  • Ran _select anvilprod.shared && make -C terraform/shared apply or this PR is not labeled deploy:shared
  • Deleted PR branch from GitHub
  • PR is assigned to only the operator
  • Deleted PR branch from GitLab anvilprod
  • Status of linked issue is Stable
  • Status of promoted1 PRs is Merged stable
  • Status of promoted1 issues is Stable

1 Promoted issues and PRs are referenced in the titles of the commits
that the promotion branch introduces to the stable branch. Prior to the
promotion, the status of promoted issues (PRs) is Lower (Merged lower).
Promoted PRs in status Done do not need to be moved.

Operator (reindex)

  • In anvilprod, deleted the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • In anvilprod, deindexed the sources sepcified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • In anvilprod, indexed the sources specified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • In anvilprod, indexed the catalogs specified in the notes or this PR is missing either the reindex:partial or the reindex:anvilprod label, or both
  • Started full reindex in anvilprod or this PR is not labeled reindex:anvilprod or it is labeled reindex:partial
  • Checked for, triaged and possibly requeued messages in both fail queues in anvilprod or this PR is not labeled reindex:anvilprod or it is labeled reindex:partial
  • Emptied fail queues in anvilprod or this PR is not labeled reindex:anvilprod or it is labeled reindex:partial
  • Restarted the Data Browser pipeline for the ucsc/anvil/anvilprod branch on GitLab in anvilprod or this PR is not labeled reindex:anvilprod
  • Restarted deploy_browser job in the GitLab pipeline for this PR in anvilprod or this PR is not labeled reindex:anvilprod

Operator (mirroring)

  • Started mirroring in anvilprod or this PR is not labelled mirror:anvilprod
  • Checked for, triaged and possibly requeued messages in mirror fail queue in anvilprod or this PR is not labelled mirror:anvilprod
  • Emptied mirror fail queue in anvilprod or this PR is not labelled mirror:anvilprod

Operator

  • PR is assigned to only the system administrator

System administrator

Shorthand for review comments

  • L line is too long
  • W line wrapping is wrong
  • Q bad quotes
  • F other formatting problem

hannes-ucsc and others added 30 commits August 14, 2026 00:58
These targets ran scripts/rename_resources.py and
scripts/import_default_vpc.py as part of every plan/apply/destroy.
Both scripts are no longer needed and have been moved to the attic.
All downstream targets now depend directly on validate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move aws_kms_key and aws_kms_alias resources from the main component
into a new `base` component that is applied before `make lambdas`.
This eliminates the chicken-and-egg problem where Lambda packaging
needs the KMS keys to exist but they were created by the same
Terraform apply that depends on the Lambda packages.

The main component now uses data source lookups for the KMS key ARNs
and `removed` blocks to drop the resources from its state without
destroying them. The base component uses `import` blocks to adopt the
existing keys during the first apply.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ul-private#377)

The test for the unit `_log_request` reproduces the leak of the
`authorization` header value in the `Received … request` log message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ul-private#377)

The `CaseInsensitiveMapping` branch of `_LogJSONEncoder` serialized
request headers verbatim. Apply `redact_header` (renamed from the
private `_redact_header`) to each header value, using the same policy
as `LoggingHttpClient`: pattern-based redaction, falling back to full
redaction for `authorization` headers with unrecognized secret types.

Also rename `_redact_headers` to `redact_headers` and add a
`Mapping[str, str]` type hint to its `headers` parameter.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Issue dependencies are now expressed with GitHub's built-in issue
dependencies, PR merge order with the field of the same name in the Azul
project, and pipelines with the statuses of that project.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Developers may still chain PRs while working on them, but a PR must be
unchained and its base PR merged before review of it can be requested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…updates

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@hannes-ucsc hannes-ucsc added reindex:anvilprod [process] PR requires reindexing anvilprod upgrade [process] PR includes commit requiring manual upgrade labels Aug 27, 2026
@hannes-ucsc hannes-ucsc self-assigned this Aug 27, 2026
@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.63557% with 114 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.54%. Comparing base (38041a3) to head (5e419ec).
⚠️ Report is 1 commits behind head on anvilprod.

Files with missing lines Patch % Lines
test/integration_test.py 0.00% 52 Missing ⚠️
src/azul/indexer/cache_service.py 79.47% 47 Missing ⚠️
src/azul/terra.py 54.54% 5 Missing ⚠️
src/azul/oauth2.py 33.33% 2 Missing ⚠️
src/azul/plugins/metadata/hca/indexer/aggregate.py 96.15% 2 Missing ⚠️
src/azul/lib/urls.py 94.44% 1 Missing ⚠️
src/azul/plugins/repository/tdr_anvil/__init__.py 97.56% 1 Missing ⚠️
src/azul/service/manifest_service.py 50.00% 1 Missing ⚠️
src/azul/service/user_controller.py 80.00% 1 Missing ⚠️
src/azul/service/user_service.py 50.00% 1 Missing ⚠️
... and 1 more
Additional details and impacted files
@@              Coverage Diff              @@
##           anvilprod    #8262      +/-   ##
=============================================
- Coverage      84.54%   84.54%   -0.01%     
=============================================
  Files            166      169       +3     
  Lines          24577    25248     +671     
=============================================
+ Hits           20779    21345     +566     
- Misses          3798     3903     +105     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coveralls

coveralls commented Aug 27, 2026

Copy link
Copy Markdown

Coverage Status

coverage: 84.619% (-0.007%) from 84.626% — promotions/2026-08-25-anvilprod into anvilprod

@hannes-ucsc hannes-ucsc added API API change affecting callers sandbox [process] Resolution is being verified in sandbox deployment labels Aug 27, 2026
@hannes-ucsc
hannes-ucsc force-pushed the promotions/2026-08-25-anvilprod branch 2 times, most recently from 3ed7782 to 273ddbe Compare August 28, 2026 22:09
#8245 (comment)

Derive HTTP timeout from Lambda context's remaining time

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@hannes-ucsc
hannes-ucsc force-pushed the promotions/2026-08-25-anvilprod branch from 273ddbe to 2aeed9a Compare August 29, 2026 03:52
hannes-ucsc and others added 2 commits August 28, 2026 23:32
@hannes-ucsc
hannes-ucsc force-pushed the promotions/2026-08-25-anvilprod branch from 2aeed9a to e58c1c7 Compare August 29, 2026 06:32
hannes-ucsc and others added 4 commits August 30, 2026 18:29
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…size (DataBiosphere/azul-private#414)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ate#377)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@hannes-ucsc
hannes-ucsc force-pushed the promotions/2026-08-25-anvilprod branch from 38a175c to fab6d3e Compare September 2, 2026 16:14
@hannes-ucsc hannes-ucsc linked an issue Sep 3, 2026 that may be closed by this pull request
7 tasks
…ng (#8282)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@hannes-ucsc hannes-ucsc changed the title Promotion 2026-08-25 anvilprod (#8255) Promotion 2026-08-25 anvilprod (#8245, #8255, #8282) Sep 5, 2026
@hannes-ucsc hannes-ucsc changed the title Promotion 2026-08-25 anvilprod (#8245, #8255, #8282) Promotion 2026-08-25 anvilprod (#8245, #8255, #8282, DataBiosphere/azul-private#377, DataBiosphere/azul-private#414) Sep 5, 2026
@hannes-ucsc
hannes-ucsc merged commit 36a7013 into anvilprod Sep 5, 2026
9 checks passed
@hannes-ucsc
hannes-ucsc deleted the promotions/2026-08-25-anvilprod branch September 5, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API API change affecting callers reindex:anvilprod [process] PR requires reindexing anvilprod sandbox [process] Resolution is being verified in sandbox deployment upgrade [process] PR includes commit requiring manual upgrade

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Redundant TDR requests for snapshot info during indexing Promotion 2026-08-25 Concurrent cache fetch retries waste work

4 participants