Skip to content

ci: run a release through trunk flakytests before promoting it - #1217

Merged
trunk-io[bot] merged 1 commit into
mainfrom
dylan/promote-check-trunk-flakytests
Oct 7, 2026
Merged

trunk-io[bot] merged 1 commit into
mainfrom
dylan/promote-check-trunk-flakytests

Conversation

@dfrankland

@dfrankland dfrankland commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Adds the precondition from this review thread. Before Promote Release marks a version latest, it runs TRUNK_ANALYTICS_CLI_VERSION=<version> trunk flakytests --version and requires the output to be that version.

It checks both trunks that customers install:

  • the bash launcher from https://trunk.io/releases/trunk, which reads TRUNK_ANALYTICS_CLI_VERSION from launcher 1.3.5 (trunk-io/trunk#34317)
  • the new CLI from https://install.trunk.io, which reads it from CLI 0.0.9 (trunk-io/trunk2#6728)

Both releases are live. Each trunk runs with its own empty TRUNK_CACHE, so the version is downloaded fresh. The step runs before AWS credentials are set up, so a failure leaves GitHub and S3 untouched.

@trunkio/launcher on npm and the Windows trunk.ps1 aren't checked. Neither has ever routed trunk flakytests to analytics-cli, so there is nothing there to pin.

Testing

I ran the step's run: block locally, extracted from this branch's promote_release.yml and run under bash -eo pipefail with CI=true, against the published launcher 1.3.5 and CLI 0.0.9:

VERSION Result
0.16.0 (latest) passes: both trunks report 0.16.0
0.15.6 passes: both report 0.15.6, so the pin is honoured rather than defaulting to latest
9.9.9 (no such release) fails: ::error::launcher could not download or run 9.9.9

The new CLI downloaded from trunk.io/releases/analytics-cli/ (published by #1216), without falling back to GitHub. actionlint (with shellcheck) is clean.

🤖 Generated with Claude Code

@trunk-staging-io

trunk-staging-io Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

Failed Test Failure Summary Logs
pending_quarantine_test should be quarantined when run with variant A test marked as pending was expected to fail but unexpectedly passed. Logs ↗︎
variant_quarantine_test should be quarantined when run with variant A test expected the sum of 2 + 2 to be 5, but it was actually 4, indicating a failing assertion. Logs ↗︎

View Full Report ↗︎ ⋅ Docs

@trunk-io

trunk-io Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Static Badge   Static Badge   Static Badge

View Full Report ↗︎ ⋅ Docs

@codecov-commenter

codecov-commenter commented Oct 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.06%. Comparing base (23474de) to head (d3e6de4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1217      +/-   ##
==========================================
+ Coverage   83.86%   84.06%   +0.19%     
==========================================
  Files          74       74              
  Lines       17888    17888              
==========================================
+ Hits        15002    15037      +35     
+ Misses       2886     2851      -35     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dfrankland
dfrankland marked this pull request as ready for review October 6, 2026 20:46
@dfrankland
dfrankland changed the base branch from dylan/publish-releases-to-s3 to main October 7, 2026 04:18
@trunk-io

trunk-io Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

😎 Merged successfully - details.

Both `trunk`s customers install, the bash launcher and the new CLI, must
run the version pinned with TRUNK_ANALYTICS_CLI_VERSION before it becomes
latest. Needs trunk-io/trunk#34317 and trunk-io/trunk2#6728 released.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@dfrankland
dfrankland force-pushed the dylan/promote-check-trunk-flakytests branch from 7fd3841 to d3e6de4 Compare October 7, 2026 04:19
@trunk-io
trunk-io Bot merged commit dda4670 into main Oct 7, 2026
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants