Skip to content

be tries a download again when GitHub answers 504 (v0.12.0) - #11

Merged
igorlamos merged 1 commit into
devfrom
feature/download-retries
Nov 8, 2026
Merged

igorlamos merged 1 commit into
devfrom
feature/download-retries

Conversation

@beplus-agent-claude

Copy link
Copy Markdown
Contributor

Pull Request

Problem

GitHub answers release downloads with an HTTP 504 now and then, most often to Actions runners, and be made each request once, so a single 504 failed the whole install (be auto, be latest, …). actions/setup-node rides out the same 504s by waiting and trying again:

Unexpected HTTP response: 504
Waiting 20 seconds before trying again

Solution

Every request be makes now goes through fetch in bin/be: the tarball's preflight (HEAD), its SHA256SUMS, the tarball itself and the release index.

  • Tried again: HTTP 408, 429, 500, 502, 503 and 504, a host that does not resolve or answer, a dropped connection, a timeout. Up to 5 attempts in all, waiting 5, 10, 20 and 40 seconds (about 75 seconds).

  • Not tried again: any other answer, such as a 404, which fails at once as before.

  • Settings: BE_DOWNLOAD_ATTEMPTS (1 turns retrying off) and BE_DOWNLOAD_RETRY_DELAY (the first wait in seconds, doubling after that). be doctor shows both. A value that is not a whole number is refused.

  • Log: each retry is a line on stderr, shown with --quiet too, and a final failure says gave up:

    curl: (22) The requested URL returned error: 504
        retrying : https://github.com/beplus/cli/releases/download/v2.0.0/beplus-cli-v2.0.0-macos-arm64.tar.gz answered HTTP 504; attempt 2 of 5 in 5 seconds
    
  • No half bodies: a release index or SHA256SUMS is output only once an attempt succeeds, and each attempt starts the tarball file afresh.

  • wget: be makes one try at a time (--tries=1) and reads the status from --server-response. GNU wget used to retry a 504 twenty times on its own but never a 503. BusyBox wget accepts both options.

Tests

test/tests/retries.bats (13 tests) runs against test/tests/flaky-mirror.js, a small node server on 127.0.0.1 that answers 504 (or any status) or drops the connection, following a plan file per path, and logs every request. That way the tests check exactly how many requests each failure causes. One test hides curl from PATH to run the wget path; it skips where wget is missing.

  • Pass on macOS with curl, under bash 5 and the system bash 3.2.
  • Pass in the ubuntu-wget test container (GNU wget 1.25, no curl), with nodejs added.
  • Against v0.11.0's be, the 9 retry tests fail and the 4 "fails at once" tests pass.

The whole suite passes locally under bash 5 and bash 3.2. While this was written, GitHub answered beplus/cli release downloads with 504s on and off for about 45 minutes, and some spells lasted longer than the 75-second retry window. So the tests that download real releases (install-contents, install-options, install-versions, run-which, uninstall) passed only once GitHub was back. The same runs show the retries getting through two real failures, each of which used to fail the install:

  • the v2.0.0 tarball's preflight answered 504 twice and went through on attempt 3;
  • the S3 release index reset the TLS connection (curl exit 35) and went through on attempt 2.

A longer outage still fails after 5 attempts; BE_DOWNLOAD_ATTEMPTS=8 in CI stretches the window to about 10 minutes.

Release

  • Releases v0.12.0: ran bin/bump minor and described it in CHANGELOG.md
  • No release: bin/be is unchanged

GitHub answers release downloads with a 504 now and then, most often to
Actions runners, and one 504 failed the whole install. Every request be
makes now goes through fetch, which retries a 408, 429, 500, 502, 503 or
504, a host that does not resolve or answer, a dropped connection or a
timeout: up to 5 attempts, waiting 5, 10, 20 and 40 seconds. A 404 still
fails at once. BE_DOWNLOAD_ATTEMPTS and BE_DOWNLOAD_RETRY_DELAY tune it;
each retry is logged on stderr, --quiet or not.

With wget, be makes one try at a time (--tries=1) and reads the status
from --server-response, so a 503 is retried too and a 504 is not retried
twenty times by wget as well.

test/tests/retries.bats runs against flaky-mirror.js, a local server that
answers 504 or drops the connection on demand.

Co-Authored-By: Igor Lamos <igor@be.plus>
@igorlamos igorlamos self-assigned this Nov 8, 2026
@igorlamos
igorlamos merged commit e834d84 into dev Nov 8, 2026
4 checks passed
@igorlamos
igorlamos deleted the feature/download-retries branch November 8, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant