Repository navigation
fix: detect prerequisite cycles and read each stackerfile once - #812
Open
NopeCharon wants to merge 1 commit into
Open
NopeCharon wants to merge 1 commit into
NopeCharon wants to merge 1 commit into
Conversation
NewStackerFiles() read a stackerfile and then recursed into its config.prerequisites without checking whether a prerequisite had already been read. Prerequisites that form a cycle (a.yaml -> b.yaml -> a.yaml, or a file listing itself) made the recursion run until the goroutine stack limit was hit and stacker died with "fatal error: stack overflow". The cycle check in NewStackerFilesDAG() was never reached. Shared prerequisites were also re-read and re-parsed once per path leading to them, which grows exponentially with the depth of the prerequisite graph. Load the stackerfiles into a single map, skipping files that have already been read, and keep track of the chain of prerequisites being followed. When a file shows up in its own chain, return an error that names the cycle, e.g. prerequisite cycle detected: /p/a.yaml -> /p/b.yaml -> /p/a.yaml Add unit tests for shared prerequisites and for both kinds of cycle. Fixes project-stacker#811 Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
NopeCharon
requested review from
hallyn,
mikemccracken,
raharper,
rchamarthy,
rchincha,
smoser and
tych0
as code owners
October 24, 2026 19:49
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #812 +/- ##
===========================================
- Coverage 55.06% 13.57% -41.49%
===========================================
Files 55 55
Lines 5911 5921 +10
===========================================
- Hits 3255 804 -2451
- Misses 2085 4982 +2897
+ Partials 571 135 -436 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
bug
Which issue does this PR fix:
Fixes #811
What does this PR do / Why do we need it:
NewStackerFiles()read a stacker file and then recursed into itsconfig.prerequisites, without checking whether a prerequisite had already been read.a.yaml→b.yaml→a.yaml, or a file listing itself) recursed until the goroutine stack limit was hit, and stacker died withfatal error: stack overflow. The cycle check inNewStackerFilesDAG()was never reached.This PR loads stacker files into a single map and skips files that have already been read. It also tracks the chain of prerequisites being followed, and returns an error naming the cycle when a file appears in its own chain:
The resulting map is the same as before for acyclic inputs, so the DAG, build order and duplicate-layer-name check are unchanged.
If an issue # is not available please add repro steps and logs showing the issue:
See #811.
Testing done on this change:
Added
pkg/types/stackerfiles_test.go:TestNewStackerFilesPrerequisites:crequiresaandb, andbrequiresa. All three files are loaded and keyed by absolute path.TestNewStackerFilesPrerequisiteCycle: a two-file cycle and a self-reference each return the cycle error.Without the fix, the cycle test crashes:
With the fix,
go test ./pkg/types/passes (on linux/arm64, using the Makefile's go-test build tags):Automation added to e2e:
No. The existing
test/prerequisites.batscovers the acyclic build order, and the new behavior is covered by Go unit tests.Will this break upgrades or downgrades?
No.
Does this PR introduce any user-facing change?:
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.