bundle: keep resources.json in step with DMS, marked by a state feature - #6475
Draft
shreyas-goenka wants to merge 1 commit into
Draft
bundle: keep resources.json in step with DMS, marked by a state feature#6475shreyas-goenka wants to merge 1 commit into
shreyas-goenka wants to merge 1 commit into
Conversation
shreyas-goenka
force-pushed
the
isaac/dms-dual-state
branch
2 times, most recently
from
September 2, 2026 02:03
7047d22 to
1fd16ac
Compare
shreyas-goenka
force-pushed
the
isaac/pr6052-fixes
branch
2 times, most recently
from
September 2, 2026 02:30
a5b1a71 to
58257c8
Compare
shreyas-goenka
force-pushed
the
isaac/dms-dual-state
branch
5 times, most recently
from
September 2, 2026 13:03
cee981f to
f7d92ee
Compare
shreyas-goenka
force-pushed
the
isaac/pr6052-fixes
branch
from
September 2, 2026 13:11
84eb12b to
faf0962
Compare
shreyas-goenka
force-pushed
the
isaac/dms-dual-state
branch
3 times, most recently
from
September 2, 2026 14:16
855650e to
0af021f
Compare
shreyas-goenka
force-pushed
the
isaac/pr6052-fixes
branch
from
September 2, 2026 14:20
faf0962 to
bd1457e
Compare
Recording made the service the source of truth for resource state, but the state file was still written on every deploy. Mark it as depending on the recording feature so an unaware CLI refuses it instead of deploying over a deployment it would leave the service behind on, and use the marker rather than "has any resources" to decide whether an existing state can be recorded. While recording, the file's resource set is never read back: the service is asked on every open, and holding no deployment means the resources are created again. Turning recording off for a marked deployment is refused. The WAL needs no special handling - whatever a replay rebuilds is replaced by what the service holds. Two tests that seed a state file now run with recording on; the ones that cannot are opted out with their real reasons recorded. Co-authored-by: Isaac <no-reply@databricks.com>
shreyas-goenka
force-pushed
the
isaac/dms-dual-state
branch
from
September 2, 2026 15:26
0af021f to
0e11e45
Compare
shreyas-goenka
force-pushed
the
isaac/pr6052-fixes
branch
from
September 2, 2026 15:29
bd1457e to
3d3c602
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Stacked on #6094. The state file keeps tracking every resource the service records, and now says so: recording writes the
record_deployment_historyfeature intoresources.json(state_version: 3, using the feature-flag scaffolding already there). The migration guard now keys off that marker instead of "has any resources", so a state this CLI recorded is accepted and only an unmarked one is refused.Why
A CLI without this feature would otherwise deploy over a recorded deployment and leave the service behind. The feature list is exactly the existing forward-compat gate for that: an unaware CLI refuses the state and tells the user to upgrade. Keying the guard off the marker also makes the state file a usable fallback when the deployment is gone — the resource is updated rather than created a second time.
Tests
bundle/dms/recordshowsresources.jsoncarrying the marker alongside the full resource set;bundle/dms/existing-statecovers both sides of the guard;bundle/state/feature_flagscovers a marked state loading with recording off. Unit test for the known-vs-unknown feature split. Full acceptance suite green apart frombundle/templates/lakeflow-integrations, which fails identically on this PR's base (no network for a pypi download).This PR was written by Claude Code.