Repository navigation
fix(data): validate explicit split columns - #196
Open
DevChiniwala wants to merge 1 commit into
Open
DevChiniwala wants to merge 1 commit into
DevChiniwala wants to merge 1 commit into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Explicit validation or test datasets that omit a training target or feature pass Phase 2 preparation. The schema check removed when explicit split support was added in #189 was left as a TODO, so invalid data reaches later workflow stages before producing a useful error.
Root cause
prepare_datamaterializes explicit splits but does not check that they contain the training columns before splitting or sampling.Fix
Validate each explicit split against the training column names immediately after materialization. Raise a
ValueErrorthat identifies the split and missing columns. Split-only extra columns remain accepted.Regression coverage
The deterministic unit test covers missing targets and features in both explicit validation and test splits, and asserts validation occurs before downstream splitting or sampling. Existing explicit-split coverage verifies extra columns remain accepted.
Validation
pytest tests/unit/— 113 passed, 14 skipped (Python 3.12 project environment)black --check .— passedruff check .— passedgit diff --check— passedFollow-up to the explicit split workflow in #189.