Skip to content

fix(data): validate explicit split columns - #196

Open
DevChiniwala wants to merge 1 commit into
plexe-ai:mainfrom
DevChiniwala:fix/validate-explicit-split-schema
Open

DevChiniwala wants to merge 1 commit into
plexe-ai:mainfrom
DevChiniwala:fix/validate-explicit-split-schema

Conversation

@DevChiniwala

Copy link
Copy Markdown

Problem

Explicit validation or test datasets that omit a training target or feature pass Phase 2 preparation. The schema check removed when explicit split support was added in #189 was left as a TODO, so invalid data reaches later workflow stages before producing a useful error.

Root cause

prepare_data materializes explicit splits but does not check that they contain the training columns before splitting or sampling.

Fix

Validate each explicit split against the training column names immediately after materialization. Raise a ValueError that identifies the split and missing columns. Split-only extra columns remain accepted.

Regression coverage

The deterministic unit test covers missing targets and features in both explicit validation and test splits, and asserts validation occurs before downstream splitting or sampling. Existing explicit-split coverage verifies extra columns remain accepted.

Validation

  • pytest tests/unit/ — 113 passed, 14 skipped (Python 3.12 project environment)
  • black --check . — passed
  • ruff check . — passed
  • git diff --check — passed
  • Staged integration suite not run; it requires live provider credentials.

Follow-up to the explicit split workflow in #189.

Copilot AI balanced review requested due to automatic review settings October 10, 2026 05:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants