feat: Allow FailureInfo to persist only failing rule columns

[Danila Pechenev (Danila-Pechenev)]

Motivation

Fixes #296.

Persisted FailureInfo objects can become very wide because they include an output column for every validation rule, even when most rules never fail.

Changes

• Add the keyword-only only_invalid_rules option to FailureInfo.write_parquet().
• When enabled, persist all data columns but only rule columns containing at least one False.
• Keep the parquet metadata consistent with the retained rule columns.
• Preserve existing lossless behavior by default.
• Support round-tripping reduced empty failure information.
• Document that the reduced representation intentionally omits successful and unknown-only rule outcomes.

Testing

• Added coverage for failing, successful, unknown, and empty rule columns.
• Added eager-read and lazy-scan round-trip coverage.
• Added repeated reduced-serialization coverage.
• Verified counts and co-occurrence counts remain unchanged.

[Copilot]

🟢 Approval recommended

The implementation matches the stated behavior, keeps parquet metadata consistent, preserves default lossless behavior, and includes targeted tests and documentation for key edge cases.

Pull request overview

This PR adds an opt-in serialization mode to reduce the width of persisted FailureInfo parquet files by writing only rule-output columns that actually contain at least one failing (False) value, addressing issue #296.

Changes:

• Added keyword-only only_invalid_rules to FailureInfo.write_parquet() to omit always-successful / unknown-only rule columns when enabled.
• Adjusted FailureInfo.details() to handle the edge case of an empty rule_columns list.
• Added tests and documentation covering reduced serialization, metadata consistency, and round-tripping (including empty failure info).

File summaries

File	Description
tests/failure_info/test_parquet.py	Adds fixtures and tests for reduced parquet serialization, metadata, and round-trip behavior (eager + scan).
docs/guides/features/serialization.md	Documents the new only_invalid_rules=True option and its intentionally lossy semantics.
dataframely/filter_result.py	Implements only_invalid_rules in FailureInfo.write_parquet() and handles empty rule-columns in details().

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Oliver Borchert (borchero)]

Thank you for the contribution Danila Pechenev (@Danila-Pechenev)! Just some small suggestions

[Oliver Borchert (borchero)]

Why is this necessary?

[Danila Pechenev (Danila-Pechenev)]

This is needed for an empty FailureInfo persisted with only_failing_rules=True. Since no rule contains a failure, the reduced representation has rule_columns=[]. After reading/scanning it back, details() would otherwise call pl.col() and raise a TypeError. Returning invalid() makes details() work for that reduced state as well.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (2b03dad) to head (1ada3f9).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #400   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           46        46           
  Lines         2590      2598    +8     
=========================================
+ Hits          2590      2598    +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.