Skip to content

Preserve Text subclass behavior when compiling models - #191

Open
narayana006 wants to merge 1 commit into
jsvine:masterfrom
narayana006:candidate/preserve-compiled-text-subclasses
Open

narayana006 wants to merge 1 commit into
jsvine:masterfrom
narayana006:candidate/preserve-compiled-text-subclasses

Conversation

@narayana006

Copy link
Copy Markdown

Non-in-place Text.compile() reconstructs a plain Text object, losing a custom model's class, formatting hooks and constructor configuration. For example, a word_join hook that removes POS tags emits clean words before compilation but raw tagged tokens afterward.

Shallow-copy the model and replace its chain with the independently compiled chain. This preserves subclass behavior and configuration without invoking a constructor that may require additional arguments. Mutable custom attributes retain shallow-copy semantics.

Seven regression/control tests cover subclass identity and sentence splitting, custom word output, required constructor configuration, retain_original=False, recompilation, original-chain independence and in-place compilation. Five fail before the fix; two controls pass. With the fix, all 92 tests pass, pytest-cov reports 100% statement coverage, and project make lint (Black and Flake8) passes on macOS/Python 3.11.15.

The full Python/platform CI matrix and NLTK/spaCy integrations were not run; the tagging regression uses a small deterministic subclass.

AI assistance: OpenAI Codex assisted with investigation, implementation, regression tests and local validation. Publication was explicitly approved by the contributor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant