Description
config.splitting is part of the redaction policy schema and this port never reads it. There is no
split service, and splitting appears nowhere in phileas/. A policy configuring it is accepted,
round-trips intact, and is filtered as though the setting were absent.
The round-trip is what makes it quiet. Policy keeps the config object whole
(phileas/policy/policy.py:94, "Raw config object, kept whole so nothing in it is dropped"), so
a policy with splitting enabled serializes back byte for byte. Nothing in the round-trip, the model,
or the filtering result indicates the setting had no effect.
What a policy author loses
- Documents are never split, whatever
threshold says, so a large document is filtered whole.
overlap does nothing, including the boundary-straddling entity recovery it exists for
(philterd/phisql schema 1.2.0).
method does nothing.
The Java and .NET ports both implement splitting, so a policy moved between ports behaves
differently in a way the policy itself gives no hint of.
Scope note
This is a larger gap than the split-method defects filed for the other two ports
(philterd/phileas#392, philterd/phileas-dotnet#105), which are about an unknown method name
silently falling back. Those do not apply here, because there is nothing to fall back from. They are
worth reading first anyway: both ports resolve the method name loudly now, and this port should not
reintroduce a silent fallback when it gains one.
Two things learned implementing splitting in .NET that are worth carrying over rather than
rediscovering:
- Locate each piece in the input and apply the replacements once to the original text. Filtering
pieces separately and concatenating them makes span offsets index the output rather than the input,
and loses the input's whitespace (philterd/phileas-dotnet#92).
- The piece locator has to skip exactly the characters the splitter trims, which is every
character at or below U+0020, not what a general "is whitespace" test reports. Testing the wrong
set silently drops the whole document onto the uncorrected path.
Acceptance criteria
Related
philterd/phileas-dotnet#92 and #82: the located-split mechanism and overlap, with a working
implementation and tests.
philterd/phileas-dotnet#105 and philterd/phileas#392: the unknown-method defect in the ports
that do implement splitting.
Description
config.splittingis part of the redaction policy schema and this port never reads it. There is nosplit service, and
splittingappears nowhere inphileas/. A policy configuring it is accepted,round-trips intact, and is filtered as though the setting were absent.
The round-trip is what makes it quiet.
Policykeeps the config object whole(
phileas/policy/policy.py:94, "Rawconfigobject, kept whole so nothing in it is dropped"), soa policy with splitting enabled serializes back byte for byte. Nothing in the round-trip, the model,
or the filtering result indicates the setting had no effect.
What a policy author loses
thresholdsays, so a large document is filtered whole.overlapdoes nothing, including the boundary-straddling entity recovery it exists for(
philterd/phisqlschema 1.2.0).methoddoes nothing.The Java and .NET ports both implement splitting, so a policy moved between ports behaves
differently in a way the policy itself gives no hint of.
Scope note
This is a larger gap than the split-method defects filed for the other two ports
(
philterd/phileas#392,philterd/phileas-dotnet#105), which are about an unknown method namesilently falling back. Those do not apply here, because there is nothing to fall back from. They are
worth reading first anyway: both ports resolve the method name loudly now, and this port should not
reintroduce a silent fallback when it gains one.
Two things learned implementing splitting in .NET that are worth carrying over rather than
rediscovering:
pieces separately and concatenating them makes span offsets index the output rather than the input,
and loses the input's whitespace (
philterd/phileas-dotnet#92).character at or below U+0020, not what a general "is whitespace" test reports. Testing the wrong
set silently drops the whole document onto the uncorrected path.
Acceptance criteria
config.splittingis bound on the policy model:enabled,threshold,method,overlap.thresholdis split before filtering whenenabledis true, and is notwhen it is false.
methodselects the splitter, and an unrecognised name raises rather than silently selectinganother.
overlapgives each piece after the first the trailing characters of the previous one, andspans duplicated at a seam are de-duplicated.
every overlap, including a policy whose replacements change the text length.
redact is returned exactly as it was.
spec/v1.2.0/examples/splitting-overlap.jsonboth round-trips and takes effect.Related
philterd/phileas-dotnet#92and#82: the located-split mechanism and overlap, with a workingimplementation and tests.
philterd/phileas-dotnet#105andphilterd/phileas#392: the unknown-method defect in the portsthat do implement splitting.