Remove the duplicate NN-NNNNNNN pattern from the SSN filter

[jzonthemtn]

Description

ssn_filter.py and ein_filter.py compile the same regular expression, and each attributes it to
a different entity type.

phileas/filters/ein_filter.py:34:

_PATTERNS = [
    # Canonical EIN: NN-NNNNNNN. A neighbouring hyphen makes it part of a longer
    # identifier, not an EIN (philterd/phileas#343).
    re.compile(r"(?<![\w-])\d{2}-\d{7}(?![\w-])"),
]

phileas/filters/ssn_filter.py:41:

# TIN: NN-NNNNNNN, the shape the ein filter also detects. Scored below the SSN
# forms so `ein` wins the span wherever both filters are enabled.
_TIN_PATTERNS = [
    re.compile(r"(?<![\w-])\d{2}-\d{7}(?![\w-])"),
]

_TIN_CONFIDENCE = 0.90

The comment is candid about the duplication: the copy in the SSN filter exists only to be outscored
by the filter that already detects the same thing. Span.drop_overlapping_spans keeps the
highest-confidence span, and EINFilter runs at the default 1.0 against this 0.90, so the ein
span wins whenever ein is enabled. When it is not, a value matching NN-NNNNNNN is reported as an
SSN.

Why remove it rather than keep it as a fallback

PhiSQL's entity catalog declares no TIN entity type and describes NN-NNNNNNN as the EIN format.
It is: ITINs and ATINs are SSN-shaped (NNN-NN-NNNN) and PTINs are P followed by eight digits, so
the only taxpayer number written NN-NNNNNNN is an EIN. A span reported as ssn for that value is
mislabelled, gets the SSN filter's strategies applied, and reaches a reviewer as a Social Security
Number.

EINFilter already exists, is registered in _BUILTIN_FILTERS (filter_service.py:71), and its
pattern already carries the adjacent-hyphen boundary fix. Nothing has to be built: the fix is to
delete the copy.

Behaviour change to call out

A policy that enables ssn and not ein stops detecting NN-NNNNNNN. That is the intent, but it
is a detection regression for anyone relying on the current behaviour and needs a release note that
says so plainly.

Acceptance criteria

• _TIN_PATTERNS and _TIN_CONFIDENCE are removed from phileas/filters/ssn_filter.py, and
  SSNFilter.detect returns only the SSN forms.
• A value matching NN-NNNNNNN produces a span typed ein, never ssn.
• EINFilter is unchanged: it already covers the shape and already excludes an adjacent hyphen.
• A test asserts that a policy enabling only ssn does not detect 12-3456789, and that one
  enabling ein does.
• Existing SSN tests still pass unchanged, confirming no SSN form depended on the removed
  pattern.
• The final redacted text is asserted, not only the detected spans.
• Any documentation describing the SSN filter as detecting TINs is corrected, and the EIN
  documentation is the place the NN-NNNNNNN form is described.
• RELEASE_NOTES.md records the change, stating that a policy enabling only ssn no longer
  detects NN-NNNNNNN.

Related

• philterd/phileas#390 does the same removal in the Java port, where the pattern also has to be
  moved rather than deleted, because that port's EinFilter lacks the hyphen and wrap handling
  this port's already has.
• philterd/phileas-dotnet#95 covers the .NET port, which never carried the duplicate.

[nazasnow]

Hi @maintainers! I've analyzed this issue and have a proposed solution ready adhering to the requirements.

Proposed Plan: I will remove the duplicate NN-NNNNNNN pattern from the SSN filter in ssn_filter.py and ein_filter.py to avoid unnecessary duplication and potential confusion, ensuring that the SSN filter accurately identifies NN-NNNNNNN as an EIN.

Could you kindly assign this to me (@nazasnow)? I'll submit a tested Pull Request promptly. Thanks!

[jzonthemtn]

@nazasnow Thanks for the interest - another PR came in before I saw your request to be assigned.