eu-drift-watch

Cross-EU statute-watch for the eu-legal-mcp connector family. Once a month it checks that a registry of anchor statutes still exists in each national source under the title we recorded. A renamed or repealed act trips the check.

Inspired by Aku Nikkola's Säädösvahti (Claude for Legal Finland), which watches one jurisdiction against one source. The difference here: nine EU jurisdictions at once (DE, AT, ES, FI, IE, NL, SE, LU, FR). That is a trust signal a single-country watcher cannot give.

(The badge reads badge.json, refreshed by CI on every run.)

How it works

Each jurisdiction has a registry, registry/<cc>.json, listing anchor acts with the stable identifier the national connector understands. check.mjs is a generic MCP-stdio harness: it spawns the <cc>-eli-mcp connector over uvx, calls its fetch-by-id tool with the registry's arguments verbatim, and confirms the expected title is still in the response.

node check.mjs registry/fi.json
# [OK   ] TSL2018  (get) -> Dataskyddslag
# FI: 4/4 match the registry

node aggregate.mjs        # every keyless jurisdiction -> report.json + badge.json

The registry is pure data: each act is { id, tool?, args, expect, mode? } and the harness passes args straight through as the tool-call arguments. A new jurisdiction is a JSON file, not a code change.

Two modes

• get (default, deterministic): fetch exactly one act by its stable id and check expect as a substring of any string in the response. There is only one act in the reply, so a broad match is safe. This is the strong signal.
• search (fallback): when a connector has no fetch-by-id tool, query its search tool and check expect against the titles of the top results. This is noisier, so it is used only where it has to be (see Austria below).

Coverage

Jur	Acts	Mode	Source	Stable id
DE	4	get	NeuRIS (rechtsinformationen.bund.de)	ELI
AT	5	search	RIS (data.bka.gv.at)	- (search only)
ES	4	get	BOE (boe.es)	BOE-A-... id
FI	4	get	Finlex (opendata.finlex.fi)	year + number
IE	4	get	Irish Statute Book	year + number + type
NL	4	get	KOOP SRU (BWB)	BWBR... id
SE	4	get	Riksdagen (data.riksdagen.se)	SFS number
LU	3	get	Legilux (data.legilux.public.lu)	ELI (native)
FR	1	get	Legifrance via PISTE	LEGITEXT id (credentials)

Eight jurisdictions are keyless and run in CI. France needs PISTE OAuth credentials, so it is excluded from the default run; pass them in the environment and add --all to include it.

What we learned building it

• A deterministic fetch beats a search. The first version searched by name and got false hits: searching "Grundgesetz" matched a different act with that word in its title. Anchoring on a stable id and fetching exactly that act removes the ambiguity. Registries hold ids, not phrases.
• The baseline is a live snapshot, not a guess. expect is the title the source returns today, recorded verbatim, not the title we assumed. Finland is bilingual and its connector returns the Swedish canonical form (Dataskyddslag, not Tietosuojalaki); Spain's Civil Code comes back as Real Decreto de 24 de julio de 1889 por el que se publica el Código Civil. Recording what the source actually says is the whole point: that is the line a future rename has to cross.
• Some sources don't carry the famous codes. NeuRIS is still in beta and does not yet hold the BGB, StGB or Grundgesetz, so the German anchors are flagship federal acts that do resolve (BDSG, VStGB, GmbHG, DDG).
• Austria is a weaker signal, and says so. The Austrian connector has no fetch-by-ELI, only RIS search, so AT runs in search mode against gazette records. It confirms the act is still named in the gazette, not that a specific consolidated version is in force. That is a softer guarantee than the other eight, and fixing it needs a fetch-by-id path on at-eli-mcp.

Run it yourself

# one jurisdiction
node check.mjs registry/de.json

# all keyless jurisdictions, write report.json + badge.json
node aggregate.mjs

# include France (needs PISTE credentials in the environment)
export FR_ELI_CLIENT_ID=... FR_ELI_CLIENT_SECRET=...
node aggregate.mjs --all

Requires Node 18+ and uv (for uvx). Each connector is fetched from PyPI on first run and cached.

CI

.github/workflows/drift-watch.yml runs aggregate.mjs on the first of each month (and on demand), then commits the refreshed badge.json back to main. France is included only if FR_ELI_CLIENT_ID / FR_ELI_CLIENT_SECRET are set as repository secrets.

The job distinguishes a real change from a flaky source. A government API that times out or 500s is retried, then classified as unreachable (yellow badge), and the job still passes - a momentary outage is not a legal drift and should not page anyone. Only a real drift - an act that resolves but no longer carries its recorded title - turns the badge red and fails the job. So a failure email means a statute actually changed, not that a server hiccuped.

Registry format

{
  "jurisdiction": "FI",
  "source": "Finlex open data",
  "package": "fi-eli-mcp",
  "getTool": "fi_get_act",
  "acts": [
    { "id": "TSL2018", "args": { "year": 2018, "number": 1050 }, "expect": "Dataskyddslag" }
  ]
}

• package: the connector run via uvx.
• getTool / searchTool: default tool per mode (an act may override with tool).
• acts[].args: passed verbatim as the tool-call arguments. The tool signature differs per country (de_get_act{eli}, fi_get_act{year,number}, se_get_act{sfs_number}, ...); read the connector's server.py.
• acts[].expect: a distinctive substring of the title the source returns live.
• acts[].mode: get (default) or search.
• requiresCreds: true: registry excluded from the default run (needs credentials).

License: Apache-2.0. Maintainer: Wiesław Mazur / MateMatic Solutions.