Leaderboard submission: THE DETECT LAW v8.1 (zero-shot, Llama-3.2-1B + 1B-Instruct)

[devkancheti4-design]

Adds leaderboard/submissions/the-detect-law-v81/ (predictions.json + metadata.json) for all 672,000 test ids.

THE DETECT LAW v7 was merged in #212. v8.1 replaces its decision law and keeps the same two models.

Detector. Zero-shot. It has no training on RAID or on any detection data. A compiled integer (int32) sequential law reads per-token statistics from two small open models:

• A = Llama-3.2-1B (base): surprisal L, entropy E and variance V.
• B = Llama-3.2-1B-Instruct: the log-probability contrast D against A, with its mean K and variance VC under A.

There are three arms. Each sums a centred statistic X = x − c·U:

• surprise: x = clip(L − E, ±8 bits), U = V
• contrast: x = clip(D − K, ±6 bits), U = min(VC, 36)
• repetition: x = q − R, U = q(1 − q). Here R = 1 if the token already appeared in the window, and q = A's probability mass on tokens already seen.

Each arm is normalised with a variance that allows for differences between texts: ρ·W + κ²·W². The constants c, ρ and κ were measured on human RAID training texts only.

The submitted score is √(law_score / 256) = √(Σ max(z_arm, 0)²). It counts only evidence toward AI, and higher means more likely machine-generated. It ranks texts exactly as the law's integer law_score does, but on a z scale.

Why the z scale: find_threshold starts at the human mean and moves in fixed 0.5 steps, at most 50 of them. On the raw 256·z² scale it would stop near 30% FPR. Tested on 4,800 human RAID-train texts scored with this exact pipeline, the raw scale gives 28–36% FPR, while the submitted scale gives exactly 5.00% and 1.00% in every domain.

Input and run.

• Inputs are cleaned first: NFKC normalisation, removal of zero-width/format characters, homoglyph mapping, whitespace collapse.
• Each text is scored on its first 1,024 tokens in 200-token windows. Each window starts from a fresh BOS.
• Windows are fed to the law in order. Once the law reaches a final decision, the text's remaining windows are skipped, since the law ignores them.
• Models ran in fp16 on one A30 (rows taken from the end) plus 2× T4 on Kaggle (rows 0–279,999). Some rows were scored by both. On those 223,424 rows, 99.94% of verdicts and 89.7% of scores are identical, and the rest differ by fp16 rounding.

Texts that are empty after cleaning (only zero-width characters) get score 0.

🤖 Generated with Claude Code

[devkancheti4-design]

respected sir, thank u for merging the previous work this is the updated version and final version.

[devkancheti4-design]

sir,could u please eval this final version.

[devkancheti4-design]

sorry sir ,about the two times eval but this is the final version.

[github-actions]

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

THE DETECT LAW v8.1

Release date: 2026-10-07

I've committed detailed results of this detector's performance on the test set to this PR.

On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 96.48 and a TPR of 88.99% at FPR=5% and 81.37% at FPR=1%.
Without adversarial attacks, it achieved AUROC of 97.61 and a TPR of 92.33% at FPR=5% and 86.54% at FPR=1%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

[devkancheti4-design]

Sir,Thank you for evaluating v8 and sharing the results. I would appreciate
it if you could merge the pull request and publish the results on the RAID
leaderboard as soon as possible.

Thank you