Replication package for the manuscript Verification, Safety, and Governance of Agentic AI Systems in Safety-Critical Infrastructure (under review, JAAMAS).
It contains two things.
- SAS-Shield — an implementation of the Safety Assurance Stack described in the manuscript, together with the evaluation protocol the manuscript specifies: three baselines, the full per-layer ablation lattice, 30 seeds per arm with confidence intervals, a CSDS curve over eleven shift magnitudes, and an adaptive adversary reported as a function of query budget.
- The systematic review dataset — the coding sheets for the 95 primary studies and 12 grey-literature instruments, and the bibliography.
Every number in the pilot-instantiation section of the manuscript is produced by
experiments/run_experiments.py and its companions and is written to results/.
Nothing in results/ is edited by hand.
Read this before quoting any number from it.
The planner is not a language model. The reported arms use
ScriptedPlanningAgent, a deterministic instruction-following surrogate: it consumes
the natural-language perception text, and it will adopt a directive found in that text
in preference to its own policy, which is the behaviour that makes indirect prompt
injection possible at all. The surrogate buys exact reproducibility and zero inference
cost. It does not exercise the semantic failure modes specific to any particular
model, and no result here is evidence about one. An optional LLMPlanningAgent is
provided for groups who wish to run that arm; it is disabled by default, it raises
rather than silently falling back, and it produced none of the committed results.
results/run_manifest.json records zero LLM calls and zero tokens.
The testbed is pedagogical scale. IEEE 14-bus, eleven loads, five dispatch units. The manuscript is explicit that this scale is adequate for the ablation and adversarial arms and inadequate for any claim about production-scale performance. No claim about production scale is made.
Latency criterion F4 is not tested. Measured decision latency is the time spent in perception, planning and Layers 1-5, and it is well under a millisecond because there is no model inference in the loop. The 1,000 ms budget the manuscript sets is dominated by the reasoning engine this pilot does not include, so F4 is reported as untested rather than as passed.
The scenario suite was constructed to span four failure classes. It is not a sample from an operational distribution. That the layered stack outperforms any single layer on a suite built to span the classes the layers were designed for is a check that the implementation does what it claims, not a discovery.
SAS-Shield/
├── run_simulation.py # single-episode runner, for inspection only
├── experiments/
│ ├── run_experiments.py # scenarios x arms x seeds; the main results
│ ├── run_csds.py # CSDS curve over the shift family (F2)
│ ├── run_adversarial.py # attack success vs adversary query budget (F3)
│ ├── run_sensitivity.py # Layer 1 persistence sweep; conformal coverage;
│ │ # quadratic baseline vs full AC OPF
│ ├── make_figures.py # regenerates every figure and LaTeX table
│ └── sync_manuscript.py # fills and re-verifies the manuscript's numbers
├── sas_shield/
│ ├── scenarios.py # S1-S4 and the eta-parameterised shift family
│ ├── grid_env.py # IEEE 14-bus twin, contingencies, sensor model
│ ├── memory.py # Phi_M with the belief argument and a staleness bound
│ ├── agent.py # scripted surrogate, incumbent controller, LLM adapter
│ ├── adversary.py # adaptive white-box injection suite
│ ├── layer1_conformal.py # split-conformal perception test
│ ├── layer2_guardrail.py # semantic rail: bounds, authority claims, justification
│ ├── layer3_fsa.py # finite-state monitor; native and NuSMV back-ends
│ ├── layer4_simplex.py # CBF admissibility test and QP projection
│ ├── layer5_audit.py # Ed25519-signed, hash-chained JSONL log
│ ├── interpretability.py # measured IS by blind causal-chain reconstruction
│ ├── dispatch.py # shared economic-dispatch target
│ ├── metrics.py, stats.py # metric definitions; bootstrap CIs and paired tests
│ ├── arms.py # baselines, ablation lattice, supplementary arms
│ ├── calibration.py # Layer 1 calibration and hold-out sets
│ └── orchestrator.py # the decision pipeline, configurable by arm
├── assets/
│ ├── grid_monitor.smv # Layer 3 reference model and its LTL properties
│ ├── guardrails.co, config.yml # optional NeMo Guardrails profile (not exercised)
├── data/
│ ├── qualitative_synthesis_coding_sheets.csv
│ └── references.bib
├── docs/
│ ├── PRISMA_flowchart.md
│ ├── qualitative_synthesis_coding_sheets.md
│ ├── RESULTS.md # generated summary of results/
│ └── figures/
├── results/ # generated; see below
└── tests/
├── test_layers.py # per-layer properties
└── test_orchestration.py # end-to-end properties over the orchestrator
| Layer | Mechanism | Guarantee actually delivered | Response on intercept |
|---|---|---|---|
| 1 | Split-conformal test on the demand and voltage channels, calibrated on a separate nominal environment across the admissible demand range, thinned to reduce AR(1) dependence | Marginal 1 − α coverage conditional on exchangeability; suspended off the calibrated topology | Escalate after L1_PERSISTENCE consecutive flags |
| 2 | Policy bounds, authority-claim lexicon, and a justification rule comparing the action class against observed telemetry | None. A heuristic rail with a measured miss rate | Refuse; substitute a null queryPMU |
| 3 | Finite-state monitor over (mode, cumulative shed count, dwell), enforcing a cumulative-safety invariant and a bounded-liveness monitor | Formal, within the abstraction and within the bound | Refuse and escalate |
| 4 | Discrete-time CBF admissibility test plus QP projection onto the admissible interval | Fallback-to-baseline, not invariance: the baseline is an economic optimum with no invariance certificate | Project and proceed |
| 5 | Ed25519-signed, SHA-256 hash-chained JSONL record of the full decision tuple | Third-party verifiable, append-only under key custody | — |
Three of these differ from the earlier version of this package, and the differences are corrections rather than preferences.
- Layer 5 signs with Ed25519, not HMAC-SHA256. The manuscript withdraws the HMAC construction explicitly: a symmetric tag is verifiable only by a party that could also have forged it, so it cannot adjudicate the operator-versus-vendor dispute it was introduced to adjudicate.
- Layer 1's calibration set comes from a separate environment, not from advancing the environment about to be evaluated.
- Layer 3's model-checked path constrains the proposed action as an input, so the verdict depends on the action under test.
Layer 1 is a split-conformal test on the telemetry vector. It is not an Isolation Forest, and it does not consume a PMU stream distinct from the rest of the telemetry; an earlier version of this file claimed both.
pip install -r requirements.txt # or: pip install -e .[dev]
pytest -q # 32 tests
python run_simulation.py --scenario S3 --arm FULL --traceReproduce the full evaluation (about 80 minutes on 11 cores):
python experiments/run_experiments.py --seeds 30 --jobs 11
python experiments/run_csds.py --seeds 20 --jobs 11
python experiments/run_adversarial.py --seeds 30 --jobs 11
python experiments/run_sensitivity.py --seeds 15 --jobs 11
python experiments/make_figures.pyEvery measured quantity in the manuscript's pilot section was inserted from these
files by experiments/sync_manuscript.py, which records each substitution in
results/manuscript_numbers.json. After re-running the experiments:
python experiments/sync_manuscript.py --check ../Agentic.texre-reads that record, re-reads results/, and exits non-zero if any number in the
paper no longer matches what the code produces. No number reaches the manuscript by
being typed, so drift between the paper and this repository is a failing check rather
than something a reader has to notice. The four LaTeX tables in the pilot section are
written by make_figures.py into results/tables/ and appear in the manuscript
verbatim.
The first run builds and caches the Layer 1 calibration set and the derated branch
ratings under results/; later runs load them. Delete
results/calibration_set.npz and results/line_ratings.npz to force a rebuild.
reproduce_results.ipynb reads the generated results/ files and renders the tables
and figures used in the manuscript. It does not re-run the experiments, and it carries
stored outputs from an actual execution.
| File | Contents |
|---|---|
raw/episodes.csv |
one row per episode: the seed distribution itself, not a summary |
raw/csds_episodes.csv, raw/adversarial_episodes.csv |
the same for the two sweeps |
main_results.json |
per (scenario, arm) means with percentile bootstrap intervals, plus the full MTTUA distribution and its censoring fraction |
comparisons.json |
paired Wilcoxon tests and Cliff's delta for each declared comparison |
csds_curve.json |
CSDS at each of eleven shift magnitudes, and the F2 dominance verdict |
adversarial_sweep.json |
pooled attack success at each query budget, blocking layer attribution, and the F3 verdict |
sensitivity.json |
Layer 1 persistence sweep, realised conformal coverage, and the quadratic-baseline versus AC-OPF comparison |
run_manifest.json |
package and dependency versions, seeds, wall clock, and the LLM call and token counts (both zero) |
manuscript_numbers.json |
every number the manuscript quotes, with the results path it came from |
tables/*.tex |
the LaTeX tables the manuscript includes verbatim |
audit/*.jsonl |
retained signed audit logs for the audit sample; verify_log_integrity passes on each |
The pandapower conversion of MATPOWER case14 does not supply a usable operating
envelope, so two are declared here. Both are stated in sas_shield/grid_env.py next to
the code that applies them.
- Voltage. The band [0.94, 1.09] p.u. applies to the load buses only. The slack and generator terminal voltages are regulated setpoints fixed by the case data at values up to 1.09 p.u.; they are not free variables and would otherwise register a constant violation on every step of every arm, including the do-nothing arm.
- Thermal. The converted case ships branch ratings three to four orders of magnitude above base-case flow, so branch loading never binds. Each branch is derated to 1.15 times the largest current it carries anywhere in the economic operating envelope (demand 0.90 to 1.30 of nominal, with the single-unit perturbations ordinary redispatch passes through). Consequence: ordinary control on an intact network is thermally feasible, and a thermal violation is caused by a contingency, a demand excursion, or a dispatch no economic controller would choose.
Each episode begins from the operating point the incumbent controller holds, reached by three warm-start steps not counted against the horizon. The base case14 solution leaves three of four local units at zero output, which is not an operating point any controller would hold.
Search window January 2024 to March 2026, with 2016+ historical records. Following
PRISMA 2020: 2,175 records identified, 1,847 after deduplication, 280 full texts
assessed, 95 primary studies included, plus 12 grey-literature instruments from a
separate stream of 28. See docs/PRISMA_flowchart.md,
which also records the one point at which the coded dataset and the manuscript
bibliography do not correspond.
The structured comparison of Table 9 covers 35 works: 31 primary studies and 4
grey-literature instruments. A regulation is not a primary study and the table does
not treat it as one. The Structured Comparison (Table 9) column of the coding sheet
carries exactly 31 Yes values.
data/references.bib contains 201 entries: the manuscript's 198-entry bibliography
verbatim, followed by three entries used only by this repository, each labelled with
the reason it is here.
Load and summarise the dataset:
python scripts/load_data.pydocker build -t sas-shield .
docker run --rm sas-shield pytest -q
docker run --rm sas-shield python experiments/run_experiments.py --seeds 5The image compiles NuSMV. With NuSMV on the path, Layer 3 uses the model-checked
back-end; tests/test_layers.py::test_native_checker_agrees_with_nusmv cross-validates
the two back-ends over the whole reachable state space and is skipped when NuSMV is
absent. The committed results were produced on the native back-end, and
results/raw/episodes.csv records which back-end each episode used, so the two are
distinguishable rather than conflated.
One caveat on this section: the image has not been built in the environment that produced the committed results, which had no Docker daemon and no NuSMV on the path. The Dockerfile is therefore a description of an intended environment rather than a tested one, and the NuSMV cross-validation test has never run. Anyone who builds it and finds that the two Layer 3 back-ends disagree has found a real defect, and we would want to know.
Dual-licensed. Code (sas_shield/, experiments/, tests/, run_simulation.py,
scripts/) under the MIT Licence. Review dataset and documentation (data/, docs/)
under CC BY 4.0. See LICENSE.
@article{akarma2026verification,
author = {Akarma, Ali},
title = {Verification, Safety, and Governance of Agentic {AI} Systems in
Safety-Critical Infrastructure},
journal = {Autonomous Agents and Multi-Agent Systems},
year = {2026},
note = {Under review}
}