Statistical analysis library for Sugar Sugar study results.
It implements Section 7 (Statistical Analysis Plan) of the study design (Human Prediction of Next-Hour Glucose from Prior CGM Context).
The package is split into five stages so each part can be imported as a library module, exercised in a Jupyter notebook, or run end-to-end via the CLI.
| Stage | Module | What it does |
|---|---|---|
| 1. Data gathering | sugar_data_processing.gathering |
Load prediction_statistics.csv, explode rounds, build person-level table |
| 2. Data verification | sugar_data_processing.verification |
Schema / structural checks + demographic & metric quality flags |
| 3. Statistical tests | sugar_data_processing.statistics |
H1–H5 (§7.3–7.4) on person-level MAE |
| 4. Data comparison | sugar_data_processing.comparison |
Human MAE vs GlucoBench / literature bands (§7.5) |
| 5. Output | sugar_data_processing.output |
PNG figures + one merged markdown / JSON / explorer.html |
| 6. AI scoring | sugar_data_processing.ai |
Rebuild the 3-hour game window, score models, fold into the same report |
Orchestration: sugar_data_processing.pipeline.run_analysis.
| Hypothesis | Question | Test path |
|---|---|---|
| H1 | PwD vs non-PwD MAE (category first, then generic vs own) | Shapiro → t-test / Mann–Whitney |
| H2 | CGM vs non-CGM MAE (category first, then generic vs own) | Shapiro → t-test / Mann–Whitney |
| H3 | Diabetes duration (months) vs MAE, generic and own | Pearson / Spearman + log exploratory |
| H4 | CGM experience (months) vs MAE, generic and own | Pearson / Spearman + log exploratory |
| H5 | Own vs generic MAE (paired) | Shapiro on diffs → paired t / Wilcoxon |
| H6 | Human vs baseline models | Same 3-hour windows; persistence / linear / SugarOne W5.3 and SugarOne R.28 (ONNX, downloaded with HF_TOKEN) / the other ONNX models below. A local SugarOne 1.0 checkpoint is used only when the W5.3 bundle is absent. |
The dashboard uses the model names the paper will use: SugarOne is the W5.3 forecast
ONNX (GlucoseDao/FORECAST-SugarOne-W5.3-10.7h-mard_loss-mv1-s42), SugarOne R.28 is the
dual-mode arm scored with nothing after the origin, CitricSugar is CITRAS-FM W38.2,
Kronsugar is KronLSTM R.80 and Sugarstral is Timestral-28M R.81. On the 9 November 2026
game windows W5.3's person-level MAE was 14.37 mg/dL and R.28's was 21.47 mg/dL, so W5.3 is the
primary SugarOne. Both arms are scored, and a publish is refused if either is missing.
CITRAS-FM W38.7 is not scored: on the game windows its MAE was 15.07 mg/dL against W38.2's
14.97 mg/dL. GluMind is not either: on the joined Loop + AI-READI test set SugarOne 1.0 has
an MAE of 12.40 mg/dL against GluMind's 12.73. The game has no insulin or carbs, so the
forecast SugarOne sees those channels as zero.
Person-level MAE (mean of round MAEs) is the analysis unit so repeated rounds from the same participant do not inflate degrees of freedom.
Primary H1/H2 comparisons use two-sided α = 0.025 (Bonferroni for two primary hypotheses). Other layers are exploratory at nominal α = 0.05. See corrected recruitment planning and the dated data assessment. Sample-size targets are conditional on effect size, variance, group balance, and eligibility; the committed synthetic fixture cannot establish those quantities for the study.
The published production briefing summarizes the 24 October 2026 audit in plain language. Production records run through 9 October: 207 runs, 184 browser IDs, 155 linked identity groups and 109 primary-eligible identities (104 in the stricter quality sensitivity). The revised operational target is 320 eligible for a 3.5 mg/dL effect with planning SD 10, or 250 eligible for a 4 mg/dL effect. See the linked analysis plan for group quotas, recruitment attrition and assumptions.
To regenerate the audit and briefing from the local scrubbed, identity-linked production CSV without model downloads:
PYTHONPATH=src .venv/bin/python scripts/assess_production.pyAn optional participant_key column supplies pseudonymous identity linkage
prepared before contact scrubbing. When present, the loader uses it as the
analysis identity and preserves original_study_id; use
use_participant_keys=False only for the explicit study-ID sensitivity.
Without linkage, study-ID counts can overstate independent participants.
| Input | Path / source | Role |
|---|---|---|
| Real export | data/raw/prediction_statistics.csv (gitignored) |
sugar-sugar app statistics export |
| Synthetic fixture | data/fixtures/synthetic_prediction_statistics.csv |
Committed demo cohort sized for H1–H5 |
| CLI override | --csv /path/to/prediction_statistics.csv |
Explicit file, any location |
Required columns (enforced on load + re-checked in verification):
study_id, run_id, timestamp, format, is_example_data, uses_cgm,
cgm_duration_years, diabetic, diabetes_duration, rounds_played,
overall_mae_mgdl, overall_rmse_mgdl, overall_mape_pct, per_round_metrics
Optional current-app columns (kept when present):
round_context, generic_intervention, challenge_unknown,
challenge_unknown_pct, paper_mention, paper_full_name, diabetic_type,
predicted_values, real_values, prediction_times
cgm_duration_years may be a bare year (legacy) or value,unit (6,months).
Per-round is_example_data / data_source_name (in per_round_metrics or
round_context) decide generic vs own — the run-level source is only the last
dataset of that session. Email / location are dropped; personal upload filenames
are redacted to own_upload.
- Primary (H1–H4): ≥6 generic segments (format A, or odd rounds in mixed C)
- Own-data: ≥6 own segments (format B, or even rounds in mixed C)
- H5: participants meeting both thresholds
src/sugar_data_processing/
gathering/ # stage 1 — load CSV, explode rounds, participant table
verification/ # stage 2 — schema checks + quality flags
statistics/ # stage 3 — H1–H5 tests + effect sizes
comparison/ # stage 4 — GlucoBench / literature bands
output/ # stage 5 — plots + human markdown/JSON/HTML
ai/ # sequence export + model ingest + AI report
fixtures/ # synthetic CSV generator
pipeline.py # run_analysis() wires stages 1→5 (+ AI export)
cli.py # Typer entry point
notebooks/
study_analysis.ipynb # thin walkthrough over the library (same paths as CLI)
data/
raw/ # drop real prediction_statistics.csv here (gitignored)
anonymized_raw/ # verifier copy of that CSV (gitignored; see below)
fixtures/ # committed synthetic demo data
processed/ # parquet/csv intermediates (written by output stage)
output/
figures/ # PNGs
reports/ # analysis_report.md + explorer.html + milestone.html
tests/ # pytest (real synthetic data, no mocks)
docs/
analysis-plan.md # study design §7 → module map
uv syncFor the notebook (dev group includes Jupyter):
uv sync --group devsdp is the short CLI name (uv run sugar-data-processing is the same).
The public site does not serve the research CSV. Pull it over SSH, scrub contact
columns into data/raw/ (gitignored), then run H1–H5.
Once per machine
uv sync
cp .env.template .envEdit .env (never commit it):
SUGAR_REMOTE— the sameuser@host:/pathyou alreadysshto (the CSV, or thedata/inputdirectory that contains it)SUGAR_SSH_IDENTITY— the same key you pass tossh -i, if you use one
Each time you want fresh online numbers
uv run sdp fetch --analyzeThat is the whole online path: scp → blank email / location /
paper_full_name → write data/raw/prediction_statistics.csv → reports.
What that file contains is below.
Pull only (no tests):
uv run sdp fetchRe-run H1–H5 on the last pulled CSV (no SSH):
uv run sdp analyzefetch also looks for prediction_statistics_ai.csv in the same directory as the human file. Keep those two files together and run them as a pair: analyze reads the human path you pass, then loads prediction_statistics_ai.csv from that same folder. In-game rows become in_place and that model is the primary AI comparison. A missing companion is a note; post-factum scoring still runs.
uv run sdp fetch copies one file, prediction_statistics.csv, and writes it to data/raw/prediction_statistics.csv. Consent records, the email-to-study-id map, and the CGM files participants uploaded stay on the server.
On the way in, three columns are blanked:
| Column | After fetch |
|---|---|
email |
empty |
location |
empty |
paper_full_name |
empty |
Everything else in that export is kept, because the tests read it:
| Kept | What it is |
|---|---|
study_id, run_id, timestamp, format, rounds_played |
which session |
age, gender, diabetic, diabetic_type, diabetes_duration, uses_cgm, cgm_duration_years |
inputs to H1–H4 |
predicted_values, real_values, prediction_times, per_round_metrics, round_context, the overall MAE / RMSE / MAPE columns |
the game window and the errors |
data_source_name |
a public corpus name (D1NAMO-…, BIGIDEAS-…) or the participant's own upload filename |
challenge_unknown, generic_intervention, paper_mention |
study flags. The paper's name is the blanked column above |
The upload filename is the piece that can still identify someone: it can be a person's name or a Nightscout host, and it is repeated inside per_round_metrics and round_context. It remains in data/raw/. Loading the CSV for a report replaces those filenames with own_upload, so the HTML does not show them. Glucose values and the demographic columns are the health data the statistics are computed from.
uv run sdp anonymize writes a second copy for evaluators. Same rows and the same study_id and metrics. Corpus filenames stay. Each person gets one stand-in address (email1@gmail.com, email2@gmail.com, …). Upload filenames become initials (AL.csv, and AL2.csv when those initials are already used).
Give evaluators this repo plus the anonymized CSVs. study_id is unchanged, so the HTML is the same analysis. They put both files in one folder:
data/anonymized_raw/prediction_statistics.csv
data/anonymized_raw/prediction_statistics_ai.csv
Then:
uv sync
uv run sdp analyze --csv data/anonymized_raw/prediction_statistics.csvanalyze reads the human file you pass and the AI file sitting next to it. Do not run the two CSVs as separate commands.
Someone who already has the anonymized human file can drop a later prediction_statistics_ai.csv in that same folder and re-run the same analyze --csv line. The join is study_id / run_id. Human-only HTML still builds if the AI file is missing; H1–H5 do not need it.
Pass --csv. A plain uv run sdp analyze reads data/raw/prediction_statistics.csv when that file exists. Reports land in output/reports/; explorer.html is the full page, milestone.html is the human page. See Where the numbers are.
Build the files you send (from the last fetch). The AI companion is copied when it sits next to the raw human CSV:
uv run sdp anonymizeuv run sdp make-fixture
uv run sdp analyze --fixtureuv run sdp fetch --sibling --analyzeOr:
uv run sdp analyze --csv ../sugar-sugar/data/input/prediction_statistics.csvResolution order for analyze without --csv / --fixture:
data/raw/prediction_statistics.csvif present- else fixture (with a yellow warning)
| File | What it is |
|---|---|
output/reports/analysis_report.md |
Merged report: human H1–H5, then same-person human vs AI by task |
output/reports/explorer.html |
Tabs: Overview, Human, AI, People (actual / human / AI lines) |
output/reports/milestone.html |
Milestone page: human participants only (Overview + Human) |
output/reports/analysis_report.json |
Machine-readable human + AI payload |
data/processed/ai/prediction_points.csv |
Per-point sequences for post-factum model scoring |
data/processed/model_cache/forecasts.parquet |
Trained-model forecasts kept between builds (see below) |
output/figures/*.png |
Plots (also copied under output/reports/figures/) |
data/processed/participants.csv |
Person-level analysis table |
Scoring the trained models (CITRAS-FM, KronLSTM, Timestral, SugarOne) on every window takes
minutes, so each build keeps their forecasts in data/processed/model_cache/. The
next build scores only windows that are new or whose visible three hours changed;
new model files or a change to a model's scoring code score every window again.
Set SDP_SCORE_CACHE=off to rescore everything, or point it at another file.
Export sequences only (no full report rewrite):
uv run sdp export-aiAfter a model writes predictions (join keys study_id, run_id, round_number, point_index):
uv run sdp ingest-ai --predictions path/to/model.csvevaluation_mode is post_factum for replays of saved games. in_place is the forecast from prediction_statistics_ai.csv when the app recorded it during the game.
Human H1–H5 use the whole export. Model scoring, H6, and the People tab use the most recent participants, including the latest two. Each player is shown with only the models that scored their rounds. The latest day's players are Kronos. Earlier players keep every model scored on those rounds.
uv run sdp milestone-aiThe page is output/milestone_ai/milestone_ai.html (gitignored). It is the same explorer as the full report: Overview, Human, AI, and People. Put prediction_statistics.csv and prediction_statistics_ai.csv in the same folder and pass that human path as --csv. player_models.csv in the dest folder lists the models for each of those players. Pass --predictions <model_predictions.csv> to rebuild from an already scored file instead of running the models again.
uv sync --group dev
uv run sugar-data-processing make-fixture
uv run jupyter notebook notebooks/study_analysis.ipynbSelect the sugar-data-processing (.venv) kernel. The notebook only calls library
APIs (prepare_session, gathering, verification, statistics, comparison, output).
It writes to the same place as the CLI: output/reports/ (no notebook-specific
report directory).
from pathlib import Path
from sugar_data_processing import run_analysis
result = run_analysis(
Path("data/fixtures/synthetic_prediction_statistics.csv"),
Path("output"),
)
print(result.verification.passed, result.report_path)Or stage-by-stage:
from pathlib import Path
import polars as pl
from sugar_data_processing.gathering import load_prediction_statistics, build_participant_table
from sugar_data_processing.verification import verify_dataset
from sugar_data_processing.statistics import run_all_hypotheses
from sugar_data_processing.comparison import benchmark_context
from sugar_data_processing.output import write_report
csv = Path("data/fixtures/synthetic_prediction_statistics.csv")
runs = load_prediction_statistics(csv)
participants = build_participant_table(runs)
verification = verify_dataset(runs, participants)
suite = run_all_hypotheses(participants)
eligible = participants.filter(pl.col("eligible_primary"))
benchmarks = benchmark_context(eligible if eligible.height > 0 else participants)
write_report(
runs=runs,
participants=participants,
suite=suite,
benchmarks=benchmarks,
verification=verification,
output_dir=Path("output"),
source_csv=csv,
)| Stage | In-memory | On disk (via analyze / write_report) |
|---|---|---|
| Gathering | runs, participants Polars frames |
— |
| Verification | VerificationReport (schema + quality) |
Embedded in report JSON / markdown §6 |
| Statistics | HypothesisSuite (H1–H5) |
Report §3–4 + JSON hypotheses |
| Comparison | BenchmarkContext |
Report §5 + JSON benchmarks |
| Output | report path | output/reports/analysis_report.*, explorer.html, output/figures/*, data/processed/* + data/processed/ai/ |
- Schema: required columns, non-empty IDs, format ∈ {A,B,C}, parseable
per_round_metrics, non-negative MAE,rounds_playedvs metrics length - Quality flags: implausible age/durations, flag mismatches, MAE IQR
outliers, short sessions, duplicate
run_id, below-threshold participants
High-severity schema issues set verification.schema_ok = False. Quality
flags are reported but do not alone fail verification (they inform the report).
uv run pytest- Models only see the 3-hour game window (24 visible points, left-padded to
128).
explorer.htmlPeople tab overlays actual CGM, the human line, and each AI forecast. - See
docs/analysis-plan.mdfor the §7 → module mapping and format conventions.