Cianfhoghlaim — long-distance, enduring learning. A research-and-deployment platform for the British Isles education corpus (8 nations × 5 stages × bilingual Goidelic + Brythonnic), agentic AI, self-hosted infrastructure, and minority-language machine learning. Maintained by Cian Mac an Déisigh Uí Liatháin (Deacy-Lyons), a Mathematics & Education teacher, Dioplóma C1 in Irish, and agentic-AI engineer based in Galway and East Belfast.

Useful companion resources: To understand the container, data, indexing, storage, and infrastructure patterns used throughout this project, see the hands-on labs at iximiuz Labs, the database references at DBQuacks, and the official documentation, blogs, and examples for CocoIndex, dlt, LanceDB, DuckDB, MotherDuck, Komodo, and Pangolin. These are useful external guides to the key technologies and design choices represented in the repository.

The 3 priority mise run tasks shipped by the 2026-07-30 → 2026-08-01 trilogy at a glance. Read this first; the full priority quick-refs are in AGENTS.md.

Plus the safety gate for any iac:bootstrap, iac:plan, or km deploy stack <arm-oci-*> from opencode: preflight:arm-oci (mandatory per the 2026-07-09 repo-boundary-lockdown openspec change).

The project is sprawling on purpose. It is a research-and-deployment platform for the British Isles education corpus, an agentic AI fleet, a self-hosted infrastructure mesh, and a minority-language ML playground — all federated by a single bun + uv + turbo monorepo. Because the goal is a deliberately overcomplicated Master's completed in a year, many subsystems that do work are partially obscured by the design choices meant for my team's downstream use. Read this section before forming an opinion about the repo.

The value for anyone looking at this project today is in the specific combinations of already well-calculated open-source packages. The data engineering (DLT + Dagster + BAML + CocoIndex + DuckLake + LanceDB + MotherDuck + Marimo), the DevOps (Komodo + Pangolin + Infisical + Locket + Pocket ID + Traefik + Garage S3), the web-development (TanStack Start + Convex + Hono + CopilotKit + oRPC + AG-UI + Cloudflare), and the agent layer (Agno + Google ADK + Pydantic AI + LiteLLM + Letta + Cognee + Graphiti + Langfuse + MLflow) are each real, working open-source compositions. You can copy any slice independently.

The 12-MCP agent surface is the new foundation layer (added 2026-08-21). All agent runtime tools are now wired via opencode.json

• .mcp.json with one canonical surface per domain — ccc for code search, firecrawl + crawl4ai + chrome for web data, dlt-workspace + motherduck for data engineering, cognee + graphiti + design-system for knowledge/memory, langfuse for observability, infisical for secrets, huggingface for model hub. See openspec/changes/2026-08-21-mcp-server-revival-overview.md for the canonical inventory.

You can take the same data pipelines and rewire them for any other jurisdiction. The dlt_sources/ + baml_src/ + cocoindex_flows/ + motherduck/ + notebooks/ stack already produces the official legal, medical, education, and government documents for the British Isles. Retargeting it for, say, the US CMS, the WHO IRIS repository, the Bundesministerium für Bildung, or the French Ministère de l'Éducation means changing the jurisdiction enum, the BAML schemas, the scrape cache roots, and the destination namespaces — the core pipeline shape stays the same. Use USE_LOCAL_SCRAPES=true while you rebuild; the curated cache at stedding/ingest_queue/ lets you iterate without spending scrape credits.

The cheapest way to do this is a cheap coding agent. A €20/month Gemini Deep Research Pro subscription is the right tool for the first hour (finding the authoritative ministry, exam-board, or medical-register endpoints). A MiniMax coding plan (or the local OpenCode Go CLI) is the right tool for the next two days (the iterative file-local refactor across dlt_sources/ + baml_src/ + cocoindex_flows/ + motherduck/). GitHub Copilot is a fine runner-up for single-file edits. None of them need the full monorepo.

Use my notes as a blueprint for your own deep-research tangent. The leabharlann repository (3.4 GB, 2,400+ docs × 7 subdirs) holds the source materials that informed every sub-agent, BAML schema, and CocoIndex flow in this repo. They are unique to my circumstances; treat them as a worked example of how to do a deep-research tangent in this domain, not as what to copy.

The full per-area deep-cuts report — what each package composes, which 3-5 files to copy, which 5 lines to change, which 1 assumption is hardest to escape, and which coding agent to point at which job — lives at docs/CHOP_AND_CHANGE_GUIDE.md.

⚠️ Important Disclaimer

The data engineering pipelines in this repository — DLT ingestion, BAML extraction, CocoIndex v1 embeddings, Cognee cognify passes, the 8 NCCA Leaving Certificate subject asset groups — process official Government syllabus exam papers + marking schemes of the Republic of Ireland (NCCA Leaving Certificate, Junior Cycle, Primary) and the wider British Isles education system. They are grounded in the 8 British Isles nations (Ireland, England, Scotland, Wales, Northern Ireland, Isle of Man, Jersey, Guernsey) and bilingualised in EN + GA (Gaeilge).

The web UIs (cianfhoghlaim-web, tuatha-ui, croilar-web, croilar-portal, oideachais, oideachais-dashboard, the tuatha-demo, game_showcase, the Hono API gateway) are packaged demos that prove the agentic-web wiring works. They are not yet the final pedagogical surface — the UI work is deferred until the data pipeline has produced the canonical syllabus-accurate assets.

To self-host your own instance you need your own domain name and a free Cloudflare account — see bonneagar/README.md.

cianfhoghlaim is a polyglot monorepo (bun + uv + turbo) that:

1. Ingests the curriculums, exam papers, marking schemes, and syllabi of the 8 British Isles nations (with bilingual EN + GA extraction for the Irish strand).
2. Extracts structured data via the BAML schema library at baml_src/british_isles/ (NCCA, SEC, CCEA, SQA, WJEC, Edexcel + the European Union EUR-Lex / ECDC / EMA / Eurostat / Eurydice + multi-nation Commonwealth + Americas expansions).
3. Embeds in vector + graph form via 60 CocoIndex v1 Apps (7 model families) + Cognee cognify layers + the Graphiti temporal knowledge graph.
4. Surfaces through marimo reactive notebooks, MotherDuck Dives, TanStack Start web apps, and a 13-agent meaisínfhoghlaim fleet (LiteLLM-routed via the OpenCode Go API; 7-tier fallback alias minimax).
5. Hosts itself on a self-managed 93-stack Docker Compose fleet (bunchloch MacBook M4 Max for the data plane + arm1-oci Oracle Cloud free-tier for the control plane + Garage S3 storage + the Komodo / Pangolin / Infisical / Locket / Pocket ID / TinyAuth / Traefik mesh).
6. NEW 2026-08-23 — Ingests any university student's personal archive (the maintainer's three UoG programmes are the worked example: BA Maths & Education, HDip Software Design, Diploma sa Ghaeilge C1) at F-granularity (per-question answers, per-assignment topics, handwritten-maths HTR) and joins it to the official ModuleDescriptor corpus via 10 typed Cognee edges. See Tertiary-Level Personal Archive Pipeline below.

The author is a registered member of the Teaching Council, with verified memberships in Fine Gael and the Alliance Party of Northern Ireland, and the Deacy Tribe of the Morris-Conroy tribes of Galway. See the Personal credential corpus section below for the full verified-PDF table, the Verified academic archive section for the leabharlann/ subdir summary, and the Family history section for the Triple-Crown synthesis.

The platform has one canonical source of truth for every model, schema, pipeline, and stack (post-2026-08-15). It replaces the ~70 hardcoded model strings + 96 hand-written Pydantic duplicates + 54 nearly-identical CocoIndex Apps that the audit found.

The 4 canonical artifacts:

• meaisinfhoghlaim/models/model_registry.py — the 60-entry MODEL_REGISTRY across 7 families (ocr_vision × 20 / text_llm × 21 / embedder × 3 / rerank × 3 / image_gen × 5 / voice × 5 / translation × 3).
• notebooks/_shared/schema.py — the 5 introspection helpers (schema_introspect, schema_introspect_table, list_dlt_sources, list_cocoindex_apps, list_baml_classes).
• notebooks/00_control_panel.py — the 5-tab marimo control panel (Models / Pipelines / Datasets / Stacks / Registry).
• deployment-choice.yaml — the canonical enablement file (read/written by the notebook + web UI + CLI).
• opencode.json — the 12-MCP agent surface (added 2026-08-21). One canonical entry per domain: ccc / firecrawl / crawl4ai / chrome / dlt-workspace / motherduck / cognee / graphiti / design-system / langfuse / infisical / huggingface.

The 4 supporting artifacts:

• scripts/registry_audit.py — drift detector (fails CI on hardcoded model strings).
• agents/adk/litellm_agent.py — make_litellm_agent() + litellm_model("minimax") wrappers.
• orchestration/defs/2_materials/_base/jurisdiction_assets_base.py — the JurisdictionAssetsBase for the per-jurisdiction Dagster asset wrappers.
• 3 CocoIndex factories (cocoindex_flows/european_nations/_factory.py et al.).

The canonical model_for() pattern:

from meaisinfhoghlaim.models import model_for

default = model_for("text_llm", "default")              # → "minimax-m3"
irish   = model_for("text_llm", "irish")                # → "uccix-mistral-24b"
embed   = model_for("embedder", "default")              # → "BAAI/bge-m3"

The canonical schema_introspect() pattern:

from notebooks._shared.schema import schema_introspect, list_dlt_sources
conn = ibis.duckdb.connect("md:cianfhoghlaim")
rows = schema_introspect(conn)             # every BIEP DuckDB table's columns
print(f"{len(list_dlt_sources())} DLT sources, {len(rows)} columns")

Lint gate: mise run lint:registry — fails on any hardcoded model string in agents/, baml_src/, notebooks/, web/, orchestration/, meaisinfhoghlaim/.

Full guide: .agents/skills/centralized-registry/SKILL.md.

Two language graphs live side by side, orchestrated by turbo.json and a single mise.toml toolchain. Post-v7 (2026-07-17), the Python package IS the repo root — no more cianfhoghlaim/ nesting.

The GitOps infrastructure lives in bonneagar/ and is reached via bun run --cwd bonneagar iac:<command> from the root package.json. The IaC is no longer a separate GitHub repo; the archive-bonneagar remote is a frozen read-only relic. See bonneagar/README.md for the full architecture, the 93-stack inventory, the 7-day operator quick start, and the known gaps.

The pipeline walks a corpus from raw disk to an agent-consumable, semantically-indexed artifact. Five sequential stages live under orchestration/defs/<stage>/, with two 4_* siblings alongside 4_asset_generation/ for budget tracking and the docs-index memory job:

Each stage has a matching Component class in orchestration/components/ that dg-style YAML defs.yaml files instantiate (11 components in total; see orchestration/README.md for the architecture, the JurisdictionAssetsBase pattern, and the R1–R4 conformance check at scaffold time).

The 6 Irish LC priority subjects — Mathematics, Chemistry, Geography, Gaeilge, English, Computer Science — get the deep treatment: NCCA syllabus + SEC exam papers + marking schemes + 7 v1 CocoIndex flows + 42 Dagster assets + 6 marimo notebooks + 4 MotherDuck Dives + a daily MotherDuck Flight.

The BAML extraction schemas (baml_src/british_isles/ireland/education/lc_extraction/*.baml) produce structured LeavingCertSyllabus, LeavingCertPastPaper, LeavingCertMarkingScheme objects — rubric-anchored to NCCA PDF page references. Web search by the agent fleet always lands on an in-pipeline asset, never on a stale URL.

For the wider corpus (Junior Cycle, Primary, the 5 English boards, SQA, WJEC, CCEA, plus the EU EUR-Lex / ECDC / EMA / Eurostat / Eurydice + multi-nation Commonwealth + Americas expansions), the same JurisdictionAssetsBase pattern scales to ~30-line per-jurisdiction subclasses instead of ~380-line hand-written asset files.

A student's personal university archive is treated as a first-class citizen at the same level as the Leaving Cycle subject pipeline. The pipeline lifts any user's leabharlann/<university>/ corpus (auto-discovered from folder structure, no curated drop-PDF UI as primary entry) to feature parity with the Leaving Cycle subject pipeline — typed artefacts → assignments → questions → topics → code cells → reading items → CA marks → transcript rows at F-granularity (per-question), joined to the transcript for ground truth, embedded in LanceDB via 4 CocoIndex v1 Apps, with 10 typed Cognee cross-archive edges (including a cross-module Topic-RELATED-TO-Topic graph), and surfaced via the canonical 8-tab Marimo notebook + Convex chat action + CopilotKit component + Genie UI tile + Google ADK agent.

Reference: openspec/changes/2026-08-23-uog-personal-archive-tertiary-modules-v1/

The University of Galway (Ollscoil na Gaillimhe) corpus — covering three programmes the maintainer completed — is the canonical worked example:

Three example usage purposes the pipeline surfaces end-to-end (the per-question answers are chatable via the ADK agent personal_archive_module_assistant):

The CS4423 (Networks) folder is the canonical worked example for F-granularity extraction. From the 5 assignment PDFs plus the lecture notes plus the past exam, the pipeline produces:

• Module dossier — every assignment + every question + my answer text + my mark + the HTR backend used + the LaTeX form of any maths + the topics covered.
• Topic graph — eigenvector_centrality, neighbours_in_networks, graph_laplacian, etc. The Cognee Topic-RELATED_TO-Topic edge then connects these to the M.Sc. AI future-modules story (because eigenvector centrality shows up in graph neural networks, which the maintainer will study next year on the MA in AI).
• Transcript join — exact match on (module_code, academic_year) against the BA Maths & Education transcript; CS4423, 2020–21, A1.

MP491 (Non-Linear Systems) is the canonical worked example for the HTR pipeline. The handwritten answers were authored on an iPad with an Apple Pencil in the Goodnotes app, then exported as vector PDFs. The pipeline routes them through the 6-backend HTR ensemble (nougat + olmocr-2-7b + CogVLM + gemma-3 majority-vote consensus, with nougat as the single-VLM best-of-breed fallback for scientific papers):

• my_answer_text — the verbatim OCR'd answer.
• my_answer_latex — the LaTeX form (e.g. \int_0^1 x^2 \, dx).
• htr_backend_used — which of the 6 backends produced the answer.
• htr_confidence — 0.0–1.0, exposed in the Marimo notebook for manual override.

Numerical Analysis 2 covers splines and interpolation — topics directly relevant to the M.Sc. AI modules the maintainer is starting in 2027-09. The pipeline extracts the topics, joins them to the official UoG ModuleDescriptor (from the existing 2026-07-15-cianfhoghlaim-university-deep-extraction-v1 change), and emits the cross-module Cognee edge that lights up in the future M.Sc. AI Marimo notebook.

Any university student can point the pipeline at their own leabharlann/<university>/ corpus by setting the 9 UNIVERSITY_* env vars (see .env.example) and calling personal_archive_source(UniversityPersonalArchiveConfig(...)). The same 8 DLT resources, 7 BAML functions, 4 CocoIndex Apps, 10 Cognee edges, 6 Dagster assets, 8 Marimo tabs, 5 Convex queries, CopilotKit

• Genie + ADK agent, and 12 tests run unchanged. The canonical UniversityPersonalArchiveConfig Pydantic v2 model is the single surface for "who is the student, what is the institution, what regex matches the module codes, where do the transcripts live".

# Validate the openspec change
openspec validate 2026-08-23-uog-personal-archive-tertiary-modules-v1 --strict

# Run the personal-archive test suite (12 passing)
uv run pytest tests/personal_archive/ -v

# Materialise the DuckLake tables
uv run python -c "
import duckdb
from dlt_sources.lakehouse import register_personal_archive_tables
con = duckdb.connect(':memory:')
register_personal_archive_tables(con)
print(sorted(t[0] for t in con.execute('SHOW TABLES').fetchall()))
"

# Open the 8-tab Marimo notebook (Health / Filters / Materials /
# URL Health / Heatmap / Recent / Lance Search / SQL Console)
marimo edit notebooks/15_personal_archive.py

# Auto-classify a sample artefact
uv run python -c "
from pathlib import Path
from dlt_sources.filesystem.uog_personal_archive import _classify_file
p = Path('leabharlann/ollscoil_na_gaillimhe/mata/networks/CS4423 - Networks/cian_mac_liathain_assignment_3.pdf')
print(_classify_file(p))
"

The 13 root agents (1 Custom + 8 ADK + 3 Agno + 1 image-generation) plus the 8 NCCA Leaving Cert subject specialists (gael_agent, math_agent, appm_agent, chem_agent, comp_agent, engl_agent, geog_agent, hist_agent) all route through the LiteLLM gateway at litellm.cianfhoghlaim.ie:4000 and the canonical 7-tier minimax fallback alias. The fleet spans 5 frameworks (Custom + ADK + Agno + Pipecat + CopilotKit), wires through a single canonical surface (AGENT_REGISTRY), and is observed by a 5-layer observability stack (Langfuse + Logfire + MLflow + RAGAS + structlog) backed by a 5-backend memory layer (Cognee + Graphiti

• LanceDB + FalkorDB + Memgraph). See agents/README.md for the architecture diagram
• implementation decisions + how to lift a single agent out independently.

These are the records that ground the project's claims. The full long-form index lives at cian_mac_an_déisigh_uí_liatháin/README.md; this table is the canonical subset a new visitor needs to verify the author + the lineage:

The course material + examination scripts + the personal academic corpus live at github.com/cianfhoghlaim/leabharlann — a separate 3.4 GB repo, the only remaining separately-managed repo. It contains 7 top-level subdirectories that feed the data platform:

The Tuatha educational MMO's Ard-Rí na hÉireann framing is grounded in 7 long-form research PDFs under leabharlann/gemini_deep_research/culture/:

1. claiming_rí_na_gaillimhe_a_synthesis.pdf — Rí na Gaillimhe: An Ethnohistorical and Jurisprudential Warrant (15 pp.)
2. heraldic_research_for_dual_blood_lineage.pdf — The Heraldry of the Corrib Crown (14 pp.)
3. british_isles_cianfhoghlaim.pdf — Strategic Blueprint for Inter-Celtic Linguistic Acquisition, AI Integration, and Transnational Educator Credentialing (16+ pp.)
4. claiming_irish_kingship_through_lineage.pdf — The Crown of the Corrib: An Ethnohistorical and Genealogical Warrant (13 pp.)
5. researching_neil_deacy's_galway_heritage.pdf — The Socio-Economic, Athletic, and Genealogical Topography of the Deacy Family in Galway (12 pp.)
6. royal_titles_celtic_heritage_and_claims.pdf — The Crown of the Corrib and the Imperium of the Irish Sea (13 pp.)
7. deacy_family_heritage_research.pdf — The Deacy and Conroy Dynasties: An Ethnohistorical Analysis of Galway's Commercial and Maritime Lineage (9 pp.)

The author's lineage is the triple-crown union of four kindreds of Connacht and Munster:

1. Deacy (maternal surname; Irish Uí Dhéisigh) — the sept of the Déisi Muman resettled in south Connacht (Co. Galway) during the 12th century; the family gave their name to the late Éamonn Deacy and the Eamonn Deacy Park in Galway.
2. Lyons (paternal grandfather's lineage; Irish Mac Liatháin) — the Uí Liatháin sept of Munster, who (per the Historia Brittonum) colonized Wales and Cornwall alongside the proto-Déisi.
3. Morris (maternal great-grandmother Christina Morris) — of the City of Tribes merchant families of Galway.
4. Conroy (maternal great-great-grandmother Polly Conroy; Irish Mac Conraoi / Ó Conaire) — the Sea-Kings of Connacht who held the tuath of Delbhna Tír Dhá Locha (the barony of Moycullen in Connemara). Polly Conroy was a cousin of Pádraic Ó Conaire, the canonical modern Irish-language writer from Galway.

The 3-stream synthesis is the 4-line modern incarnation: Cian Mac an Déisigh Uí Liatháin (Deacy-Lyons). The Deacy side carries the galwegian-historical pedigree (Cooke's Corner, Aston Villa, Galway United, Eamonn Deacy Park). The Lyons side carries the pan-Munster-Brythonic-imperial pedigree (Uí Liatháin of Castlelyons and the Welsh / Cornish colonies). The hyphenation preserves both branches of the Triple Crown and acknowledges the previous achievement of the Lyons lineage while respecting the author's choice to bear his mother's Mac an Déisigh name first.

The full discursive narrative — the Triple-Crown synthesis, the Brehon-Law saoí framing, the sacred topography of Shantalla (Sean Talamh), the mythological warrant of Cian mac Cáinte (the swine-god and father of Lugh Lámhfhada), the philological restoration of the Aos Sídhe vow, and the dual-monarchy synthesis with King Charles III — lives at cian_mac_an_déisigh_uí_liatháin/FAMILY_HISTORY.md (619 lines, preserved verbatim per the author's prior instruction that "the validated references to my credentials and family history relevant to the name and location of the project" stay in the repo).

bonneagar/ is in-tree and authoritative here; the standalone repo at github.com/cianfhoghlaim/bonneagar is a one-way published mirror of it, in root layout (stacks/, komodo/, iac/ at the repo root rather than under bonneagar/).

The mirror was frozen between 2026-07-12 and 2026-08-28, during which it fell 643 files behind. It was resynced on 2026-08-28 by exporting bonneagar/ with git filter-repo --subdirectory-filter, which force-updated the mirror's main. The pre-sync tip is preserved on the mirror as backup/pre-monorepo-sync-2026-08-28.

Never commit to the mirror directly — it is force-updated from this monorepo and any direct commit will be discarded on the next sync. Refresh it with mise run bonneagar:mirror.

openspec/ is the single source of truth for capability specs. The workflow: list → write proposal/tasks/spec deltas → validate --strict → implement → archive. 96 specs live under openspec/specs/; 34 changes are currently pending under openspec/changes/. See openspec/AGENTS.md for the full workflow and openspec/project.md for conventions.

openspec list --specs                    # 96 capability specs
openspec list                            # 34 pending changes
openspec validate <change-id> --strict   # MUST pass before commit
openspec archive <change-id> --yes       # after deploy

The 3-way contract: the dev-baile Infisical vault is the source of truth → the committed .infisical.env template holds every value as an infisical://dev-baile/... reference → the gitignored .env is hydrated by mise directory hooks + the Locket sidecar at runtime. The IaC binds each stack's secrets.env to Infisical via the typed InfisicalClient at bonneagar/iac/clients/. Full guide: bonneagar/SECRETS-MANAGEMENT.md.

CCC (bun run ccc:search "<query>") gives every agent a per-project semantic index at .cocoindex_code/target_sqlite.db (CocoIndex BGE-M3 embeddings over ~7,000 tracked files). Cognee (cognee cognify) gives a knowledge-graph memory layer at agents/meaisinfhoghlaim/memory/ across 7 clusters. Firecrawl (firecrawl_search / firecrawl_scrape / firecrawl_research_*) covers upstream + biomedical + arXiv literature. Per the 2026-08-14 dual-search convention, every Firecrawl call MUST be paired with a ccc:search so both tool names appear in the Langfuse trace.

Agentically. The canonical configuration is at opencode.json. The 13 + 8 agent fleet is wired through agents/agent_registry.py:AGENT_REGISTRY with a single canonical surface (the same conventions as agents/tuatha/wiring.py:SubjectAgentWiring). See AGENTS.md for the durable rules every coding agent follows in this repo.

For: Anyone extending cianfhoghlaim — adding a new LC subject, a new jurisdiction, a new CocoIndex flow, a new MotherDuck Dive, or a new Dagster asset.

Canonical reference: dlt_sources/DATA_PLATFORM_ROUTER.md (the canonical single-router for the 5 sub-area AGENTS.md files that document the data platform surface).

Per-area deep-dives: dlt_sources/AGENTS.md (1,957 files / 920 @dlt.source) + baml_src/AGENTS.md (319 files / 3 canonical clients) + cocoindex_flows/AGENTS.md (94 explicit Apps + 378 factory Apps) + orchestration/AGENTS.md (~833 assets / 11 Component classes) + meaisinfhoghlaim/README.md (22 VISION_MODELS + 6 CLASSICAL_OCR + 7 PDF converters).

Cianfhoghlaim is a 4-stage data pipeline walking the British-Isles education corpus from raw disk to agent-consumable, semantically-indexed artefacts:

Stage 1 — INGESTION        DLT sources → DuckLake raw
Stage 2 — MATERIALS         BAML/Docling extraction → typed structures
Stage 3 — MODEL LIFECYCLE    CocoIndex v1 embedding + Cognee cognify
Stage 4 — ASSET GENERATION   marimo dashboards + TanStack pages + oRPC routes
Stage 4b — BUDGET            Firecrawl credit tracking (the meter for the BIEP freshness loop)
Stage 4c — MEMORY            docs-index Cognee cognify job (the 6th sync layer)
Stage 5 — AGENT OPS         the 13-agent fleet + the 8 NCCA subject specialists

The lakehouse is the data platform backbone: MotherDuck (cloud DuckDB via Postgres endpoint) + LanceDB (vector store) + DuckLake (ACID on object storage) + Garage S3 (object storage) + Lakekeeper (Iceberg REST catalog) + Lance Namespace.

A new contributor who wants to extend cianfhoghlaim typically wants to do one of 5 things. For each, the canonical 3-5 files are:

# Read these 3 files first
$EDITOR docs/per-subject-pipeline.md          # the canonical 9-file template per subject
$EDITOR baml_src/AGENTS.md                     # the 5 canonical lc6 BAML functions
$EDITOR orchestration/components/biep_subject_component.py  # the canonical Dagster Component

The 9-file template per subject: qpack_<subject>.baml + dlt/subjects/<subject>/{__init__,sources,schema}.py + dagster/assets/<subject>_assets.py + cocoindex/<subject>_embedding.py + agents/meaisinfhoghlaim/educational/<subject>_agent.py + notebooks/leaving_cert/<subject>.py. Per docs/per-subject-pipeline.md.

$EDITOR dlt_sources/british_isles/_cross/jurisdiction_pipeline_base.py  # the canonical base class
$EDITOR dlt_sources/README.md           # §"The BIEP v3 jurisdiction pipeline pattern"
$EDITOR orchestration/components/biep_subject_component.py  # the per-jurisdiction Component

Subclass JurisdictionPipelineBase (at dlt_sources/british_isles/_cross/jurisdiction_pipeline_base.py:33) and add a ~30-line subclass instead of a ~380-line hand-written asset file. The pattern is documented in dlt_sources/README.md.

$EDITOR cocoindex_flows/AGENTS.md       # the 4-rule R1-R4 conformance contract
$EDITOR cocoindex_flows/infrastructure/cocoindex_v1_conformance.py  # the R1-R4 linter
$EDITOR cocoindex_flows/_shared/_lifespan.py  # the canonical lifespan (LANCE_DB + EMBEDDER ContextKeys)

Every new v1 App MUST satisfy the R1-R4 contract (per cocoindex_flows/infrastructure/cocoindex_v1_conformance.py:13-31):

• R1 — The App imports from .._shared._lifespan (or ..._shared._lifespan for 3-deep dirs, etc.)
• R2 — Uses the canonical ContextKeys (LANCE_DB, EMBEDDER, RESOLVED_FILE_REGISTRY)
• R3 — Has a module-scope coco.App(...) instantiation
• R4 — Has at least one @coco.fn(memo=True) decorator

The audit is mise run cocoindex:conformance. Conformance violations raise ConformanceViolation at scaffold time (before Dagster tries to run the App).

$EDITOR motherduck/README.md   # the canonical entry-point + the DiveRegistry class
$EDITOR motherduck/dives/<new_dive>.py  # the new Dive definition

A Dive is a SQL/Python artefact that gets registered in the MotherDuck workspace. Use the existing 4 BIEP v1 Dives (lc_syllabus_topics / lc_exam_difficulty / lc_marking_complexity / gov_circulars_archive) as templates.

$EDITOR orchestration/AGENTS.md  # the 5-layer architecture + the 2 parallel trees (defs/ + pipelines/)
$EDITOR orchestration/components/__init__.py  # the registry surface (every new Component MUST be re-exported here)
$EDITOR orchestration/components/layer<N>_<stage>.py  # the appropriate stage Component

orchestration/components/__init__.py is the registry surface — every class instantiated from defs.yaml via type: MUST be re-exported there, or Dagster raises DagsterUnresolvableSymbolError at load time.

The lakehouse is the data platform backbone, deployed as the lakehouse Docker Compose stack (per bonneagar/stacks/lakehouse/):

The canonical database namespace is md:cianfhoghlaim (per dlt_sources/_shared/motherduck_options.py). The 53 MotherDuck Dives are registered under md:cianfhoghlaim.<subject>.<stage>. The 4 BIEP v1 canonical Dives are:

• lc_syllabus_topics — the 6 LC subjects × NCCA syllabus topics
• lc_exam_difficulty — the exam paper difficulty metrics
• lc_marking_complexity — the marking scheme complexity analysis
• gov_circulars_archive — the gov.ie circulars archive

The canonical reference is dlt_sources/british_isles/ireland/education/ireland_jurisdiction_pipeline.py. To add a new jurisdiction (per dlt_sources/README.md):

from dlt_sources.british_isles._cross.jurisdiction_pipeline_base import JurisdictionPipelineBase
import dlt

class MyJurisdictionPipeline(JurisdictionPipelineBase):
    STAGE = "leaving_certificate"

    def build_pipeline_resource(self, pipeline):
        @dlt.resource(name="documents", write_disposition="merge", primary_key=["content_hash"])
        def documents():
            for subject in self.subjects():
                yield {"subject": subject, "content_hash": ..., "url": ..., "text": ...}
        return documents

my_pipeline = MyJurisdictionPipeline("my_jurisdiction")
my_pipeline.run()  # writes to md:cianfhoghlaim.education.my_jurisdiction.*

This is the canonical pattern that replaces the ~380-line hand-written asset files with a ~30-line subclass.

Per docs/baml/biiep-v3-client-canon.md (the 2026-08-07-biep-v3-hardening-v1 change). The fragmented client setup (ExtractEn / ExtractEnStrong / LlamaSwapClient / LocalVision) is consolidated into 3 canonical clients:

For each client ExtractEn / client ExtractEnStrong reference in BAML function signatures:

• Single-subject quick extraction → client BIEPV3Extract
• Multi-step syllabus extraction → client BIEPV3ExtractStrong
• OCR/VLM pipeline integration → client BIEPV3Vision

For: Operators bringing up the platform from cold on arm1-oci (production) + bunchloch (MacBook M4 local-dev), OR forking the repo to retarget for a different jurisdiction.

Canonical reference: docs/DEPLOYMENT-PLAYBOOK.md (the canonical 13-step recreation + 6-step confirmation + 4-blocker "if it doesn't work" recipe — also rendered inline below).

Per-area IaC deep-dives: bonneagar/README.md (the 6-step IaC golden path) + bonneagar/DEPLOYMENT-STRATEGY.md (the 2-host topology + the 8-phase bootstrap state machine) + docs/CHOP_AND_CHANGE_GUIDE.md (the chop-and-change recipe for your own domain).

Per bonneagar/DEPLOYMENT-STRATEGY.md §1:

Host names are not Pangolin site names. The machine called bunchloch is registered in Pangolin as the site named macbook. There is also a site named bunchloch (a dead registration that has been offline since 2026-07-27). Automation addresses sites by niceId, not by either name.

Per bonneagar/README.md §1-§6:

Step 1 — Verify mise + secrets            mise run validate-env
Step 2 — 6-way health check               mise run iac:health
Step 3 — Plan (read-only diff)            mise run iac:plan
Step 4 — Validate 88 stacks               mise run cic:stack-doctor
Step 5 — Bootstrap a new cluster          mise run iac:bootstrap (8-phase state machine)
Step 6 — arm1-OCI safety preflight        mise run preflight-arm-oci

Every one of the 88 Docker Compose stacks follows the same 6-file pattern (per bonneagar/GOLD_STANDARD.md):

The stack-doctor audit (mise run cic:stack-doctor) walks bonneagar/stacks/*/compose.yaml and reports:

• CRITICAL: stacks missing compose.yaml or failing docker compose config --quiet
• WARNING: stacks missing one of the 6 GOLD_STANDARD files
• INFO: stacks that pass all checks

Target: 0 CRITICALS.

Per docs/DEPLOYMENT.md §3. When bringing up the platform from cold, start the 13 stacks in this exact order — skipping a tier breaks downstream tier dependencies:

1.  secrets        →  infisical                  (everything depends on this)
2.  storage        →  motherduck, lakehouse      (LLM gateways need to log traces)
3.  LLM            →  litellm, unsloth-serve     (Unsloth first; LiteLLM routes to Unsloth)
4.  observability  →  langfuse                   (LLM gateways need to flush traces)
5.  browser        →  crawl4ai, stagehand        (Firefox + Chrome bring-up is slow)
6.  monitoring     →  changedetection            (sensors must come after the sources they watch)
7.  governance     →  komodo, pangolin, locket   (orchestration + routing + secret-injection)
8.  UI             →  openchamber                (web UI last; it depends on everything above)

Per bonneagar/DEPLOYMENT-STRATEGY.md §4. A deploy at the moment will hit these 4 known issues:

The canonical 13-step recreation recipe. For the full recipe, see [docs/DEPLOYMENT-PLAYBOOK.md](./docs/DEPLOYMENT-PLAYBOOK.md §1).

Quick path:

# Step 1 — prerequisites
git clone https://github.com/cianfhoghlaim/cianfhoghlaim
cd cianfhoghlaim

# Step 2 — install
mise install        # bun + uv + dagger + pulumi + duckdb + opencode
bun install          # TypeScript workspace deps
uv sync              # Python sub-packages

# Step 3 — secrets
bun run scripts/create-env.ts    # creates the dev-baile environment in Infisical
bun run scripts/init-vault.ts    # reads .env + .infisical.env, creates / updates vault secrets

# Step 4 — verify
mise run validate-env            # verifies all required env vars are hydrated
mise run iac:health              # 6-way health check
mise run iac:plan                # read-only diff between IaC-declared and actual state
mise run cic:stack-doctor        # validate 88 stacks against 6-file GOLD_STANDARD

# Step 5 — bootstrap (8-phase state machine)
mise run iac:bootstrap           # the 8-phase state machine: Pulumi → Infisical → Pangolin → Komodo → Newt → syncs → one-shot → first sync

# Step 6 — materialise the lakehouse
mise run dagster:dev             # launches the Dagster UI on :3000
uv run python -m orchestration.cli list-assets      # verify ~833 assets loaded
uv run python -m orchestration.cli materialise-leabharlann  # the canonical first materialise

# Step 7 — run the 4 BIEP v1 Dives
python -c "from motherduck import BIEP_DIVES; print([d.name for d in BIEP_DIVES])"
python -c "from motherduck import save_all; save_all()"

# Step 8 — the canonical CI gate
mise run core:ci                 # lint + test + openspec:validate-all + devops:validate-stacks

The canonical 6-step confirmation recipe. For the full recipe, see [docs/DEPLOYMENT-PLAYBOOK.md](./docs/DEPLOYMENT-PLAYBOOK.md §2).

Quick path:

# The 6 weekly health-checks
mise run iac:health                 # 6-way health check (Komodo + Pangolin + Infisical + Newt + Pocket ID + Tinyauth)
mise run devops:validate-stacks     # 88 stacks against GOLD_STANDARD
mise run data:dagster:up            # Dagster UI on :3335 + ~833 assets
mise run ml:registry:audit          # 24 VISION_MODELS live on HF Hub
mise run openspec:validate-all      # CI gate (131 items pass)
mise run sync:all                   # 14 sync layers

Business Source License 1.1 — see LICENSE.md. Granted for non-commercial, non-profit, cultural preservation, and academic research use within the legal jurisdictions of Ireland, Northern Ireland, the United Kingdom, the European Union, the British Isles, the Commonwealth, the Crown, the United States of America, Mexico, Brazil, Taiwan, Tibet, Nepal, South Korea, Japan, and China.

Excludes: sanctioned organisations, paramilitary groups, entities in violation of international human rights conventions. Change Date: 4 years from publication. Change License: AGPL v3.0.

Start here. The canonical onboarding doc is NEW-USER-ONBOARDING.md (~380 lines: 1-command setup + the 10-step cluster bringup + the 12-MCP verification checklist + the 3 secrets + the troubleshooting FAQ). The 60-second quick path is CHEATSHEET.md.

The per-stack onboarding scripts (Pocket ID + Komodo + Pangolin for the auth mesh; Tuatha + SpacetimeDB for the educational MMO) live in their respective hub READMEs rather than the root, to keep this entry-point scannable:

• Pocket ID + Komodo + Pangolin (single-Passkey login mesh) → AGENTS.md § opencode-safety + bonneagar/README.md
• Tuatha Educational MMO (Babylon.js + FastAPI + TanStack UI) → tuatha/README.md
• BIEP data platform (DLT + Dagster + BAML + CocoIndex + MotherDuck) → orchestration/README.md + dlt_sources/DATA_PLATFORM_ROUTER.md
• Self-host a fresh cluster → docs/DEPLOYMENT-PLAYBOOK.md (the 13-step recreation + 6-step confirmation recipe) + bonneagar/README.md (the 6-step IaC operator quick start)
• Onboarding wizard (the 3-credentials TUI) → scripts/onboard-pocketid.sh

Cianfhoghlaim-Nua ("new Cianfhoghlaim") is the consolidated platform target. The V6 era (2026-09-01) shipped 19 openspec changes + ~10,000 LOC that lift the GCP-first gemini_hackathon/ sister-repo learnings into the canonical OSS-first cianfhoghlaim/ substrate. The 5-pillar pattern: BAML → Convex → A2UI → Hono → React.

Per the operator's direction (2026-09-01), the v6 era plan was extended with 10 follow-on steps to close the feature parity gaps:

# 18 tests, all green
uv run pytest tests/test_adk_subject_actions.py tests/test_phase7_certificate_pipeline.py -v

# 19 openspec changes, all valid
for d in openspec/changes/2026-09-01-*/; do
  uv run openspec validate "$(basename $d)" --strict
done

# 22 BAML functions newly reachable (per the Step 4-8 + Step 9 adds)
uv run python -c "
from baml_client.baml_client.sync_client import b
fns = ['GenerateStudyPlanAssets', 'GenerateOralStudyPlan', 'ExtractNCCAPolicyCriteria', 'Extract<6 NCCE subjects>', 'Extract<6 per-subject marking schemes>', 'ExtractEnglandSubjectSpec', 'ExtractWalesSubjectSpec', 'ExtractNorthernIrelandSubjectSpec', 'ExtractIsleOfManSubjectSpec', 'ExtractScotlandSubjectSpec', 'ExtractWelshSubjectSpec', 'ExtractScottishGaelicSubjectSpec', 'ExtractManxSubjectSpec', 'ExtractBretonSubjectSpec', 'ExtractCornishSubjectSpec', 'ExtractJerseyFrenchSubjectSpec', 'ExtractGuernseyFrenchSubjectSpec', 'ExtractUlsterScotsSubjectSpec']
for f in fns:
    fn = f.replace('<6 NCCE subjects>', 'ComputerScienceLearningGraph').replace('<6 per-subject marking schemes>', 'AccountingMarkingScheme')
    if hasattr(b, fn): print(f'  ✓ {fn}')
"

# 1. Open the consolidated app
cd web/apps/cianfhoghlaim-nua && bun install && bun dev

# 2. Visit /lc/chemistry/study-plan
# 3. The Phase 1 BAML function is reachable
uv run python -c "from baml_client.baml_client.sync_client import b; print(b.GenerateStudyPlanAssets)"

Agent Telemetry (Last Updated: 2026-07-29 22:30:52 UTC)

• Total Cached Structural Documents: 0
• Examinations.ie Cache: 0 files
• NCCA.ie Cache: 0 files
• CurriculumOnline Cache: 0 files