Markdown Data Definition Language (MD-DDL)

Version 0.10.0

MD-DDL is a simple, standard way to organise domain ontology & semantic data models. Data modelling historically uses complex tools which have equally complex underlying data stores. These components are not very 'accessible' for both AI and humans.

md-ddl is: AI‑native · Human‑friendly · Version‑controlled · Semantically rich · Ready for automation

Read the spec: 1-Foundation.md or MD-DDL-Complete.md for single-file AI context

---

What MD-DDL covers

• Domain layer — domains, entities, enums, relationships, events, and constraints
• Source layer — source system declarations and column-level transformation rules (direct, derived, conditional, lookup, reconciliation, deduplication, aggregation), entity fan-out, and worked examples that become tests
• Data products — source-aligned, domain-aligned, and consumer-aligned products declaring scope, shape, consumers, SLA, governance, and masking — driving automated artifact generation
• Governance — classification, PII, retention, regulatory scope, access roles, and masking strategies living with the model, not in a separate system
• Physical artifacts — dimensional star schemas, normalized 3NF DDL, wide-column schemas, knowledge graph (Cypher), JSON Schema, Parquet contracts
• Synthetic data — Python Faker factory classes generated straight from your entity definitions; source, canonical, or destination scope; safe-mode PII for shared environments, realistic mode for local dev

---

Quick Start

Install from PyPI — the standard, its agents, and the linter ship as a Python package, so it can be pulled through a corporate artifactory like any other dependency.

pip install md-ddl
md-ddl init

md-ddl init unpacks the standard into .md-ddl/ and installs the agent wrappers for Claude Code and GitHub Copilot, with their paths rewritten to resolve against the unpacked copy. Run it in an existing project or in an empty directory.

md-ddl init [DIR] --ai claude|copilot|both   which wrappers to install (default: both)
                  --name NAME                project name for the generated instructions
                  --force                    overwrite existing wrappers and instructions
                  --no-instructions          skip CLAUDE.md / copilot-instructions.md
                  --track                    commit .md-ddl/ instead of ignoring it

By default .md-ddl/ ignores itself, so the unpacked standard stays out of your history — upgrade it with pip install --upgrade md-ddl && md-ddl init.

md-ddl init also creates .md-ddlignore (if missing) with .md-ddl/ and venv/, so md-ddl lint . focuses on your model files instead of the installed standard and virtual environment.

init finishes by verifying that every {{INCLUDE: ...}} directive in the installed agent prompts resolves, so a prompt that cannot reach its spec section is reported at setup time instead of silently loading truncated. Re-run that check at any time with md-ddl check. The package also installs the pre-flight linter as md-ddl lint <domain-folder>; exclusions come from .md-ddlignore.

The PyPI package carries the specification, agents, guides, examples, and the architecture references. It omits references/industry_standards/ — the 63 MB of raw BIAN, FHIR and TM Forum snapshots — which stays in this repository; the distilled standards markdown the agents actually load ships with the agents.

---

Or start a new project using the bootstrap script — it sets up git, adds MD-DDL as a submodule, and installs the agent wrappers for your AI tool in one step.

Bash (macOS / Linux / WSL):

bash <(curl -fsSL https://raw.githubusercontent.com/Semprini/md-ddl/main/scripts/start-project.sh)

PowerShell (Windows):

Invoke-Expression (Invoke-WebRequest https://raw.githubusercontent.com/Semprini/md-ddl/main/scripts/start-project.ps1).Content

Or download start-project.sh / start-project.ps1 and run them locally.

---

Learn by conversation: MD-DDL includes Agent Guide an AI learning companion available from the repo via Claude or CoPilot in VS Code. It adapts to your role and goals, teaches through discussion rather than documentation, and routes you to the right specialist agent when you're ready to work.

Example prompts (Claude AI uses /agent-guide, CoPilot uses @agent-guide):

/agent-guide I'm new to MD-DDL — walk me through the key concepts and help me get started.

@agent-guide I'm a data architect at a retail bank. We have 15+ legacy source systems and no canonical data model. Give me an overview of MD-DDL and help me decide where to start.

/agent-guide I need to model a Customer domain. We track individuals and business accounts. Walk me through the MD-DDL approach.

---

Workflow

md-ddl is not rigid or dogmatic. A typical flow is:

1. Position — discuss the architectural approach with Agent Architect: compare to alternatives, prepare material for governance councils or CIOs
2. Discover — scope the domain with Agent Ontology: identify entities, relationships, events, and governance posture
3. Model — write domain.md, entity files, enums, and events
4. Map sources — declare source systems and column-level transforms
5. Publish — declare data products with scope, shape, SLA, and masking
6. Generate — produce physical artifacts and synthetic test data with Agent Artifact
7. Test — compile worked examples, constraints, and SLAs into tests and run them locally (dbt-core + DuckLake) with Agent Test
8. Govern — audit standards conformance and regulatory posture with Agent Governance

Agent Guide helps you navigate between these stages and explains any concept along the way.

---

Using MD-DDL in your project

Your model files live in your own repository; MD-DDL provides the specification, agents, and examples. Bring it in either as a pip dependency (pip install md-ddl && md-ddl init, above) or as a git submodule.

Manual setup

If you prefer not to use the scripts and set up manually:

mkdir myproject
cd myproject
git init
git submodule add https://github.com/Semprini/md-ddl .md-ddl
git submodule update --init

Then copy the agent wrappers for your AI tool:

• Copilot: .md-ddl/.github/agents/*.agent.md → .github/agents/
• Claude: .md-ddl/.claude/commands/*.md → .claude/commands/

If you use Claude, update the copied .claude/commands/*.md files so their agents/ paths read .md-ddl/agents/ (md-ddl init and the bootstrap scripts do this for you).

Next, create your copilot-instructions.md or CLAUDE.md. See the start project scripts for examples.

Update MD-DDL to a new release later:

git submodule update --remote .md-ddl

Suggested project layout

your-project/
  .md-ddl/                   ← the standard: unpacked by `md-ddl init`, or a git submodule
  .github/agents/            ← Copilot agent wrappers  (Copilot users)
  .claude/commands/          ← Claude slash commands    (Claude users)
  domains/
    customer/
      domain.md
      entities/
      products/
  sources/
    salesforce-crm/
      source.md
      transforms/
  generated/                 ← Agent Artifact output (commit alongside the model)
    customer/
      ddl/                   ← SQL DDL, JSON Schema, Parquet contracts, Cypher
      dbt/                   ← dbt project per data product (models from Agent Artifact, tests from Agent Test)
      synthetic/
        factories.py         ← Faker factory classes
        test_factories.py    ← integrity and consistency tests

---

Examples

Five reference domains at increasing complexity:

Example	Focus	Complexity
Simple Customer	Minimal — one domain, three entities, one event	Starter
Financial Crime	AML/KYC/CTF — BIAN alignment, 15+ entities, sources, products, generated artifacts	Intermediate
Healthcare	FHIR R4 — HIPAA governance, source transforms, knowledge-graph product	Intermediate
Telecom	TM Forum ODA — PCI-DSS, associative entities, new relationship types, dimensional product	Advanced
Retail Sales + Retail Service	Bounded Context — two greenfield domains defining Customer differently, cross-domain Customer 360	Advanced

The feature coverage matrix maps every spec feature to the example that demonstrates it.

---

Repository layout

md-ddl-specification/        Normative standard
  1-Foundation.md            Start here to understand the model
  2-Domains.md … 10-Adoption.md
  MD-DDL-Complete.md         Single-file version for AI context windows

guides/                      Non-normative companion guides
  adoption-playbook.md       Brownfield adoption methodology
  diagram-style.md           Mermaid diagram conventions
  lifecycle-versioning.md    Change management and versioning process
  validation-tooling.md      Validation levels, pre-flight checks, tool interface

agents/                      Canonical agent prompts and skills
  agent-guide/               Learning companion and navigator
  agent-ontology/            Domain modelling and source mapping
  agent-artifact/            Physical schema generation
  agent-architect/           Architecture philosophy, data product design, ODPS
  agent-governance/          Standards conformance and compliance auditing
  agent-test/                Test generation and local execution from worked examples

examples/                    Reference examples
  Simple Customer/
  Brownfield Retail/
  Financial Crime/
  Healthcare/
  Telecom/
  Retail Sales/
  Retail Service/

references/                  Architecture and industry reference data
  industry_standards/        BIAN, FHIR, TM Forum reference datasets
  architecture/              Data Autonomy blog series, external references, Mermaid diagrams

src/md_ddl/                  The `md-ddl` PyPI package
  cli.py                     `md-ddl init` / `md-ddl lint` / `md-ddl check` / `md-ddl path`
  lint.py                    Pre-flight linter (also `python scripts/md_ddl_lint.py`)

---

This work is licensed under a Creative Commons Attribution 4.0 International License.