WhatIf

Meta AI hackathon · Indie Developer · Build for Your World

Built during October 26–November 4, 2026 by a solo approved participant. The runtime reasoning model is Meta Muse Spark Open (meta-models/muse-spark-open-1026), served through Baseten. Targeted bonus category: Best Build on Meta Open Weight Models.

Visit WhatIf · Watch the demo · Judge quickstart and Meta evidence · Measured worker benchmark

Keyless numerical demo: uv sync --frozen, then bash scripts/demo.sh. Replay a recorded real Meta response with uv run --frozen python evaluation/replay_reasoning.py.

Run the decision before you make it.

Watch the hackathon demo on YouTube

WhatIf compares a proposed intervention with a baseline in reproducible synthetic worlds. The hackathon demo follows a business pricing decision through paired numerical experiments, individual event records, and schema-validated Meta interpretation. It supports a product-price demo, FX and household scenarios, and modular workplace, economy, flood, and contact-health simulations. Reports retain inputs, assumptions, numerical evidence, uncertainty, and execution identity so a result can be inspected and replayed.

These are conditional simulations. Their outputs do not establish real-world human behavior or prove that an intervention causes an observed outcome. The optional language model interprets structured evidence; numerical outcomes come from the simulation engines.

How it works

1. Describe the world. A project owns versioned scenarios, model choices, reviewed data mappings, population inputs, and explicit assumptions. Private workflows require the owner's session. The seeded demo is publicly readable, while running or changing it requires ownership.
2. Prepare the population. The legacy demo generates agents from declared synthetic priors. The household workflow resamples complete supplied household records, optionally fits declared marginal targets, and records source scope, uncertainty, and expansion weights. A blocked review does not become a usable population.
3. Compare paired worlds. The API freezes the execution inputs and places a job on Redis. A worker runs Python CLIs for the selected engine. Baseline and intervention use common random numbers: each pair shares its stochastic realization so the comparison reflects the declared intervention. Separate replicas describe simulation variability.
4. Retain evidence. The engines produce JSON manifests, summaries, and Parquet trajectories or events. The worker verifies identity and artifact hashes before publishing results. PostgreSQL stores ownership, versions, job state, leases, and report metadata; S3-compatible storage holds private evidence files.
5. Inspect results. The UI shows distributions, paired differences, applicable mechanism evidence, and readiness or failure diagnostics. Optional explanations are schema-validated. Calibration separates training, validation, and sealed holdout scoring; a price-module score does not validate the whole world.

Within each pair, the population and random streams match. Comparing independent pairs shows variation inside the declared simulation.

flowchart LR
    Browser[Browser] --> Web[Next.js and tRPC]
    Web --> DB[(PostgreSQL)]
    Web --> Queue[(Redis / BullMQ)]
    Queue --> Worker[TypeScript worker]
    Worker --> Python[Python simulation and analysis CLIs]
    Python --> Worker
    Worker --> Objects[(S3-compatible evidence storage)]
    Worker --> DB
    Web --> Objects
    Worker --> Reasoning[Optional reasoning endpoint]

Loading

Python services exchange explicit file contracts with the worker; they do not query the product database directly. See the architecture decisions and repository layout for the boundaries.

Requirements

• Node.js 22.21+ in the 22.x line, or 26.x. .nvmrc pins the CI and production-worker baseline to 22.21.0.
• pnpm 10.19.0, pinned by packageManager.
• Python 3.12+ and uv.
• Docker with Compose supporting up --wait for PostgreSQL, Redis, and MinIO.
• Linux with Landlock and seccomp support for calibration fitting. Fitting is blocked when its native isolation requirements are unavailable.

The web app defaults to port 3001, the worker health listener to 3101, PostgreSQL to 5433, Redis to 6379, and MinIO to 9000 with its console on 9001. The PostgreSQL host port differs from the container's 5432 port.

Local setup

From the repository root:

nvm install
nvm use
pnpm install --frozen-lockfile
uv sync --frozen
test -e .env || cp .env.example .env

The first two commands assume nvm is installed. Otherwise, install a supported Node version and the pinned pnpm version with your preferred package manager. The copy command preserves an existing .env; keep credentials in that file or the process environment and commit only .env.example. In an existing .env, set BETTER_AUTH_URL="http://localhost:3001" for the default dev server, or use the public origin that matches your custom proxy or port.

Start the backing services:

docker compose -f infrastructure/docker/docker-compose.yml --profile storage up -d --wait
pnpm db:migrate
pnpm -F @whatif/db seed

The database seed creates the demo account, project, and price scenario. To prepare the complete sample workspace on Linux, first check that Landlock and seccomp isolation are available, then run the full sample seed:

uv run --frozen whatif-calibration --isolation-capabilities --out /tmp/whatif-calibration-capabilities \
  && pnpm demo:seed

Run the seed only if the capability check succeeds. It creates artificial FX/price observations, reviewed households, two paired FX variants, a calibration report, four modular reports, and a product-price example. It needs the storage profile and runs without separate web or worker servers. A successfully completed seed can be rerun to verify and reuse its retained examples.

On unsupported calibration hosts, skip pnpm demo:seed: the base project and other supported workflows can still be used, but the complete sample workspace and calibration fitting are unavailable. pnpm db:seed is a convenience command that runs both the base and full seeds; it can fail after creating the base project if the full seed's requirements are not met.

The storage profile builds MinIO Community Edition from a checksum-pinned source revision. Its first build can take several minutes. The pinned upstream is archived; assess its maintenance requirements before a public deployment. The example environment matches this Compose configuration, including storage keys. Use the same database, Redis queue, and object store for the API and worker.

In two separate terminals, run:

pnpm dev:next

pnpm dev:worker

Open http://localhost:3001. Sign in with the development account seeded by the database seed: demo@whatif.local / demo-password. Those values are configurable through DEMO_USER_EMAIL and DEMO_USER_PASSWORD; replace them in shared environments. New accounts can create their own projects.

For a development storage substitute, start PostgreSQL and Redis without the storage profile, then run pnpm -F @whatif/storage dev-s3 in a third terminal. It uses the configured storage endpoint and bucket. This is a local test server, not a production storage service. See storage configuration.

To run only the legacy demo without object storage, set OBJECT_STORAGE_ENDPOINT="" in .env. In that mode the API and worker must share WHATIF_ARTIFACTS_ROOT (the repository's artifacts/ directory by default), and durable artifact publication is unavailable. Historical-data, reviewed-household, calibration, and modular workflows need configured object storage.

Try the workflows

Guided sample workspace

Open Demo — Decision laboratory and choose a workflow. The prepared household FX variants share reviewed sources and population so their intervention effects are comparable. Workplace, economy, flood, and contact-health examples expose both aggregate model outcomes and retained entities/accounts/events. Calibration links back to its exact household source and separates fit, validation, and holdout. All sample inputs and results remain explicitly synthetic.

The project overview combines the run histories. A report links to Aggregate outcomes, Individuals and events, and Inputs and evidence. Household records can follow a member across arms and periods; modular reports compare entity/account state and retain event causes. The product-price example exposes its initial agents and actual per-world events. A copied scenario requires a new review and retention choice before it can run as another variation.

Read the public guides at /guides in the running app or the documentation index. Full retained examples are owner-only; anonymous visitors can browse the public overview. Use the configured seeded account locally, or create an account and prepare your own project.

Product-price example

The presentation example follows 1,000 synthetic agents over 90 days across 10 paired worlds. These are retained median results for that experiment: adopters decrease from 999 to 768, switchers increase from 0 to 228, and revenue falls by about 12% in model currency units. They are conditional on the scenario assumptions, not validated predictions of real people.

Open the demo project in /app, run its price-increase scenario, and watch paired worlds finish. Inspect outcome percentiles, directional probabilities, and any applicable product-switching cascade. The variant action changes the intervention while retaining the population and seed for a comparable result.

The same legacy pipeline can run from the terminal without a database or Redis:

scripts/demo.sh
scripts/demo.sh --full

Fast mode uses 10 agents and 10 worlds. Full mode uses the scenario's declared size and world count. Both replace artifacts/demo/ with generated evidence and explicitly use the mock explanation provider. Use OUT=/tmp/whatif-demo to select a different output directory.

Reviewed household and FX simulations

Create a project, review its world and source mappings, generate a compatible household population, confirm model assumptions, and prepare a baseline and FX intervention. Read historical data ingestion, input populations, and household simulation in that order. Missing scope, evidence, or parameter confirmations block execution rather than being filled with invented values.

Calibration and holdout evaluation

The calibration workflow fits explicitly selected FX-to-price pass-through and lag candidates. It locks source hashes and time splits, fits with training and validation data in an isolated child, then scores sealed results against holdout observations. The owner declares acceptance criteria and may inspect joint coefficient uncertainty and supported sensitivity alternatives.

Read calibration operations and the service guide. A retained real-observation example runs entirely from checksum-pinned local snapshots:

pnpm calibrations:observed --out /tmp/whatif-observed-calibration

The WFP/World Bank dataset guide explains licensing, transformations, missing coverage, and the example's failed acceptance criteria. Do not adjust its protocol after inspecting holdout scores.

Modular simulations

The project workbench at /app/projects/{id}/modular combines reviewed modules with explicit entities, accounts, stocks, parameters, and intervention rules. The scheduler enforces declared resource and accounting constraints. Private reports at /app/modular-runs/{id} retain paired summaries, mechanism evidence, replay identity, and lifecycle state.

Read the modular engine guide and evaluation guide for supported mechanisms, parameter candidates, observed-data hooks, and applicability limits.

Optional explanations

The hackathon presentation used Meta Muse Spark Open through Baseten. The numerical engines compute the outcomes; the model interprets retained evidence and proposes follow-up experiments. In the illustrated run, switching clusters were detected in 10 of 10 synthetic replicas; the largest recorded cluster contained 35 agents. This does not mean every replica had the same cluster.

Explanations are disabled by default. To use a real OpenAI-compatible chat endpoint, set these worker variables:

WHATIF_REASONING_PROVIDER=openai
REASONING_BASE_URL=https://your-provider.example/v1
REASONING_MODEL=your-model-id
REASONING_API_KEY_ENV=YOUR_PROVIDER_KEY
REASONING_MAX_TOKENS=4096

Export YOUR_PROVIDER_KEY through your shell or deployment secret mechanism. Alternatively supply REASONING_API_KEY privately. Set WHATIF_REASONING_PROVIDER=mock for explicit development demonstrations. A provider failure is recorded separately and numerical results remain available. Remote model revisions and generated text are not guaranteed to replay exactly. See the reasoning service.

Quality checks

pnpm lint
pnpm lint:ws
pnpm format
pnpm typecheck
pnpm build
pnpm lint:python
pnpm build:python
pnpm test
pnpm test:python

Formatting covers supported source, documentation, and configuration throughout the repository. Checksum-pinned dataset evidence and generated lockfiles are excluded. pnpm test runs each workspace's declared suite, including shared modular contracts. API and queue integration tests require migrated PostgreSQL and Redis; use a disposable test database and do not point them at production data. Python tests exercise engine behavior, determinism, malformed inputs, contract parity, evidence integrity, and calibration isolation.

Browser tests need installed Chromium and a migrated, seeded test database:

pnpm -F @whatif/web exec playwright install --with-deps chromium
pnpm e2e

Playwright starts the web app and worker when they are not already running. Additional storage and failure suites use explicit environment flags. The container gates create isolated stacks and verify crashes, lease recovery, artifact publication, populated migration upgrades, private reports, and restore:

bash scripts/reliability-containers.sh
bash scripts/paired-evidence-integration.sh
bash scripts/modular-platform-integration.sh

Run these gates sequentially because they share host ports. See CONTRIBUTING.md for the test setup and CI for the exact automated commands.

Operations and failure behavior

A worker owns a renewable database lease while processing a run. Attempts have separate artifact directories, and publication requires the current lease. Expired work can be recovered after a crash; cancellation is persisted in the database. The worker exposes /healthz for liveness and /readyz for dependency readiness. Deploy the API, worker, Python services, and migrations together when contracts or runtime identity change.

Private evidence has retention and erasure workflows. Reconciliation commands such as pnpm datasets:cleanup, pnpm populations:cleanup, pnpm simulations:cleanup, pnpm calibrations:cleanup, and pnpm modular:cleanup are dry runs by default. Inspect their output and pass --apply only when the proposed deletion matches your policy. Database metadata alone cannot restore deleted evidence.

See local development and upgrades, reliability, and the deployment checklist for leases, backfills, backup/restore, storage policies, and deployment requirements.

Documentation and limits

• Architecture and package map
• Simulation engines
• Population generation
• Analytics and evidence checks
• Datasets and provenance
• Evaluation methodology
• Cloud deployment

Percentiles and probabilities describe the declared synthetic experiment. A seed reproduces a numerical run only with compatible inputs and runtime. Marginal population fit does not identify unmeasured joint distributions, and assumed social ties are not observed relationships. Product-switching cascade analysis is scoped to the product-price mechanism. Calibration tests one conditional price equation and does not establish a valid FX forecast or a complete household model. Read each engine's applicability and uncertainty notes before using its results to inform a decision.

Licensed under MIT.