getting-started

Data recipes, connection helpers, and guides for making external data queryable in Zetaris and building a project from it. Source licenses and verification limits are recorded in the catalogs.

Start here

Read START-HERE.md first. Participants use their own Zetaris instance, either downloaded locally or deployed through AWS.

1. Confirm access. For a local installation, start at Zetaris Cloud to obtain the platform bundle, then follow the installation guide. For an existing instance, use your supplied URL and account. This repo supplies the recipes and guides; platform accounts and JDBC drivers are separate.
2. Connect and execute one query. Choose SQL Editor, agent JDBC, or the HTTP helpers in the connection guide.
3. Verify a small dataset. Follow your first dataset: a minimal PokéAPI JSON subset; PUDL is now known to fail. Execute complete statements separately and inspect actual rows.
4. Choose your project. Follow an analysis, data-product, or application path, then use the demo template.

CREATE/SELECT pairs are available for the active REST and file sources. Read their headers and select the statements you need. The single-query helper's --file option sends the whole file as one request. For multi-command files use the new run_sql.py runner; source/model dependencies are declared in open_data/manifest.json and planned by onboard.py. REST verification queries are generally commented out. Do not treat a whole recipe as an automatically executed migration.

Use recipe readiness to choose later sources and troubleshooting for partial setup or errors. See the event page for support and submission details.

Documentation map

The GitHub Pages guide describes how to preview and publish the onboarding website from this branch. It renders the existing guides directly from Markdown.

Doc	What it's for
docs/install/	Installing and configuring Zetaris (local or AWS), and the record of what's been tested
docs/connections/README.md	SQL Editor, Cursor/Codex/Claude Code JDBC and REST choices; included recipe and full command references
Full Lightning command reference	Platform command syntax; use the smaller recipe reference for the starter
open_data/rest_apis/	REST/JSON API sources — catalog, HOWTO, and sql/*_create.sql + *_select.sql pairs
open_data/parquet_csv/	Parquet/CSV file sources — same catalog/HOWTO/sql pattern
open_data/usl/	Unified Semantic Layer contrast build — the same EDGAR+SIC data product, rebuilt with USL instead of REST+VDM
scripts/	Preflight, dependency-aware onboarding, multi-command execution, API helpers, and local fetchers
open_data/manifest.json	Source/model IDs, status, and dependencies for onboard.py
docs/guides/project-paths.md	Optional project directions, including the PUDL chart and EDGAR/PUDL/NOAA guide
docs/guides/zetaris-lightning-sql-companion.md	SQL reference: shapes, quoting, gotchas, and confirmed platform limitations — read alongside the Zetaris Kbase while writing your own SQL
docs/plans/FUTURES.md	The full data-source roadmap, priority order, and status per category
docs/plans/	Active plans for in-progress or upcoming work; docs/plans/archive/ holds completed or superseded plans and the original source-research manifest, kept as historical record

Repo layout

open_data/
  rest_apis/      REST/JSON API sources — HOWTO, source catalog, sql/*_create.sql + *_select.sql pairs
  parquet_csv/     Parquet/CSV file sources — same HOWTO/catalog/sql pattern
  usl/             Unified Semantic Layer contrast build (EDGAR+SIC, rebuilt with USL instead of REST+VDM)
scripts/
  check_zetaris.*, query_zetaris.*, ping_postgres.*, zetaris_api.* — TypeScript and Python helpers
examples/
  pudl-chart/      One query through the HTTP helper to a local HTML chart
docs/
  connections/     Route choices, Cursor/Codex/Claude Code prompts, and recipe command reference
  install/         Installation & configuration guide, plus its test record
  guides/          First dataset, project paths, readiness, recovery, and demo guidance
  plans/           Active roadmap and category plans (FUTURES.md, recipes/); archive/ for completed plans

Folders are added as each category in the roadmap actually ships, rather than scaffolded up front — see docs/plans/FUTURES.md for where this is headed.

Client scripts

scripts/ now includes preflight.py, run_sql.py, and dependency-aware onboard.py, alongside TypeScript (Deno) and Python helpers for connecting to the Zetaris HTTP API and checking a PostgreSQL connection, plus two standalone Python downloaders for the Parquet/CSV package's not-yet-registered sources. See scripts/HOWTO.md for setup, environment variables, and usage for both languages.

Contributing

Adding a new data source, documenting one that doesn't work, archiving a finished plan, or touching anything else in this repo? See CONTRIBUTING.md for the conventions this repo relies on — the _create.sql/_select.sql pattern, the known_to_fail/ writeup format, license verification, and more.