icebergsim-dataExplorer

A generic, local-first data workbench. Import CSV / TSV / Excel / JSON / R (.rds, .RData) files, explore them with the classic drag-and-drop PivotTable.js UI, and keep everything — datasets, named views, and your session — in the browser's own database. No account, no server, no upload.

▶ Try it live: https://ebergel.github.io/icebergsim-dataExplorer/

What it does

• Import CSV/TSV (papaparse), Excel (SheetJS, multi-sheet picker), JSON (array-of-objects or {col: values[]}), and R data frames via WebR (R compiled to WebAssembly). Types are inferred for the tabular formats; for R the mapping is fixed (factor→label, Date→ISO string, logical→boolean, numeric→number, NA→null).
• Explore with react-pivottable: drag fields to rows/columns/values, pick an aggregator and a table renderer.
• Save views — named pivot configurations per dataset.
• Autosave & restore — the active dataset, current pivot state, and active view are saved continuously and restored on next launch.
• Workspace export/import — one JSON file with all datasets, views, and session, for backup or moving between browsers.

Scope — read this honestly

This app is local-first and per-browser. Your data lives only in this browser's IndexedDB on this device. That means:

• No cloud sync. Nothing is uploaded anywhere. Another browser, another machine, or cleared site-data won't have your datasets.
• Storage is best-effort unless the browser grants persistence. On first import the app requests durable storage; the footer shows persistent, best-effort, or unavailable (browsers without the Storage persistence API). Under storage pressure a best-effort origin can be evicted.
• Export is your backup. Use Export workspace to save a JSON file. That file is the portability and durability story — treat it like a backup.
• Caps. Up to 500,000 rows per dataset (hard limit); an advisory banner appears at 250,000 rows or more, where the pivot UI may feel slow.

Develop

npm install --legacy-peer-deps   # a transitive dep declares a newer-React peer; the flag overrides the conflict
npm run dev                      # Vite dev server

Test

npm run check:fast    # typecheck + lint + fast unit/property tests
npm test              # fast Vitest suite
npm run test:slow     # includes the WebR-in-Node R-conversion tests (A1)
npm run test:coverage # coverage on src/lib/** (gate: >= 80%, runs slow tests)
npm run e2e           # Playwright core flow against the production build
npm run build && npm run check:bundle  # asserts WebR is loaded only via dynamic import (needs a build)

Two test tiers back the parts that can be silently wrong — file-conversion fidelity and persistence round-trips: Tier A exact fixtures (incl. a canonical hand-specified R data frame) and Tier P seeded round-trip properties (fast-check). The spec, cases, and fixtures live in spec/ and fixtures/ and are frozen; see BUILD_PLAN.md and CLAUDE.md.

Build & deploy

npm run build     # typecheck + static Vite build into dist/
npm run preview   # serve the production build locally

The output is a fully static bundle (relative asset paths, base: "./"), so it deploys to GitHub Pages or any static host. Pushes to main deploy via .github/workflows/ci.yml. WebR's WebAssembly runtime is a separate chunk, downloaded only when you open an .rds/.RData file — the base bundle stays light.

Stack

Vite · React 18 · TypeScript (strict) · Dexie (IndexedDB) · react-pivottable · papaparse · SheetJS · WebR · Vitest + React Testing Library + fast-check + fake-indexeddb · Playwright.

Documentation

• spec/SPEC.md — the frozen contract: data model, per-format rules, the R type-mapping table, the pivot-state whitelist, storage, and error codes.
• BUILD_PLAN.md — the product/architecture plan and the phase-by-phase build gates.
• CLAUDE.md — the short-form design rules and commands.
• ENGINE_NOTES.md — observed third-party library quirks (react-pivottable interop, SheetJS date handling, WebR NA semantics).

Contributing

Contributions are welcome — see CONTRIBUTING.md for setup, the ground rules (pure lib/, the persistence whitelist, dynamic-import-only WebR, frozen spec/fixtures), and the PR process. Please also read the Code of Conduct. To report a vulnerability, follow SECURITY.md — not a public issue.

License

MIT © Eduardo Bergel