Skip to content

test(e2e): real-backend first-run journey, real-shape a11y fixtures, and fixes it found - #183

Merged
velixio merged 24 commits into
mainfrom
test/e2e-real-backend
Nov 19, 2026
Merged

velixio merged 24 commits into
mainfrom
test/e2e-real-backend

Conversation

@velixio

@velixio velixio commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Builds on #175 (the combined train branch train/2026-11-19). Merge #175 first, or retarget after it lands.

Why

Every web test so far ran against a mocked API. This drives the real UI against the real Go backend (nginx, api-go, Postgres, worker) as a new user would, and compares what the real API returns with the a11y fixtures.

What changed

  • apps/web/e2e/real/first-run.spec.ts, its own playwright.config.ts and a README, run with bun run test:e2e-real against a stack named by OPENGTM_E2E_BASE_URL. Not part of test:a11y, not in PR CI. It does first-run setup, the Get started checklist (sample workbook, real run, polls state), every sidebar and admin route in light and dark (API status, console, error states, screenshot), and one action per area. Writes screenshots and findings.json.
  • a11y fixtures: new fixtures/real-shapes.ts with shapes captured from the real API (records summary, analytics, settings providers, destinations, outreach, ...). The suite now fails a page that shows "temporarily unavailable", "returned an invalid response" or "could not be verified", and waits for the query retry (1s) before auditing; previously a failing query was audited as a skeleton.
  • The /overview error: a stale fixture, not a product bug. GET /api/records/summary had no fixture, so the mock answered [] and parseRecordOverview rejected it. The real body is an object with the seven counts, workbooks[], imports[], sampled_at, and the real page renders fine on it.
  • analytics.tsx: with a real-shaped fixture the page renders for the first time in the suite and axe found low-contrast trend text (emerald-500 / red-400 on white, 2.47:1). Now 700 in light, 400 in dark.
  • onboarding: the "sample loaded" announcement said "Open it from step 3"; step 3 is done and has no actions. Now "Run it from step 4". Unit test added.
  • records search (Go): the list search cast accepted_fields to text, so domain, name or s matched every record (31 of 31 in the journey). It now searches values with jsonb_each_text. Integration test added.

Findings not fixed (bug reports)

  1. Workbook is marked "failed" when cells find no data. Steps: load the sample, run it with free providers. Result: 86 cells complete, 89 no_data, 25 no_answer (no AI provider configured); /api/workbooks/{id} status failed, 25/25 rows; overview shows a red Failed badge and "1 failed workbook"; the workbook shows "Run needs attention: Some work failed". Expected: a run that finishes with empty provider results is complete, and the missing AI provider is named. Cause: apps/backend-go/internal/workbook/run.go finalize (res.Errors > 0 gives failed), and cell.go ~846 stores no_data as status error. Bigger than one file; product decision needed.
  2. Analytics reads the legacy lead table only. With 25 companies and an enriched workbook, /analytics shows 0 leads, 0 verified, "No leads collected", while overview shows 25 companies. The page says "Scope: legacy leads" and links to "Company & people overview", so it is disclosed, but a new user lands on all zeros. Suspected: apps/web/src/pages/analytics.tsx and the /api/analytics/* handlers (leads table).
  3. login.tsx:85 fetches https://api.github.com/repos/... from the browser on the setup/login page. A self-hosted app calling a third party on first load; unauthenticated GitHub rate-limits (one run logged a 403 console error). Needs a decision (remove, or proxy and cache).
  4. Search treats % and _ as wildcards (not escaped in the ILIKE). Not changed here.

Expected, not bugs

  • /automations and /watches return 404 "automations disabled" / "intent poller disabled" on a default install (flags off), and the pages handle it. The a11y flags fixture turns them on to exercise those pages.
  • The first visit uses the dark theme (app default), so first screenshots are named -default.

How it was tested

  • Real run, fresh stack on its own Compose project (opengtm-e2e, ports 23480-23483, subnet 10.177.43.0/24, rehearsal override), bun run test:e2e-real: passed, 891 /api and /auth calls, 887 x 200 and 4 x 201, no 4xx or 5xx, no page errors. Setup, sample load, real run (settled at failed 25/25, see 1), 21 routes x 2 themes, and all 7 actions (company, audience from filter, webhook dialog cancel, template workbook, Pipeline setting, command palette, 5-row CSV import with results).
  • bun run lint: clean. bun test tests: 204 pass. bun run build: ok. bun run test:a11y: 52 pass (one earlier run had a single mount timeout on /chat mobile under load; the rerun passed). Removing the summary fixture makes the new guard fail /overview.
  • Go: go test -tags integration ./internal/records against the stack's own Postgres: ok (398s). The new test fails without the fix.

Not tested

  • Other browsers, mobile viewports on the real stack, and the Python-era data paths. The a11y baseline was not regenerated (no change needed).
  • The sample run makes real outbound requests, so its results vary by network.

…tegy

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Nine declarative manifests (Anymail Finder, Bouncer, MillionVerifier,
Findymail, ContactOut) with docs-derived fixtures and one generic test,
a wiring test across catalog, workbook and settings registries, and a
rewritten manifest format guide.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Replace every native confirm/prompt with useConfirm()/usePrompt() (focus managed, destructive style, named objects). Fix target-size at shared level via --gtm-target-min (accounts for the shell's CSS zoom), fix contrast and unnamed controls, add six routes to the axe gate, and empty the baseline.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
…ls; changelog for the Go cutover

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
…_verify words

response.value_maps translates a mapped value with an explicit default,
request.omit_empty drops empty headers, query parameters and body keys,
and values substituted into request.url are escaped. The three verifier
manifests now emit only valid, invalid, catch_all, risky, unknown.
Adds Settings key cards for LeadMagic and Prospeo.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Audience and automation delivery adds a lead to a HeyReach (LinkedIn) or lemlist (email) campaign with the workspace's own API key. Workbook output columns are not part of this change.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
…ints

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Upserts each member as an Attio person matched on email and links or creates its company by domain, with the workspace's own token.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
… train/2026-11-19

# Conflicts:
#	apps/backend-go/internal/settings/registry.json
…11-19

# Conflicts:
#	apps/backend-go/internal/settings/registry.json
…rain/2026-11-19

# Conflicts:
#	apps/web/src/components/command-menu-content.tsx
Drives the built app against a running stack (setup, Get started, sample run,
every route in light and dark, one action per area), records API statuses,
console output and error states, and writes screenshots and findings.json.
Separate config and script; not part of test:a11y and not run in CI.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
…nalytics trend contrast

The mocked suite answered unmatched endpoints with [], so /overview, /analytics and
/settings rendered their error states (or a skeleton) and were never audited with
content. Add shapes captured from the real API, fail any page that shows an
error state, and wait for the query retry before auditing. The analytics page,
now rendered with data, exposed low-contrast trend text (emerald-500 and red-400
on white); use the 700/400 pair.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
Loading the sample completes step 3, and done steps carry no actions, so
Open it from step 3 pointed at nothing. The workbook link is step 4's.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
The list search cast accepted_fields to text, so domain or name (or any
short term) matched every record. Search the values with jsonb_each_text.

Signed-off-by: Shimul Debnath <270455167+velixio@users.noreply.github.com>
@velixio
velixio requested a review from debpalash as a code owner November 19, 2026 01:07
@github-actions

Copy link
Copy Markdown

All contributors on this pull request have signed the OpenGTM CLA. Thank you!

Agreement: OpenGTM CLA 1.0

@velixio
velixio merged commit db8a718 into main Nov 19, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed All contributors signed the CLA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant