Repository navigation
docs: generate venue and endpoint tables from registry with drift test - #475
Miracle656 merged 4 commits into
Conversation
|
@Godfrey-Delight Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
…229) The venue table now renders from the `venueAdapters` registry and the endpoint table from `openapi.yaml`, both into delimited `<!-- VENUES -->` / `<!-- ENDPOINTS -->` blocks, with `tests/readme.test.ts` failing whenever the committed README disagrees with what the generator produces or a public route is missing from the table. That is the right shape for #229: the list cannot drift because nothing maintains it by hand any more. Regenerating proved the point — the hand-written table was missing twelve routes: `/venues/registry`, `/tvl`, `/rates/:asset`, `/movers`, `/correlation`, `/discovery/search`, `/price/:asset/fiat`, `/usage`, `POST /pairs`, `/price/twap/...`, `/price/vwap/...` and `/snapshot`. Four things on top of the contribution. 1. **The branch did not compile.** `type` and `description` are set on four adapters but were never declared on `VenueAdapter`, so `tsc` rejected all four object literals with TS2353. The fields are now on the interface — and **required**, not optional, which is the part that matters: the table was hand-written and already stale, and generating it from the registry only fixes that if a new adapter cannot be registered without describing itself. The generator's `adapter.type ?? 'DEX'` fallbacks are gone with them; an adapter that says nothing used to produce a row reading "DEX / X price venue", which is the same drift wearing a generator's clothes. 2. **Two adapters added since the branch was cut had no annotation** — Phoenix and DeFindex, which is exactly the omission the issue is about. Making the fields required surfaced both at compile time rather than as two silently absent rows. 3. **Five rows lost detail the README already had**, because the generator's `ENDPOINT_METADATA` map was older than the table it replaces: `/live` lost its link to `docs/health.md`, `/ready` its dependency note, `/snapshot` its query string and full description, `/impact` its `?thresholds=1,5,10` example, `/volumes` its trade-size-concentration wording. Carried across, along with `/usage/me` and the `/graphql` GraphiQL note. Generating the list must not cost the prose. 4. **No way to regenerate.** Added `npm run readme:gen`, beside `openapi:gen` and `client:gen` — a drift guard with no documented command to satisfy it just blocks the next contributor. 127 test files / 1,521 tests pass; tsc clean. Claude-Session: https://claude.ai/code/session_01USgemLt4Rnz4SGB1Srf3GB
|
Merged as And regenerating proved the premise immediately. The hand-written table was missing twelve routes:
Four things I changed on the way in. 1. The branch did not compile
I made both fields required rather than optional, and that is the part I would push back on if you disagree. Your generator had: const type = adapter.type ?? 'DEX'
const desc = adapter.description ?? `${adapter.name} price venue`With optional fields and those fallbacks, a new adapter produces a row saying "DEX | Phoenix price venue" — a row that reads as documentation and contains no information. That is the same drift the issue is about, now with a generator's authority behind it. Required fields mean a new adapter cannot be registered without describing itself; the compiler asks the question at the moment the author knows the answer. The fallbacks are gone, since they are unreachable. 2. Two adapters added since your branch had no annotationPhoenix and DeFindex were both registered in Making the fields required surfaced both as compile errors instead. I annotated them: type: 'Liquidity Pool', description: 'Phoenix AMM pools on Soroban',
type: 'Vault', description: 'DeFindex yield vaults on Soroban, priced per share',If either description is wrong, say so and I will fix it — you know these adapters better than I do. 3. Five rows lost detail the README already had
All carried across, plus One note on 4. No way to regenerateThe PR adds a drift guard and no command to satisfy it, so the next contributor who touches For anyone reading this later: Lens now has five generated artifacts under CI guard — One thing I left aloneRow order now follows I did not change it because completeness matters more than curation here and the openapi document is reasonably grouped already. If you want the grouping back, the clean way is an explicit section order in the generator — a list of path prefixes with the remainder appended — rather than relying on the order keys happen to appear in the YAML. 127 test files / 1,521 tests pass. |
A rate-limited Horizon call returned a body with no `_embedded.records`, the
caller read that as "this pair has no pools", and the venue silently stopped
producing prices until a later cycle happened to land outside the window.
`fetchHorizonJson` now retries a 429 — honouring `Retry-After` when Horizon
sends one, otherwise exponential with full jitter so four ingesters on the same
poll interval do not retry in lockstep — and raises `HorizonRateLimitError`
once the attempts are spent, which the AMM adapter rethrows rather than logging
and swallowing, so the runner counts it and backs the cycle off.
`horizon_rate_limit_remaining` is read from `X-RateLimit-Remaining`, so how
close an instance runs to the limit is visible before it starts failing.
Merged locally; two conflicts, both against work landed since the branch was
cut.
One fix on top of the contribution. **The helper used a bare `fetch(url)`.**
`main` had moved these two call sites to `fetchWithTimeout(url, { signal })`,
which supplies both the configured `HTTP_TIMEOUT_MS` and — since #471 this
afternoon — the cycle deadline's abort signal. Taking the branch's version
verbatim would have reverted both: an unbounded request inside a loop whose
whole purpose is to have an upper bound, in the module that exists because a
slow Horizon is the failure being handled. `fetchHorizonJson` now takes
`{ attempts, signal }` and passes the signal to every request *and* to the
backoff sleep, so an abandoned cycle does not sit out a 30-second retry wait
after it has been told to stop, and gives up rather than spending another
request once it has.
Also kept `main`'s structured `log.error` over the branch's `console.error`
(`src/__tests__/logger.test.ts` rejects `console.*` in non-test source).
Separately, a defect in my own #475 merge this afternoon: `tests/readme.test.ts`
byte-compares README.md against generator output joined with `\n`, so it fails
on every line of an identical table on any Windows checkout, where git supplies
CRLF. It is the fifth guard in this repo to hit that, after `openapi.json`,
`*.yaml`, `*.csv` and `schema.graphql`. README.md is now pinned to LF in
`.gitattributes` and the test normalises line endings as it reads, so a
contributor whose git config differs is not blocked by a diff that is only line
endings.
129 test files / 1,553 tests pass; tsc clean.
Claude-Session: https://claude.ai/code/session_01USgemLt4Rnz4SGB1Srf3GB
Resolve conflicts with Divinity-Labs#475, which landed scripts/generate-readme.ts and the <!-- ENDPOINTS:START --> generated README table on main: - README.md: keep main's single generated endpoint table (ENDPOINTS markers) instead of a second BEGIN/END GENERATED ENDPOINT TABLE block, regenerate it from main's openapi.yaml, and keep the note that the table is generated. - package.json: keep both sides' scripts (openapi:check, graphql:schema:gen, readme:gen, client:gen, client:typecheck). - scripts/generate-openapi.ts: openapi:gen / openapi:check now delegate the README rendering to generate-readme.ts so there is one renderer, while still regenerating/checking openapi.json and the README in one command. - tests/openapi.test.ts: drop the duplicated coverage/generator imports and assert openapi:check reports no stale artifacts (row-level README checks live in tests/readme.test.ts). - ci.yml: run openapi:check after prisma generate (the README renderer loads the venue registry).
Summary
README.mdwith generated sections built from source code registries.typeanddescriptionmetadata to theVenueAdapterinterface and populated them on all registered adapters (SDEX, AMM, Soroswap, Aquarius).scripts/generate-readme.ts(npm run readme:gen) to render the venues table fromvenueAdaptersand the REST endpoint table fromopenapi.yaml(adding the previously missingPOST /pairsendpoint).tests/readme.test.tsto assert thatREADME.mdnever drifts from the venue registry or OpenAPI specification.Related issue
Closes #229
Type of change
Checklist
npx tsc --noEmitpassesnpm run buildpasses