Skip to content

docs: add CoreWeave Serverless Inference provider - #1562

Open
jmcshane wants to merge 6 commits into
agentgateway:mainfrom
jmcshane:docs/coreweave-provider
Open

jmcshane wants to merge 6 commits into
agentgateway:mainfrom
jmcshane:docs/coreweave-provider

Conversation

@jmcshane

@jmcshane jmcshane commented Nov 18, 2026 •

Copy link
Copy Markdown

Docs for the coreweave provider preset added in agentgateway/agentgateway#4463. Content is gated to main (1.8) like Nebius. Merge after #4463.

New pages

  • Standalone integrations/llm/providers/coreweave: UI and config file setup, the optional OpenAI-Project header (requestHeaders.set) for usage attribution, serving custom LoRA adapters by wandb-artifact:/// URI, and a provider: custom fallback for versions without the preset.

  • Kubernetes integrations/llm/providers/coreweave (shared asset): Secret, AgentgatewayModel with provider: CoreWeave and policies.headers, a wildcard match.model: wandb-artifact:///<team>/<project>/* model that serves every LoRA in a project, and an AgentgatewayBackend + HTTPRoute example.

  • Response settings section (both pages): JSON mode, structured output, tool calling, and streaming pass through unchanged; cached tokens are priced as cache reads; per-caller cache_salt with a final transformation so CoreWeave's shared prefix cache is isolated by caller. A regular transformation loses cache_salt when Messages/Responses requests are converted, so the example uses finalTransformation (verified locally against a mock upstream for all three APIs).

Updated

  • Lists of providers on the custom, OpenAI-compatible, providers index, about, LiteLLM import, landing, and LLM API comparison pages
  • API reference ModelProvider enum
  • Standalone and Kubernetes release notes

Adds color and monochrome (bw/) CoreWeave icons for the page nav, the OpenAI-compatible list, and the comparison table.

Testing

  • Standalone examples run end to end against CoreWeave with a real key, including a LoRA artifact URI, OpenAI-Project, and the cache_salt final transformation (results in llm: add coreweave provider agentgateway#4463). Kubernetes examples not run in a cluster.
  • hugo build completes with no errors or warnings; both new pages render and the cross-links resolve.

🤖 Generated with Claude Code

Document the `coreweave` provider preset from agentgateway/agentgateway#4463
for standalone and Kubernetes (main / 1.8):

- New provider pages: standalone page with UI and config file setup,
  OpenAI-Project header, LoRA adapter usage, and a custom-provider
  fallback for earlier versions; Kubernetes page with AgentgatewayModel
  (including a wildcard match for LoRA artifact URIs) and
  AgentgatewayBackend examples.
- List CoreWeave on the custom, OpenAI-compatible, providers index,
  about, import, landing, and LLM comparison pages, and in the API
  reference ModelProvider enum.
- Add release notes entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James McShane <james.mcshane@solo.io>
jmcshane and others added 4 commits November 18, 2026 20:10
Add color and monochrome CoreWeave icons and use them on the provider
pages, the OpenAI-compatible provider list, and the LLM comparison table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James McShane <james.mcshane@solo.io>
Note that CoreWeave response settings pass through unchanged, that cached
tokens are priced as cache reads, and show how to isolate the prefix cache
per caller with a cache_salt final transformation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James McShane <james.mcshane@solo.io>
AI assistance: OpenAI Codex

Signed-off-by: crwvrosenblum <296972394+crwvrosenblum@users.noreply.github.com>
docs: clarify CoreWeave setup and LoRA rollouts
@jmcshane
jmcshane marked this pull request as ready for review November 19, 2026 03:13
@jmcshane

Copy link
Copy Markdown
Author

jmcshane#1 added review from the CW team

howardjohn pushed a commit to agentgateway/agentgateway that referenced this pull request Nov 19, 2026
Adds a `coreweave` preset for [CoreWeave Serverless Inference](https://docs.coreweave.com/products/inference/serverless) (`https://api.inference.wandb.ai/v1`).

CoreWeave Serverless Inference serves open models and user-uploaded LoRA adapters through the OpenAI Chat Completions API only, so the preset declares only the Completions format. Messages and Responses requests are converted to Completions. LoRA adapters are addressed by a W&B artifact URI in the `model` field (`wandb-artifact:///<team>/<project>/<artifact>:<version>`, see [Bring your own LoRA](https://docs.coreweave.com/products/inference/serverless/lora)); the gateway forwards that value unchanged, so no extra handling is needed.

Follows the shape of #4225 (Nebius):
- proto `PROVIDER_PRESET_COREWEAVE = 18`, xDS mapping, `ModelProvider` `CoreWeave` in the CRD, controller translator
- `schema/config.json` and UI provider list, display name, and icon (light and dark variants of the CoreWeave mark)
- models.dev lists CoreWeave under the `wandb` provider id, mapped to `coreweave`. The catalog change is only the coreweave section from `agctl catalog import --source models.dev --providers wandb`.
- LiteLLM import maps the `wandb/` and `coreweave/` prefixes

The optional `OpenAI-Project: <team>/<project>` header CoreWeave uses for usage attribution can be set with `requestHeaders` (standalone) or `policies.headers` (AgentgatewayModel), so the preset does not add it. Note this could be added in the future as an optional configuration value, but the PR does not add it for the time being.

**Testing**
- `cargo test -p agent-llm`, `cargo test -p agentgateway --lib import`, Go translator/catalog/api tests, UI typecheck and biome check pass; CRD and schema regenerated.

Docs: agentgateway/website#1562

- [X] As required by the [Code of Conduct](https://github.com/agentgateway/agentgateway/blob/main/CODE_OF_CONDUCT.md#generative-ai-policy), the description, docs, and comments (words meant for humans) are written by a human, not by an LLM.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants