server: agent-card discovery returns 403 to cross-origin browser fetches when allowed_origins is set

[bokelley]

Summary

With 9.0's shared Host/Origin policy (#1301), a request for /.well-known/agent-card.json that carries an Origin outside allowed_origins gets 403. On 8.0.0 the same request got 200. A request without Origin still gets 200. http_policy.py says the policy deliberately covers "protocol, discovery and operational HTTP paths".

Why it matters

The agent card is public metadata. Browser-based tools fetch it cross-origin: agent directories, registry UIs, playgrounds, a buyer's web console probing a seller. With Origin enforcement on discovery, a seller that sets allowed_origins to protect its protocol endpoints also hides its card from every browser tool it didn't list. MCP and A2A requests need the Origin check (DNS rebinding, CSRF against a local agent). A read of public metadata doesn't seem to.

Observed on the Embedded Sales Agent (scope3data/embedded-sales-agent) running 9.0.0b1 with ADCP_DNS_REBINDING_PROTECTION=false and an explicit allowed_origins:

request	8.0.0	9.0.0b1
GET /.well-known/agent-card.json, no Origin	200	200
same, Origin: https://other.example	200	403
MCP / A2A POST, foreign Origin	403	403

Ask

Either exempt the discovery documents (agent card, adagents.json if served) from Origin enforcement, or add a knob such as public_discovery_origins="*". Either way, document the decision in the 9.0 migration notes. If 403 is the intended behavior, a sentence in the 9.0 notes saying so would be enough.

[bokelley]

Triage

Classification: Bug (regression)
Bucket(s): middleware
Status: ready-to-implement

What the experts said:

• security-reviewer: No CSRF or DNS rebinding risk from exempting Origin enforcement on these GET-only public paths — Host check is the right DNS rebinding control for read-only metadata. Verdict: clear bug fix, no security hold warranted.
• ad-tech-protocol-expert: Clear regression — HostOriginMiddleware sits outside _PerRequestCardMiddleware in the middleware stack so Origin enforcement fires before the card handler can serve the request. A2A spec §4.1 leaves no optionality; the card must be unconditionally public. Option A (path-based exemption) is correct; a new public_discovery_origins knob (Option B) would incorrectly imply configurability and invite misconfiguration.

My take: The serve.py docstring already states "On A2A, /.well-known/agent-card.json stays publicly accessible per A2A spec §4.1 — the agent-card route is registered separately." That intent was honoured in A2ABearerAuthMiddleware but not extended to HostOriginMiddleware when explicit_origins was introduced in #1301. The fix is straightforward and matches an existing pattern in auth.py.

Implementation brief:

• File: src/adcp/server/http_policy.py, HostOriginMiddleware.__call__ (~line 85)
• Change: Before the check_origin block, skip Origin enforcement when scope["path"] is in _DISCOVERY_EXEMPT_PATHS (a frozenset of "/.well-known/agent-card.json", "/.well-known/agent.json", "/.well-known/adcp-agents.json"). The Host check (check_host / check_origin loop for len > 1) still runs on these paths when dns_rebinding_protection is on.
• Scope: Also verify that the transport="both" path in _build_mcp_and_a2a_app (serve.py ~line 2955) applies the same exemption — it adds a separate HostOriginMiddleware instance.
• Non-breaking rationale: The change widens access (fixes a regression) and restores the behavior documented in 8.x. No public API is renamed or removed. The adcp-agents.json path should be treated identically to agent-card (always public per AdCP discovery spec).
• Required checks: pytest tests/ (focus on any HTTP policy / discovery tests), mypy src/, ruff check .. Add a regression test: cross-origin GET to /.well-known/agent-card.json with an origin outside allowed_origins should return 200, not 403.

---

Triaged by Claude Code. Session: https://claude.ai/code/session_013XjfGCedMWpnUXfLGBhBM8

---

Generated by Claude Code