Repository navigation
Conversation
… reconnects Add system tests for the MCP server ID contract: - OpenAI hosted MCP: the RFC host canonicalization vectors (IPv4 number forms, IPv6, percent-encoded and underscore hosts) get the RFC IDs, and an IDN host falls back to the name key. - OpenAI hosted MCP: two servers on one origin share an ID and are reported as one merged mcp_servers entry with the tools of both. - Anthropic stdio MCP: two requests, each with a new server process and MCP session, report the same server entry and ID. The OpenAI cassettes are generated by gen_aiguard_mcp_cassettes.py; the Anthropic test reuses copies of the recorded tool_use cassettes. APPSEC-71867 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove the Anthropic organization, workspace and request IDs, CF-RAY, trace and timing headers, datadog-entity-id and the CSP header carrying a client token, and replace the real evaluation and message IDs with synthetic ones. Only response data and headers change: the hashed request bodies are kept. APPSEC-71867 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
|
The anthropic-py weblog /mcp/create can start the echo server from the absolute path of the interpreter (command_path). The new test checks that only the program name reaches AI Guard and the server ID is unchanged. It reuses the scrubbed cassettes, and is declared missing_feature for Python until DataDog/dd-trace-py#21929 ships. APPSEC-71867 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🎉 All green!🧪 All tests passed 🔗 Commit SHA: 47fe661 | Docs | View more details | Give us feedback! |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 47fe6610f3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Start the echo server with the absolute path of the interpreter instead of "python". | ||
| command_path: bool = False |
There was a problem hiding this comment.
Document the new MCP request parameter
INFRA-010 in .agents/rules/code-review/infrastructure-and-conventions.md requires weblog interface changes to update the corresponding specification. The Anthropic /mcp/create endpoint now accepts command_path, changing how the interpreter is launched, but this commit makes no update under docs/understand/weblogs/. Document the parameter, its default, and its behavior so other weblog implementations can reproduce this test interface.
AGENTS.md reference: AGENTS.md:L49-L49
Useful? React with 👍 / 👎.
| "*": v4.18.0-rc1 | ||
| anthropic-py@0.75.0: "irrelevant (anthropic<0.84.0 has no MCP tool helpers)" | ||
| ? tests/integration_frameworks/llm/anthropic/test_anthropic_ai_guard_mcp.py::Test_AnthropicAiGuardMcp::test_ai_guard_anthropic_mcp_command_path | ||
| : missing_feature (stdio commands are reported with their full path until DataDog/dd-trace-py#21929) |
There was a problem hiding this comment.
Quote the manifest declaration containing a hash character
META-004 in .agents/rules/code-review/activation-and-metadata.md explicitly requires quoting manifest values containing #, even when YAML accepts them unquoted. This new declaration contains DataDog/dd-trace-py#21929; quote the whole value to follow that convention and avoid ambiguity with YAML comment syntax.
AGENTS.md reference: AGENTS.md:L48-L48
Useful? React with 👍 / 👎.
Motivation
Jira: APPSEC-71867
An AI Guard MCP server ID is
mcp_followed by the first 16 hex characters of the SHA-256 of a single identity key: the URL origin, then the connector ID, then the stdio command, then the configured name. Every tracer must compute the same ID for the same server. The existing MCP tests already cover the url, connector and stdio keys, URL sanitization, ID resolution and the kill switch. These tests add the rest of the contract.Changes
test_ai_guard_openai_mcp_server_id_host_vectors(OpenAI hosted MCP): eachserver_urlhost is canonicalized with the WHATWG URL rules before the ID is computed. The shared vectors are a numeric IPv4, an IPv4 with port, an IPv6, a percent-encoded host and an underscore host, and each one gets its exact expected ID. An IDN host has nourland falls back to the name key.test_ai_guard_openai_mcp_servers_sharing_an_id_merge(OpenAI hosted MCP): two servers on the same origin with different paths share an ID. They are reported as onemcp_serversentry with the first server's fields and the tools of both.test_ai_guard_anthropic_mcp_reconnect_same_server_id(Anthropic stdio MCP): two requests, each starting a new stdio server process and MCP session, report the same server entry and ID.test_ai_guard_anthropic_mcp_command_path(Anthropic stdio MCP): the weblog/mcp/createtakes an optionalcommand_pathflag that starts the echo server from the absolute path of the interpreter. Only the program name (python) may reach AI Guard, so the server ID stays the same. It is declaredmissing_featurefor Python until chore(aiguard): merge MCP servers sharing an ID and report stdio commands by program name dd-trace-py#21929 ships.utils/scripts/gen_aiguard_mcp_cassettes.py. The new scenarios are appended last, so no existing cassette changes.test_ai_guard_anthropic_mcp_tool_usecassettes, renamed to their own test names. Cassette names hash only the request body, so the copies match unchanged. Identifying response headers and IDs were scrubbed from the copies.No manifest changes. The new tests live in files that already have entries:
v4.18.0-rc1. The WHATWG host canonicalization is not in 4.17.x.missing_featurefor OpenAI andincomplete_test_appfor Anthropic (the weblog has no/mcp/createendpoint).missing_feature.When Node.js gets MCP support it will need OpenAI cassettes of its own, because the AI Guard
metait sends is part of the hashed request body.Validation, run locally against the dd-trace-py main wheel (
python@4.18.0-rc1):./run.sh INTEGRATION_FRAMEWORKS -L python --weblog openai-py@2.0.0 tests/integration_frameworks/llm/openai/test_openai_ai_guard_mcp.py: 8 passed./run.sh INTEGRATION_FRAMEWORKS -L python --weblog anthropic-py@0.84.0 tests/integration_frameworks/llm/anthropic/test_anthropic_ai_guard_mcp.py: 5 passedtest_ai_guard_anthropic_mcp_command_path: XFAIL on the dd-trace-py main wheel, for the right reason (the response evaluation misses its cassette because the full path is sent), and passes with the chore(aiguard): merge MCP servers sharing an ID and report stdio commands by program name dd-trace-py#21929 branch loaded locally../format.shpasses, including the AI Guard MCP cassette drift check.Workflow
system-tests-reviewers: ask to one of youre co-worker familiar with the tested feature.system-tests-core(slack)🚀 Once your PR is reviewed and the CI green, you can merge it!
🛟 #apm-shared-testing 🛟
🤖 Generated with Claude Code