Skip to content

[python] Enable the untrusted deserialization IAST tests (APPSEC-71850) - #8585

Open
avara1986 wants to merge 4 commits into
mainfrom
alberto.vara/enable-python-untrusted-deserialization-iast
Open

avara1986 wants to merge 4 commits into
mainfrom
alberto.vara/enable-python-untrusted-deserialization-iast

Conversation

@avara1986

@avara1986 avara1986 commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Motivation

APPSEC-71850: Python IAST has reported untrusted deserialization since dd-trace-py v3.15.0, but system-tests never ran tests/appsec/iast/sink/test_untrusted_deserialization.py for Python. The Python weblogs had no endpoints for it and the three classes were missing_feature.

Until DataDog/dd-trace-py#21896 (APPSEC-71849), Python reported these findings as UNTRUSTED_SERIALIZATION, a type that the backend and the other tracers do not know. These tests filter on the type, so they fail on every released version.

Changes

  • Weblogs (flask, django, fastapi, aiohttp): new endpoints.
    • GET /iast/untrusted_deserialization/test_insecure?name=... deserializes the tainted name query parameter with yaml.unsafe_load.
    • GET /iast/untrusted_deserialization/test_secure deserializes a constant with the same function.
    • PyYAML 6.0.3 is already in every Python base image (pinned for fastapi and aiohttp, installed by moto[all] for flask and django), so the base images do not change.
  • Test: a Python entry in location_map (flask-poc, django-poc, fastapi), like test_ssrf.py. It checks that the finding points at the weblog file, not at PyYAML or ddtrace.
  • Manifest:
    • The three classes are enabled from v4.18.0-rc1, the version of dd-trace-py main and of the first 4.18 release.
    • aiohttp is enabled from v4.18.0-rc1 too, with its own weblog declaration: IAST taints aiohttp sources only from chore(iast): taint aiohttp request sources dd-trace-py#21471, which is not in any 4.17 or 4.16 release. This gate stays when the backport versions are added.
    • The tornado declaration does not change.

Python IAST reports untrusted deserialization since dd-trace-py 3.15.0, but
the Python weblogs had no endpoints for it and the tests were never run.

Add the untrusted deserialization endpoints to the flask, django, fastapi
and aiohttp weblogs, add a Python location_map entry, and enable the three
test classes from 4.18.0rc1, the first version that reports the findings
as UNTRUSTED_DESERIALIZATION instead of UNTRUSTED_SERIALIZATION
(APPSEC-71849). aiohttp keeps its own 4.18.0rc1 gate, because IAST only
taints aiohttp sources from that version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

manifests/python.yml                                                    @DataDog/system-tests-reviewers
tests/appsec/iast/sink/test_untrusted_deserialization.py                @DataDog/asm-libraries
tests/appsec/iast/test_vulnerability_schema.py                          @DataDog/asm-libraries
utils/build/docker/python/aiohttp/main.py                               @DataDog/system-tests-reviewers
utils/build/docker/python/django/app/urls.py                            @DataDog/system-tests-reviewers
utils/build/docker/python/fastapi/main.py                               @DataDog/system-tests-reviewers
utils/build/docker/python/flask/app.py                                  @DataDog/system-tests-reviewers

@datadog-prod-us1-5

datadog-prod-us1-5 Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Pipelines  Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7534b01 | Docs | View more details | Give us feedback!

@avara1986
avara1986 marked this pull request as ready for review November 19, 2026 21:22
@avara1986
avara1986 requested review from a team as code owners November 19, 2026 21:22
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T21:25:37.953505Z 7534b01 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7534b01f41

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

import mock
import urllib3
import xmltodict
import yaml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 Badge Pin PyYAML in the Flask and Django weblog requirements

The new Flask and Django endpoints directly depend on PyYAML, but their requirements files only install it transitively through moto[ec2,s3,all]; unlike aiohttp and FastAPI, none of the Flask or Django variants pins PyYAML. Their base Dockerfiles install those requirements without a lockfile or constraints, so rebuilding unchanged inputs can select a different version of the deserialization library and change the IAST test behavior independently of ddtrace. Add an exact PyYAML pin to the affected Flask and Django requirements files, as required by INFRA-008, which explicitly covers directly used transitive dependencies. .agents/rules/code-review/infrastructure-and-conventions.mdL50-L55

Useful? React with 👍 / 👎.

Comment on lines +1516 to +1519
@app.route("/iast/untrusted_deserialization/test_insecure", methods=["GET"])
def view_iast_untrusted_deserialization_insecure():
name = flask_request.args.get("name")
_ = yaml.unsafe_load(name)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Document the deserialization endpoint contract

The Python weblogs now expose the insecure and secure deserialization routes, but docs/understand/weblogs/end-to-end_weblog.md remains unchanged, and a search of the committed documentation found no untrusted_deserialization entry. Document both routes, the insecure route's name query parameter, and their expected vulnerable versus constant-input behavior so the shared weblog specification describes the interface these tests exercise. INFRA-010 requires endpoint interface changes to update the matching specification in the same change. .agents/rules/code-review/infrastructure-and-conventions.mdL68-L74

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant