Skip to content

fix(traces): stop dropping all dns.lookup spans - #1530

Merged
shreyamalpani merged 2 commits into
mainfrom
shreya.malpani/dropped-dns-lookup-spans
Nov 19, 2026
Merged

shreyamalpani merged 2 commits into
mainfrom
shreya.malpani/dropped-dns-lookup-spans

Conversation

@shreyamalpani

@shreyamalpani shreyamalpani commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stops the trace processor from dropping every dns.lookup span. It now drops only lookups of internal hosts: localhost, a loopback or unspecified IP (IPv4, IPv6 and IPv4-mapped), or the X-Ray daemon address.

All other filters are unchanged, so spans that were sent before are still sent. The only spans newly sent are dns.lookup spans for external hosts.

Motivation

Fixes #1514.

filter_span_from_lambda_library_or_runtime had an unconditional span.name == "dns.lookup" check, left over from the initial port (#294). It hid lookups made by the function's own code, so when a tcp.connect span was slow, users couldn't tell whether DNS or the TCP connection was slow. The Go extension only ever dropped DNS spans for internal addresses.

Testing

  • cargo test --lib traces::: all 161 trace tests pass. New tests:
    • test_filter_span_keeps_external_dns_lookup: external lookups are kept with an IPv4 address, an IPv6 address, or no address.
    • test_filter_span_drops_internal_dns_lookup: internal lookups are dropped, including ::1, ::ffff:127.0.0.1, 127.0.1.1, ::, the X-Ray address, and localhost with no address.
    • test_filter_span_non_dns_lookup_behavior_unchanged: spans other than dns.lookup are filtered exactly as before.
  • cargo clippy --workspace --all-targets --features default: clean.
  • Tested manually on a Lambda function and verified dns.lookup span shows up

🤖 Generated with Claude Code

The trace processor dropped every span named `dns.lookup`, hiding
customer DNS resolution time. Drop only lookups of internal hosts
(localhost, loopback, unspecified, or the X-Ray daemon address) and
keep the other filters unchanged.

Fixes #1514

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shreyamalpani shreyamalpani changed the title fix(traces): keep customer dns.lookup spans fix(traces): keep dns.lookup spans Nov 18, 2026
@shreyamalpani shreyamalpani changed the title fix(traces): keep dns.lookup spans fix(traces): stop dropping all dns.lookup spans Nov 18, 2026
@shreyamalpani
shreyamalpani marked this pull request as ready for review November 18, 2026 21:07
@shreyamalpani
shreyamalpani requested review from a team as code owners November 18, 2026 21:07
@shreyamalpani
shreyamalpani requested review from litianningdatadog and a balanced review from Copilot November 18, 2026 21:07
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T14:44:32.908815Z 614919a New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The X-Ray filter misclassifies external hostnames that share its IP prefix and misses IPv4-mapped X-Ray addresses because matching occurs before IP canonicalization.

1 open issue
What changed in this PR

This PR addresses #1514 by replacing blanket dns.lookup span filtering with internal-address checks, making customer DNS timing visible in Lambda traces.

Changes:

  • Retains external DNS lookup spans while preserving existing non-DNS filters.
  • Adds internal-address detection for localhost, loopback, unspecified IPs, and the X-Ray daemon.
  • Adds tests for external lookups, internal addresses, and unchanged non-DNS behavior.
File Description
bottlecap/​src/​traces/​trace_processor.rs Narrows DNS span filtering, adds an address helper, and tests filtering behavior.

Review effort: Balanced

Comment thread bottlecap/src/traces/trace_processor.rs Outdated
Comment on lines +591 to +599
if address.eq_ignore_ascii_case("localhost")
|| address.starts_with(AWS_XRAY_DAEMON_ADDRESS_URL_PREFIX)
{
return true;
}
address.parse::<IpAddr>().is_ok_and(|ip| {
let ip = ip.to_canonical();
ip.is_loopback() || ip.is_unspecified()
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for this finding.

Parse and canonicalize the address before comparing it to the X-Ray
daemon IP. A prefix match treated hostnames like
169.254.79.129.example.com as internal and missed IPv4-mapped forms
like ::ffff:169.254.79.129.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@shreyamalpani
shreyamalpani merged commit adf5bc5 into main Nov 19, 2026
64 checks passed
@shreyamalpani
shreyamalpani deleted the shreya.malpani/dropped-dns-lookup-spans branch November 19, 2026 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

All dns.lookup spans are dropped by the extension, including real customer DNS lookups

3 participants