Repository navigation
feat(trace-agent): add resource peer tags from refreshed mappings - #203
Draft
aquamatthias wants to merge 2 commits into
Draft
aquamatthias wants to merge 2 commits into
aquamatthias wants to merge 2 commits into
Conversation
Refresh the embedded peer_tags_mappings.json with the published semantic mappings artifact (74 concepts) and add six concepts to PEER_TAG_CONCEPTS: peer.aws.arn, peer.aws.sqs.queue.url, peer.gcp.project_id, peer.kafka.cluster.id, peer.gcp.pubsub.topic.name and peer.gcp.pubsub.subscription.name. The mini agent now aggregates stats on, and advertises in /info, the ARN, SQS queue URL, GCP project, Kafka cluster ID and Pub/Sub topic and subscription peer tag keys. The refresh also adds db.system.name as a fallback of peer.db.system. Add an integration test that checks Pub/Sub topic and subscription stay separate peer tags in agent-computed stats. Jira: REDUDM-1846
Collaborator
Author
|
cc @lucaspimentel: this is the serverless-components port of DataDog/datadog-agent#60159 you asked about. The Lambda extension port follows once I have write access there. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds six peer tag concepts to the mini agent's
PEER_TAG_CONCEPTSand refreshes the embeddedpeer_tags_mappings.jsonfrom the published semantic mappings artifact (v20261119075559-921ed898, 74 concepts):peer.aws.arnpeer.aws.sqs.queue.urlpeer.gcp.project_idpeer.kafka.cluster.idpeer.gcp.pubsub.topic.namepeer.gcp.pubsub.subscription.nameWith agent-computed stats (the default), the mini agent now aggregates on these source keys and lists them in
/infopeer_tags. That's 60 keys instead of 43:aws.function.arn,aws.queue.url,aws.sns.topic.arn,aws.sns.topic_arn,aws.sqs.queue.url,aws.state_machine.arn,aws.step_functions.state_machine.arn,aws.stream.arn,aws.table.arn,aws.topic.arngcloud.project_id,gcp.pubsub.topic.name,gcp.pubsub.subscription.namekafka.cluster_id,messaging.kafka.cluster_id,messaging.kafka.cluster.iddb.system.name, added by the refresh as the last fallback ofpeer.db.systemThe mappings file is copied verbatim. The crate reads only
concepts.*.fallbacks[].name, so the extra concepts and metadata are ignored. The 16 concepts the agent already used are unchanged, except for thedb.system.namefallback.The Kubernetes concepts (
peer.k8s.cluster.uid,peer.k8s.namespace) are not added; the Datadog Agent computes those itself.Motivation
REDUDM-1846, part of REDUDM-1740 ("APM: keep the identity of called resources in trace stats"). Without these keys, stats computed by the mini agent can't tell apart calls to different SNS topics, SQS queues, Pub/Sub topics and subscriptions, Kafka clusters or GCP projects. The Datadog Agent gets the same concepts in DataDog/datadog-agent#60159.
Additional Notes
Tracers that read
/infowill send up to 17 more peer tag keys. ARNs and queue URLs identify single resources, so stats cardinality grows by roughly the number of distinct resources each service calls.Describe how to test/QA your changes
test_peer_tag_keys_exact_setis updated to the 60 expected keys.test_pubsub_topic_and_subscription_peer_tags_in_flushed_statssends a Pub/Sub producer span and consumer span through the mini agent with mock intake. It checks that the topic and the subscription land in separate stats groups, that neither group carries the other's tag, and that both carrygcloud.project_id.cargo test -p datadog-trace-agent: 94 unit and 13 integration tests pass.cargo clippy -p datadog-trace-agent --all-targets -- -D warningsandcargo fmt --all -- --checkare clean.🤖 Generated with Claude Code