Skip to content

feat(idp): add scoped graph queries and bounded continuation - #840

Open
htoordd wants to merge 4 commits into
DataDog:mainfrom
htoordd:codex/idp-query-controls
Open

htoordd wants to merge 4 commits into
DataDog:mainfrom
htoordd:codex/idp-query-controls

Conversation

@htoordd

@htoordd htoordd commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Make idp entities query useful for connected context and bounded inventories while preserving the evidence needed to interpret the answer. Select related attributes and edge measurements, scope supported measurements, and continue the exact effective query through an executable argument array.

pup --read-only idp entities query 'ref:"ref:service:checkout-api"' \
  --field name,owner --include owner_teams,runtime_downstream_services \
  --fields team=name,handle \
  --edge-fields runtime_downstream_services=requests_count,error_rate

pup --read-only idp entities query 'kind:service AND owner:payments' \
  --field name,owner --order-by name:asc --limit 100 --max-results 500

Changes

  • Add per-kind/edge projections, live-schema validation, useful defaults for unfamiliar kinds, and richer kinds describe output. Reuse the shared query/list/search option parsing. Preserve returned identity attributes, edge metadata, nulls, explicit empty relationships, and server warnings; distinguish missing/malformed evidence from absence.
  • Add paired --from/--to, day/week lookbacks, and schema-supported --scope. Return next_request.args preserving effective fields, time bounds, scopes, sorting, matching mode, org profile, and cursor. Credentials remain in the caller's environment.
  • Add opt-in --max-results (1–10,000), with a smaller final request to fit the budget, explicit stop reasons, and failures for repeated cursors, empty nonterminal pages, oversized responses, or later request failures. Default queries still fetch one page; automatic paging conflicts with --raw.
  • Correct bare-text matching and native GitHub discovery guidance. Update help and the bundled skill, with real-binary tests for requests, outputs, continuation replay, and negative cases, including projections, scoped windows, and field rejection through list/search.

These controls share query preparation and continuation; keeping them together ensures a continued inventory uses the same projections and measurement context.

Validation

  • Full serial suite: 2,030 unit tests and 15 real-binary integration tests passed.
  • cargo clippy --locked --all-targets -- -D warnings, cargo fmt --check, git diff --check, and skill validation passed.
  • The live read-only CLI matrix covered 69 scenarios across this work and the companion read/summary changes: 67 passed; two exposed provider/access limitations. Targeted checks verified recovery paths. Query checks included multiple kinds, traversal, edge fields, inventory/total agreement, manual and bounded continuation executed from returned argv, time/scope controls, output formats, raw/agent/jq behavior, and input rejection.
  • Known provider limitations are documented: runtime edges remain org-wide; filtered fuzzy matching can miss known entities; native GitHub kinds need separate provider authorization. An absolute window + property scope + timeseries attribute + runtime expansion failed upstream; separate reads with the same window were verified. The CLI does not drop filters or change windows to hide these failures.
  • Rebuilt this independent PR and checked it live: related-team projection, populated runtime-edge measurements, bounded paging with returned argv executed unchanged, and an absolute measurement window with an environment property scope.
  • cargo audit reports four existing findings in the unchanged lockfile: h2 (RUSTSEC-2026-0258), rustls (RUSTSEC-2026-0285), and two rsa versions (RUSTSEC-2023-0071). No dependencies changed. Tests ran serially due to existing shared-environment races elsewhere.

Companion PRs

These PRs each build independently against main; their changes also merge cleanly together. The combined tree passed 2,032 unit tests, all 20 real-binary tests, Clippy, formatting, and skill validation.

@datadog-prod-us1-4

This comment has been minimized.

Reserve an 8 MiB stack for the MSVC pup executable. The debug command
router exceeds the default 1 MiB reserve, so real-binary integration tests
were aborting before reaching any command.

Add startup success/error smoke tests and remove colons from fixture
config directory names so they are valid on Windows. Keep all CI tests
enabled and preserve the existing build and test profiles.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant