Skip to content

fix(utils): retain spans only for active usage scopes - #158

Draft
pengpengyi92 wants to merge 1 commit into
LLMQuant:masterfrom
pengpengyi92:fix/scoped-usage-span-retention
Draft

pengpengyi92 wants to merge 1 commit into
LLMQuant:masterfrom
pengpengyi92:fix/scoped-usage-span-retention

Conversation

@pengpengyi92

Copy link
Copy Markdown

Summary

After the first usage_scope registers the process-global collector, unrelated SDK traces are retained indefinitely. A span that finishes after its scope exits can also recreate an already-evicted bucket.

Collect spans only for explicitly active scope buckets. Opening, appending, and removing a bucket use the same lock; unrelated and late callbacks cannot recreate it. Preserve token/time aggregation and additive SDK processor registration. Add seven offline regression cases using an isolated real SDK provider, covering unrelated/late spans, nested and concurrent scopes, child tasks, exceptions, and cancellation.

Related Issue

Follow-up to #130. No existing issue was found for this collector-retention defect. This does not implement the batch accounting or budget work in #127 / #132.

Verification

  • bash scripts/verify.sh on Linux, Python 3.12.14, frozen-lock dependencies including Agents SDK 0.18.3: unchanged baseline 485 passed, 86.45% coverage; candidate 492 passed, 86.50% coverage. All five harness steps pass. The pre-push hook also verified the exact committed tree in a fresh directory with networking disconnected.
  • python -B -m pytest tests/utils/test_usage.py --no-cov -p no:cacheprovider -q on Windows, Python 3.12.13 / SDK 0.18.3: 20 passed. The two retention regressions fail against the original implementation before the fix.
  • Disposable real-SDK probe: 1,000 unrelated retained spans → 0, one late retained span → 0; completed-scope aggregation remains 15 tokens / one request.
  • pre-commit run --files quantmind/utils/usage.py tests/utils/test_usage.py, commit hooks, and git diff --check: pass.
  • No live-network smoke test applies: the changed helper observes local SDK tracing only. Regression tests use a separate provider with no exporter and make no model calls.
  • Platform limitation: the unchanged full Windows baseline has 21 failures / 464 passes in ETL filesystem decoding, SQLite cleanup, and PDF tests. The full candidate Windows suite was not rerun; its focused usage suite passes. Linux full-suite results above are the release evidence, not a claim of Windows-wide compatibility.

An active scope still retains its completed spans until exit. This fixes unowned/closed-scope retention, not a fixed memory limit or an RSS/throughput guarantee.

Suggested labels: type: bug, area: utils.

Checklist

  • The title uses English Conventional Commit format: type(scope): summary.
  • The related issue or design discussion is linked when applicable.
  • bash scripts/verify.sh passes.
  • Every applicable live-network component smoke test passes, or this PR states why none applies.
  • Public behavior has focused tests, an example, and documentation where applicable.
  • The PR is complete, small, and contains no unrelated changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant