Skip to content

Document OpenTelemetry plugin span caveats for Workflow Task failures and resets - #5720

Merged
Duncanma merged 2 commits into
mainfrom
duncan/python-otel-replay-caveats
Nov 13, 2026
Merged

Duncanma merged 2 commits into
mainfrom
duncan/python-otel-replay-caveats

Conversation

@Duncanma

Copy link
Copy Markdown
Contributor

The Python observability page says Worker restarts and cache evictions never produce duplicate spans under OpenTelemetryPlugin. That holds, but two cases weren't covered:

  • Failed Workflow Tasks. Spans that ended inside the failed task are exported again with the same span identifiers. Whether they show twice depends on whether the backend deduplicates by span identifier.
  • Resets that reapply an Update or Signal. Before temporalio 1.36.0, the reapplied handler's span reused the original run's RunWorkflow span identifier. Fixed by Enable single-batch workflow activation processing by default sdk-python#2081.

Both came up while reviewing the Arize integration page in #5306, which documents them for Arize specifically. They're plugin behavior, so this states them on the general tracing page. Kept separate from #5306 so that PR stays scoped to Arize.

CI Vale config and yarn build are clean.

… and resets

The replay paragraph said Worker restarts and cache evictions never
produce duplicate spans, which holds, but didn't cover the two cases
that do re-export or collide: spans ended inside a failed Workflow Task
are exported again with the same IDs, and before temporalio 1.36.0 a
reapplied Update or Signal handler after a reset reused the original
RunWorkflow span's ID (temporalio/sdk-python#2081).
@Duncanma
Duncanma requested a review from a team as a code owner November 13, 2026 18:59
Copilot AI balanced review requested due to automatic review settings November 13, 2026 18:59
@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
temporal-documentation Ready Ready Preview Nov 13, 2026 8:39pm UTC

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📖 Docs PR preview links

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The OpenAI Agents documentation still claims spans export only when a Workflow completes, contradicting the caveat that ended spans can export during a failed Workflow Task.

1 open issue
What changed in this PR

This PR adds plugin-wide tracing caveats to the Python observability documentation, complementing the Arize-specific guidance in #5306.

Changes:

  • Explains that failed Workflow Tasks can re-export ended spans, with duplicate visibility depending on backend deduplication.
  • Documents the temporalio 1.36.0 requirement to avoid span identifier reuse when resets reapply Updates or Signals.
File Description
docs/​develop/​python/​platform/​observability.mdx Adds failed-task and reset caveats for OpenTelemetryPlugin.

Review effort: Balanced

Comment on lines +199 to +201
A failed Workflow Task is the exception. The Worker re-executes the Workflow code since the last completed Workflow
Task, and spans that already ended inside the failed task are exported again with the same span identifiers. Whether
they appear twice depends on the backend: one that deduplicates by span identifier keeps a single copy.
@Duncanma
Duncanma merged commit 13362a1 into main Nov 13, 2026
18 of 19 checks passed
@Duncanma
Duncanma deleted the duncan/python-otel-replay-caveats branch November 13, 2026 20:38

This branch was successfully deployed

1 active deployment
Preview — 3986e66e Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants