Skip to content

Add the Arize integration page for Python - #5306

Merged
Duncanma merged 18 commits into
mainfrom
dabh/arize-python-integration-docs
Nov 13, 2026
Merged

Duncanma merged 18 commits into
mainfrom
dabh/arize-python-integration-docs

Conversation

@DABH

@DABH DABH commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Adds docs/develop/python/integrations/arize.mdx, a guide to sending Temporal traces to Arize Phoenix or Arize AX with the Python OpenTelemetry plugin and the OpenInference conventions, plus the sidebar entry, the IntegrationsGrid card, the Python index bullet, and a pointer from the observability page. Vale gains Arize, OpenAI, OpenInference, OTLP, and Phoenix as heading exceptions.

The page covers the tracer provider and exporter setup, Client and Worker configuration, LLM calls in Activities (including the OpenAI Agents SDK bridge from the standalone temporalio-openai-agents package), the OpenInference attributes that make traces group by Workflow Id, and how replay, retries, Worker crashes, Workflow Task failures, Continue-As-New, and resets appear in Arize. Every behavior described was verified against a local Phoenix with the sample's runbook; the Arize AX path was validated by Arize against an AX space.

The sample merged as temporalio/samples-python#365, so the snippets are synced from main and the GitHub links resolve. Replaces the closed #5210 card. Vale CI config and yarn build are clean; yarn check:orphans reports two pre-existing pages unrelated to this change.

@vercel

vercel Bot commented Sep 11, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
temporal-documentation Ready Ready Preview Nov 13, 2026 6:57pm UTC

Request Review

@github-actions

github-actions Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

📖 Docs PR preview links

@arizedatngo

Copy link
Copy Markdown

@DABH following up on the Arize integration handoff: this docs PR and samples-python#365 are still drafts, and the docs body notes the sample must merge first. Is there any specific input needed from the Arize side to finish the sample/docs review? For the eventual evaluation references, the current URLs are https://arize.com/guides/ai-agent-handbook/agent-evaluation/ and https://arize.com/resources/llm-evaluation/. Happy to keep tracking the dependency rather than reopening #5210.

Base automatically changed from dabh/python-otel-plugin-observability to main September 15, 2026 16:10
@DABH

DABH commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Hey Dat, sorry for the slow reply here! Nothing is blocked on the Arize side - both PRs have been drafts since I found a few tracing bugs in the OpenAI Agents plugin while building the agents scenario, and in the meantime that plugin moved out of the SDK into the standalone temporalio-openai-agents package, so I'm migrating the scenario and this page over to it and then both go up for review. The one tracing fix the agents scenario needs (temporalio/ai-integrations#26) is merged but not in a release yet, so the split-worker runbook stays until it ships. Two asks if you have the time: once the preview refreshes, could someone on your side read the "Read Temporal traces in Arize" section and tell me if anything about how Phoenix renders retries or resets looks off? And once this page is live, could your Temporal guide link to it (it points at our observability page today)? I'll add the two eval links you sent under Resources. Thanks in advance!

…egration-docs

# Conflicts:
#	docs/develop/python/platform/observability.mdx
Install the agents variant from the standalone package, describe the
single-worker span-parenting bug as fixed upstream but unreleased, note
that the Arize AX exporter settings were not exercised live, and add
Arize's evaluation guides to Resources.
1.1.0 ships the span-parenting fix, so the caution about running Workflows
and Activities in separate Worker processes and the note about the
"Failed to detach context" log filter no longer apply. The install step
now asks for 1.1.0 or later, and the agents plugin snippet matches the
sample without its worker-role switch.
@justonwestern

Copy link
Copy Markdown

Ran the sample against an Arize AX space (US region) on 2026-10-23, from commit 22922af, with temporalio 1.34.0, temporalio-openai-agents 1.1.0 and Python 3.12.

Everything except reset matches the "Read Temporal traces in Arize" section:

  • Ticket triage: the tree matches the docs node for node. Temporal spans are CHAIN, LLM spans show model, messages and tokens, and the root has input, output, user, tags and metadata. user.id from baggage reaches the worker-side spans. AX also shows cost per trace and per session.

  • Sessions: reusing a Workflow Id groups both traces in one session, and the Sessions view fills first and last message from the root span.

  • Agents SDK: AGENT, LLM and TOOL spans render as in the README.

  • Replay stress and worker restart: one clean trace each, no duplicates. After the restart, RunWorkflow covers the outage (26.3s).

  • Activity retry: two RunActivity:classify_ticket spans, the first with the exception event and attempt 1, the second with attempt 2.

One problem with reset. After temporal workflow reset --type FirstWorkflowTask, the reset run's reapplied HandleUpdate:approve span has the same span ID as the original run's RunWorkflow span (4d781b8a6128224e in our trace). AX keeps both spans, so the original run's activity subtree shows up twice, once under each span with that ID, and the original RunWorkflow node is labeled HandleUpdate:approve in the tree. The new run's RunWorkflow is correct: a second child of StartWorkflow with its own temporalRunID. Phoenix deduplicates by span ID and keeps the first copy, so it would drop the new HandleUpdate span silently, which may be why this didn't show up in Phoenix testing. It looks like the deterministic ID generator in the reset run is reusing IDs from the original run. Happy to share the trace export.

After reset: the original run’s activities appear under two different spans.

Screenshot A

Clicking the HandleUpdate:approve node opens the original RunWorkflow span (26.3s, original run ID). AX highlights both HandleUpdate:approve nodes because they share a span ID.

Screenshot B

Two smaller notes:

  1. A retried-then-successful activity marks the whole trace as ERROR in AX, even though the workflow completed. A sentence in the retries section would keep people who filter on errors from being surprised.

  2. Stopping the worker while a workflow waits logs ERROR:opentelemetry.context:Failed to detach context with a stack trace, then Shutdown called, ignoring Span. The trace comes out fine, but a README note that this is expected would save people some worry when they run the restart case.

With the reset fix, the line saying the AX path wasn't exercised can come out.

DABH added 2 commits November 2, 2026 23:22
…egration-docs

# Conflicts:
#	docs/develop/python/index.mdx
#	sidebars.js
#	vale/styles/Temporal/Headings.yml
Arize validated the sample against an AX space, so the note that the AX
path was not exercised is gone. Add what they observed: AX lists a trace
with a failed attempt as an error, a stopping Worker logs a harmless
"Failed to detach context" traceback, and after a reset the reapplied
update's HandleUpdate span reuses the original RunWorkflow span id, which
Phoenix drops and AX shows as a duplicated subtree.
@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Snipsync coverage

SDK code lines in docs/ that come from Snipsync: 57.8% → 58.2% (+0.4 pts), 23,609 of 40,591 lines.

Section Snipsync lines Inline lines Snipsync % Change
develop/python 4,821 → 5,146 535 90.0% → 90.6% +0.6 pts
All sections

All languages: 50.1% (24,434 of 48,782 lines).

Section Snipsync lines Inline lines Snipsync %
design-patterns 26 7,808 0.3%
develop/python 5,146 535 90.6%
develop/java 4,639 49 99.0%
guides 34 4,494 0.8%
develop/go 4,285 80 98.2%
develop/typescript 3,702 314 92.2%
develop/dotnet 2,285 0 100.0%
develop/rust 1,571 0 100.0%
develop/ruby 1,232 0 100.0%
develop/php 30 1,144 2.6%
develop 484 606 44.4%
production-deployment 116 746 13.5%
encyclopedia 0 837 0.0%
develop/worker-performance 48 177 21.3%
cloud 0 127 0.0%
references 0 48 0.0%
best-practices 11 17 39.3%
Language Snipsync lines Inline lines Snipsync %
Python 5,289 6,615 44.4%
Go 4,425 3,181 58.2%
Java 4,724 2,593 64.6%
TypeScript 3,869 2,778 58.2%
C# 2,360 457 83.8%
Rust 1,609 31 98.1%
Ruby 1,303 159 89.1%
PHP 30 1,168 2.5%
Snipsync origin Lines
temporalio/documentation-sdk-code-examples 19,577
temporalio/samples-python 2,266
temporalio/samples-typescript 693
temporalio/features 464
temporalio/samples-go 384
temporalio/samples-java 184
temporalio/samples-php 30
temporalio/reference-app-orders-go 11

Compared with 107025d. Counts non-blank lines in fenced code blocks tagged with an SDK language. Hand-written code isn't compiled anywhere; code Snipsync pulls from a sample repository is built in that repository's CI. See UTILITIES.md.

@DABH

DABH commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for running this against AX, Juston - that's exactly the coverage we were missing, and the reset finding is a real bug. I reproduced it locally with an in-memory exporter: the reapplied HandleUpdate:approve gets the first span ID the original run drew (its RunWorkflow), because the reset run starts on the original run's random seed and the reapplied update handler runs before the SDK applies the reseed for the new run. Everything after that gets fresh IDs, which is why only that one span collides and why Phoenix hid it (it drops the duplicate). Filed as temporalio/sdk-python#2070; until it lands, the README and this page describe the behavior and verify_trace.py warns that the reapplied update span is missing in Phoenix rather than passing quietly. Also took your two notes: the retries section now says AX lists a trace with a failed attempt as an error even though the workflow completed, and the README's restart step explains the "Failed to detach context" traceback and "Shutdown called, ignoring Span" lines at worker shutdown. The "not exercised against AX" line is gone. If you can share the trace export I'd love to attach it to the SDK issue

@arizedatngo

Copy link
Copy Markdown

@DABH, one update while tracking the paired docs/sample PRs: SDK issue #2070 is closed, and the single-batch fix in temporalio/sdk-python#2081 merged November 3. The earlier run-ID proposal #2071 closed without merge. The latest published Python SDK release I can see is still 1.35.0 (October 28), which predates #2081, so I am not treating the fix as released or the sample as revalidated. Could you confirm the intended SDK version/workaround wording and review route for this docs PR and samples-python#365?

The one-line suggestion above also points the existing agent-evaluation resource directly to its current URL; the old URL still redirects. No partner branch was changed, and no new authenticated AX/Phoenix run is claimed.

AI-assisted status check and follow-up by OpenAI Codex at Dat’s request.

DABH added 2 commits November 12, 2026 12:23
Require 1.36.0 in the install step, replace the known-defect paragraph
with the fixed-version note, and match the worker snippet to the sample.
DABH added 2 commits November 13, 2026 12:18
temporalio/samples-python#365 merged, so the five snippets on the Arize
page now come from main instead of the pre-filled copies. The sample's
condensing pass shortened docstrings and comments; the prose around
each snippet still matches.
@DABH
DABH marked this pull request as ready for review November 13, 2026 18:26
@DABH
DABH requested a review from a team as a code owner November 13, 2026 18:26
Copilot AI balanced review requested due to automatic review settings November 13, 2026 18:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Dat suggested the current address on the PR; the old one still redirects.
@DABH

DABH commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the check. Status as of today: temporalio 1.36.0 shipped on November 11 with the single-batch fix from sdk-python#2081, the sample merged this morning as samples-python#365 on 1.36.0, and this page now requires 1.36.0 and no longer describes a reset workaround. The PR is out of draft and with the Docs team for review. I also took your link suggestion

- Use Workflow ID, drop the validation process note, and shorten the
  reset paragraph to the version requirement and its symptom
- Move the Agents bridge Public Preview note into a caution at the top
  of its section
- Tie the lost-spans window to schedule_delay_millis instead of the
  sample's 500 ms
- Rename the heading to Workflow retries so it doesn't read as Activity
  retries
- Require temporalio-openai-agents 1.1.0 on the OpenAI Agents
  OpenTelemetry install line, with the span parenting warning

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the Arize integration guide and related documentation updates.

No open issues 🎉

Review effort: Balanced

@Duncanma
Duncanma merged commit 36d83d2 into main Nov 13, 2026
20 checks passed
@Duncanma
Duncanma deleted the dabh/arize-python-integration-docs branch November 13, 2026 19:10

This branch was successfully deployed

1 active deployment
Preview — a3309f70 Deployed Nov 13, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants