Python: count async FunctionTool invocation exceptions

[CoralGarden52]

Motivation & Context

FunctionTool.max_invocation_exceptions is documented as the maximum number of exceptions allowed during tool invocations. Before this change, exceptions raised while awaiting an async result were not counted consistently.

For native async tools, FunctionTool.__call__ returned a coroutine successfully and the exception was raised only when that coroutine was awaited. The same gap affected synchronous functions that return an awaitable. Direct async calls also bypassed the tracked await path.

This allowed a failing async tool to continue running after its configured exception budget was exhausted. The separate max_consecutive_errors_per_request setting is request-scoped and does not replace the per-tool lifetime limit.

Description & Review Guide

• What are the major changes?
  • Count exceptions raised at the await boundary through a shared _await_invocation_result helper.
  • Route direct async FunctionTool calls through the same tracked await path.
  • Preserve the existing synchronous exception-counting behavior without double-counting failures.
  • Add regression coverage for native async invocation, direct async calls, and synchronous wrappers returning awaitables.
• What is the impact of these changes?
  • max_invocation_exceptions is now enforced consistently across all supported async execution paths.
  • After the configured number of failures, the next call raises ToolException without invoking the underlying function again.
  • Request-level max_consecutive_errors_per_request behavior is unchanged.
• What do you want reviewers to focus on?
  • Verify that each failure is counted exactly once across __call__, invoke(), and the asyncio.to_thread path.
  • Verify that the second call is rejected and both invocation counters remain unchanged after the limit is reached.

Reproduction before this change

Ran the same public tool paths against the upstream baseline c37de519b with max_invocation_exceptions=1:

native-invoke [(1, 'RuntimeError', 1, 0), (2, 'RuntimeError', 2, 0), (3, 'RuntimeError', 3, 0)]
native-direct [(1, 'RuntimeError', 1, 0), (2, 'RuntimeError', 2, 0), (3, 'RuntimeError', 3, 0)]
sync-awaitable-invoke [(1, 'RuntimeError', 1, 0), (2, 'RuntimeError', 2, 0), (3, 'RuntimeError', 3, 0)]

The final tuple fields are invocation_count and invocation_exception_count. The async failures were raised, but the exception counter remained zero and the invocation count continued to increase.

Verification after this change

Ran the identical reproduction against the current PR head 87afcbac8:

native-invoke [(1, 'RuntimeError', 1, 1), (2, 'ToolException', 1, 1), (3, 'ToolException', 1, 1)]
native-direct [(1, 'RuntimeError', 1, 1), (2, 'ToolException', 1, 1), (3, 'ToolException', 1, 1)]
sync-awaitable-invoke [(1, 'RuntimeError', 1, 1), (2, 'ToolException', 1, 1), (3, 'ToolException', 1, 1)]

This confirms that the first awaited failure is counted, the configured limit is enforced on the second call, and no additional underlying invocation is started.

Tests and checks

• uv run pytest packages/core/tests -q -m 'not integration' — passed.
• uv run pytest packages/core/tests/core/test_tools.py::test_async_tool_exception_limit_counts_awaited_failures packages/core/tests/core/test_tools.py::test_direct_async_tool_exception_limit_counts_awaited_failures packages/core/tests/core/test_tools.py::test_sync_awaitable_tool_exception_limit_counts_awaited_failures packages/core/tests/core/test_tools.py -q -m 'not integration' — passed.
• uv run ruff format --check packages/core/agent_framework/_tools.py packages/core/tests/core/test_tools.py — 2 files already formatted.
• uv run ruff check packages/core/agent_framework/_tools.py packages/core/tests/core/test_tools.py — all checks passed.
• git diff --check — no whitespace errors.

The implementation is limited to async invocation exception accounting and its regression coverage.

Related Issue

Fixes #8277

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix) — a workflow keeps the label and title prefix in sync automatically.

[Copilot]

🟡 Changes recommended

Direct async calls remain untracked, and synchronous wrappers returning awaitables lack regression coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates FunctionTool to count exceptions raised while awaiting async results.

Changes:

• Enforces max_invocation_exceptions for awaited failures.
• Adds native async regression coverage.

File summaries

File	Summary
python/packages/core/agent_framework/_tools.py	Adds await-boundary exception counting; direct async calls still bypass tracking, and the awaitable-wrapper branch lacks coverage.
python/packages/core/tests/core/test_tools.py	Adds native async limit coverage; missing a synchronous-wrapper awaitable case.

Review details

Suppressed comments (2)

python/packages/core/agent_framework/_tools.py:637

• This only tracks failures reached through invoke()'s _invoke_function path. FunctionTool is also directly callable (the existing async test awaits the result of async_test_tool(...)), and that path returns the coroutine from __call__ without passing through this helper, so a failing async direct call still leaves invocation_exception_count unchanged and never enforces max_invocation_exceptions. Please route direct async calls through the same tracked await path, or explicitly narrow/document this limit as applying only to invoke() execution.

            return await self._await_invocation_result(res) if inspect.isawaitable(res) else res

python/packages/core/tests/core/test_tools.py:354

• This regression test only decorates an async def, so it exercises the inspect.iscoroutinefunction branch. The other supported case added here—a synchronous function returning an awaitable—runs through the asyncio.to_thread branch and has no regression assertion, so that path could regress while this test remains green. Please add a second max-limit case for the wrapper and verify the second call leaves both counters unchanged.

    @tool(name="failing_async_tool", max_invocation_exceptions=1)
    async def failing_async_tool() -> str:
        raise RuntimeError("boom")

    with pytest.raises(RuntimeError, match="boom"):

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Eduard van Valkenburg (eavanvalkenburg)]

Please use the defined PR template CoralGarden52

[CoralGarden52]

Hi Eduard van Valkenburg (@eavanvalkenburg) , apologies for missing the repository’s defined PR template in the initial PR description, and thank you for pointing this out and for reviewing the PR. I’ve now updated the PR description to use the defined PR template.