Skip to content

Surface WCI limit errors as ResourceExhausted instead of Internal - #13164

Open
zainawaisn wants to merge 2 commits into
mainfrom
zainawaisn/wci-limit-error-exhausted
Open

zainawaisn wants to merge 2 commits into
mainfrom
zainawaisn/wci-limit-error-exhausted

Conversation

@zainawaisn

@zainawaisn zainawaisn commented Nov 19, 2026 •

Copy link
Copy Markdown

What changed?

  • When a namespace hits the maximum number of worker controller instances, the error now reaches the customer as ResourceExhausted with a clear message, instead of a generic "something went wrong".
  • This limit error is no longer retried, since retrying can't succeed. Temporary throttling errors are still retried as before.
  • Hitting this limit now records the existing WorkerDeploymentLimitExceeded metric, so we can see which namespaces are affected.
  • FailedPrecondition errors from WCI on the compute config update API also come back with their real error code now. They were returned as Internal before.

Why?

The WCI instance limit error was retried 5 times, then returned as Internal. Users only see something went wrong, please retry. They now get ResourceExhausted: reached maximum worker controller instances in namespace (n).

Open bug ticket: https://temporalio.atlassian.net/jira/software/c/projects/COM/boards/1520?selectedIssue=COM-375

How did you test it?

  • built
  • run locally and tested manually
  • covered by existing tests
  • added new unit test(s)
  • added new functional test(s)

Added TestWciClientErrorToActivityError unit test. The client.go mappings are not tested directly.

Potential risks

Low. This only affects errors that were previously returned as Internal.

@zainawaisn
zainawaisn requested review from a team as code owners November 19, 2026 07:20
@CLAassistant

CLAassistant commented Nov 19, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants