Skip to content

Decouple metric fetch errors from trial status in Orchestrator - #5287

Open
saitcakmak wants to merge 1 commit into
facebook:mainfrom
saitcakmak:export-D119207750
Open

saitcakmak wants to merge 1 commit into
facebook:mainfrom
saitcakmak:export-D119207750

Conversation

@saitcakmak

Copy link
Copy Markdown
Contributor

Summary:
Keep trial execution status authoritative to runner polling when metric fetching fails. Orchestrator now leaves completed trials completed, uses MetricAvailability to count incomplete optimization-config data in the existing failure-rate policy, and handles expanded multi-metric result keys without assuming they are registered experiment metrics.

This mirrors the behavior core Ax already has in the Orchestrator.

This implements the core design from the abandoned RFC D98741656 and fixes T287759832.

Differential Revision: D119207750

Summary:
Keep trial execution status authoritative to runner polling when metric fetching fails. Orchestrator now leaves completed trials completed, uses MetricAvailability to count incomplete optimization-config data in the existing failure-rate policy, and handles expanded multi-metric result keys without assuming they are registered experiment metrics.

This mirrors the behavior core Ax already has in the Orchestrator.

This implements the core design from the abandoned RFC D98741656 and fixes T287759832.

Differential Revision: D119207750
@meta-cla meta-cla Bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Sep 9, 2026
@meta-codesync

meta-codesync Bot commented Sep 9, 2026

Copy link
Copy Markdown

@saitcakmak has exported this pull request. If you are a Meta employee, you can view the originating Diff in D119207750.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.59%. Comparing base (02c0757) to head (58bd88b).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5287      +/-   ##
==========================================
+ Coverage   96.57%   96.59%   +0.01%     
==========================================
  Files         621      621              
  Lines       70912    70911       -1     
==========================================
+ Hits        68481    68494      +13     
+ Misses       2431     2417      -14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants