[jongio]

Summary

Updates the optional eval (github/evals, packages/plugin) remote plugin pin and companion documentation/eval-environment references from copilot-evals-v0.24.6 (5c3d5213f88149fe1e5476ef029277fe56fd42e4) to copilot-evals-v0.24.14 (bfae69e479308aaf1320dc2b2bac4315659a006a).

Upstream eval Remote Pin Review (update-remote-pins)

• Old pin: copilot-evals-v0.24.6 @ 5c3d5213f88149fe1e5476ef029277fe56fd42e4 (version: "0.24.6")
• New pin: copilot-evals-v0.24.14 @ bfae69e479308aaf1320dc2b2bac4315659a006a (version: "0.24.14", packages/plugin/plugin.json)
• Synchronized files: marketplace.json, .vally.yaml, README.md, docs/checks-and-evals.md, and .github/scripts/backend-defaults.test.mjs
• Upstream changes reviewed (5c3d5213...bfae69e4):
  • Isolates draft_eval vs copilotEvals_* authoring tool families across create and improve skills and updates sandbox runner-setup guidance across create, improve, and run.
  • Adds guarded YAML alias resolution and reference unsharing in the MCP/runtime bundle (dist/mcp/server.mjs), and updates Canvas UI primitives, tag layout, and grouped queued-eval execution (web/app.mjs, workbench.mjs).
  • mcp.json, copilot-extension.json, LICENSE (GitHub Limited Use Source License), hooks (none), network destinations, and public repository access (github/evals is public; no private-repository token required) are unchanged.

Repo Health Score: 97/100 (A+) -> 97/100 (A+)

Verification

• node .github/scripts/check-local-catalog.mjs
• node .github/scripts/check-remote-pins.mjs
• VALLY_TELEMETRY_OPTOUT=1 npm run lint
• node --test .github/scripts/*.test.mjs

[Copilot]

🔵 Needs a closer look

The remote plugin update includes extensive upstream runtime, native binary, UI, telemetry, and permission changes that automated synchronization checks cannot validate for compatibility, security, and operational impact.

No open issues 🎉

What changed in this PR

Updates the optional remote eval plugin from Copilot Evals 0.24.6 to 0.24.14, keeping the marketplace pin, evaluation environment, documentation, and regression-test references synchronized.

Changes:

• Updates the remote plugin ref, commit SHA, and version in marketplace.json.
• Updates the Vally checkout command and backend contract test reference.
• Refreshes README and evaluation documentation links to the new release and commit.

File	Description
README.md	Updates Evals release, documentation, setup, and license references.
marketplace.json	Pins the eval plugin to version 0.24.14 at the new commit SHA.
docs/​checks-and-evals.md	Documents the updated Evals version and pinned project-reader reference.
.vally.yaml	Updates the evaluation environment checkout ref and pinned commit.
.github/​scripts/​backend-defaults.test.mjs	Updates the documented backend contract version reference.

_{Review effort: Lite (dynamic)}