Avoid Microsoft.Bcl.Memory on modern .NET targets

[matouskozak]

Summary

Remove Microsoft.Bcl.Memory from modern .NET dependency graphs while keeping .NET Standard 2.0 support.

• Add net8.0 targets to the five Tokenizers.Data packages, TorchSharp, AutoML, CodeGenerator, and Fairlearn.
• Keep TorchSharp's Microsoft.Bcl.Memory reference only for netstandard2.0, and update the shared version to the patched 9.0.14.
• Add an explicit System.Text.Json reference to AutoML. AutoML and its bundled SearchSpace assembly still need this dependency after it stops arriving through Tokenizers.

Fixes #7589. Contributes to #7590; this change addresses Microsoft.Bcl.Memory, not every Microsoft.Bcl.* package.

Validation

• Built and packed the affected packages for netstandard2.0 and net8.0, with package validation enabled.
• Completed a full NuGet consumer restore for netstandard2.0, net8.0, and net10.0 with zero warnings or errors. The modern graphs contain no Microsoft.Bcl.Memory; the legacy graph selects 9.0.14.
• Confirmed that all existing public project dependencies remain in the package manifests, including all four AutoML image-model packages. The ten restored local package archives match the rebuilt archives.
• Passed five existing tokenizer regression tests and tokenizer/AutoML JSON smoke tests on macOS ARM64 with .NET 8 and .NET 10.

Existing published package versions are unchanged. A new release is required to deliver the fix to consumers.

[Copilot]

🟢 Approval recommended

The TFM additions and conditional dependency updates are consistent with central package management and are scoped to preserve netstandard2.0 while removing Microsoft.Bcl.Memory from modern targets.

Pull request overview

This PR updates packaging/TFM targeting to keep .NET Standard 2.0 compatibility while avoiding Microsoft.Bcl.Memory in modern dependency graphs by adding net8.0 targets and scoping the Microsoft.Bcl.Memory reference to legacy TFMs only.

Changes:

• Multi-target several packages/projects with netstandard2.0;net8.0 to produce modern TFMs and reduce legacy Microsoft.Bcl.* dependencies for modern consumers.
• Restrict Microsoft.Bcl.Memory usage in Microsoft.ML.TorchSharp to netstandard2.0 only, and bump the centrally-managed version to 9.0.14.
• Add an explicit System.Text.Json package reference to Microsoft.ML.AutoML to keep the dependency explicit after dependency-graph reshaping.

File summaries

File	Description
src/Microsoft.ML.TorchSharp/Microsoft.ML.TorchSharp.csproj	Adds net8.0 TFM and conditions Microsoft.Bcl.Memory to netstandard2.0 only.
src/Microsoft.ML.Tokenizers.Data.R50kBase/Microsoft.ML.Tokenizers.Data.R50kBase.csproj	Adds net8.0 targeting to remove legacy-only dependency impact for modern consumers.
src/Microsoft.ML.Tokenizers.Data.P50kBase/Microsoft.ML.Tokenizers.Data.P50kBase.csproj	Adds net8.0 targeting for modern dependency graphs.
src/Microsoft.ML.Tokenizers.Data.O200kBase/Microsoft.ML.Tokenizers.Data.O200kBase.csproj	Adds net8.0 targeting for modern dependency graphs.
src/Microsoft.ML.Tokenizers.Data.Gpt2/Microsoft.ML.Tokenizers.Data.Gpt2.csproj	Adds net8.0 targeting for modern dependency graphs.
src/Microsoft.ML.Tokenizers.Data.Cl100kBase/Microsoft.ML.Tokenizers.Data.Cl100kBase.csproj	Adds net8.0 targeting for modern dependency graphs.
src/Microsoft.ML.Fairlearn/Microsoft.ML.Fairlearn.csproj	Adds net8.0 targeting to reduce legacy-only dependency impact for modern consumers.
src/Microsoft.ML.CodeGenerator/Microsoft.ML.CodeGenerator.csproj	Adds net8.0 targeting to avoid legacy-only dependency impact for modern consumers.
src/Microsoft.ML.AutoML/Microsoft.ML.AutoML.csproj	Adds net8.0 targeting and explicitly references System.Text.Json.
eng/Versions.props	Updates the centrally-managed Microsoft.Bcl.Memory version to 9.0.14.

Review details

• Files reviewed: 10/10 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.89%. Comparing base (2fc13ba) to head (acde728).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7703   +/-   ##
=======================================
  Coverage   69.88%   69.89%           
=======================================
  Files        1487     1487           
  Lines      276253   276253           
  Branches    28291    28291           
=======================================
+ Hits       193053   193077   +24     
+ Misses      75710    75685   -25     
- Partials     7490     7491    +1     

Flag	Coverage Δ
Debug	69.89% <ø> (+<0.01%)	⬆️
production	64.07% <ø> (+0.01%)	⬆️
test	89.82% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.