Repository navigation
Make FailureDetails.Properties optional for DataContractSerializer - #1431
Bernd Verst (berndverst) merged 5 commits into
Conversation
FailureDetails is [Serializable] without ISerializable, so DataContractSerializer treats every field as required. Data persisted before the Properties property was added (e.g. Netherite commit logs and history) fails to deserialize with 'Expecting element _x003C_Properties_x003E_k__BackingField'. Mark the backing field [OptionalField] so older payloads deserialize with Properties = null. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4bf274ab-99ca-4766-831e-75f351459122
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
🟡 Changes recommended
The regression tests are under uppercase Test/ instead of the referenced lowercase test/ directory, so they are not compiled or discovered on case-sensitive checkouts.
1 open issue
What changed in this PR
This PR restores compatibility with older FailureDetails payloads used by DataContractSerializer-based backends, without changing the serialized shape.
Changes:
- Marks the
Propertiesbacking field optional and documents why. - Adds tests for legacy nested failures and round-tripping populated properties.
| File | Description |
|---|---|
Test/DurableTask.Core.Tests/FailureDetailsSerializationTests.cs |
Adds legacy-deserialization and properties round-trip tests. |
src/DurableTask.Core/FailureDetails.cs |
Allows deserialization when the Properties field is absent. |
Review effort: Balanced
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4bf274ab-99ca-4766-831e-75f351459122
|
@microsoft-github-policy-service rerun . |
|
@microsoft-github-policy-service agree company="Microsoft" |
|
@microsoft-github-policy-service rerun |
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
Copilot encountered an error: Your billing is not configured or you have Copilot licenses from multiple standalone organizations or enterprises. To use premium requests, select a billing entity via the GitHub site, under Settings > Copilot > Features.
|
Davide Montanari (@davidemontanari) Bernd Verst (@berndverst) This resolves a breaking change from 2025 that means the netherite backend can't be upgraded. Could you have a look please? |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: de587369-3d13-49fa-8654-999bf350e3c4
Bernd Verst (berndverst)
left a comment
There was a problem hiding this comment.
I found no breaking change or blocking defect introduced by this PR at revision 3177e39. This is a narrowly scoped backward-compatibility fix, and I would consider it safe to merge.
The production change adds [field: OptionalField] to the existing backing field. It does not remove the field, stop serializing it, or change the public property.
Impact across the repository
| Area | Assessment |
|---|---|
| Core / downstream SDK consumers | Public signatures, constructors, field layout and assembly identity remain unchanged. No source or binary compatibility break identified. |
| Azure Storage — default mode | Newtonsoft.Json output and deserialization remain unchanged. The field attribute does not affect its normal property-based contract. |
| Azure Storage — DataContract mode | Its opt-in DataContractJsonSerializer path also benefits: missing historical Properties fields become acceptable. Existing output remains unchanged. |
| Service Bus | Normal JSON messages are unchanged. The legacy DCS reader becomes more backward-compatible. |
| Service Fabric | JSON proxy contracts remain unchanged; DCS-based persisted object graphs receive the same compatibility improvement. |
| Emulator, ApplicationInsights, samples and solutions | No API adaptations, dependency changes, target-framework changes or project-reference changes are required. |
The same conclusion applies to activity failures, sub-orchestration failures, orchestration completion records and orchestration state: they all reference the same FailureDetails type. Nested failures benefit as well.
Actual side effects and caveats
- Missing data is now accepted intentionally. Historical payloads deserialize with
Properties = nullinstead of throwing. This is consistent with the property's existing nullable contract. Other required fields remain required, and retry/non-retriable behavior is unchanged. - Exported XML schema changes slightly. The backing-field element becomes optional (
minOccurs="0"). Schema-generation or schema-diff tooling can observe this, but serialized XML and JSON remain unchanged—including null, empty and populated dictionaries. - Rolling upgrades remain asymmetric for historical data. Patched output remains readable by the pre-PR version. However, an unpatched worker still cannot read historical records missing
Properties; this PR neither rewrites those records nor fixes readers that have not been upgraded. - This does not certify every older-Core-to-latest upgrade. Unrelated version changes, custom serializers and arbitrary objects stored inside
Propertiesretain their existing compatibility constraints.
Bottom line: this relaxes reading without changing writing. I found no repo SDK, project or solution that needs a corresponding change because of this PR.
… fix (#1436) * Prepare compatibility patch releases Bump Core and five companion packages to ship the FailureDetails deserialization compatibility fix from #1431. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> * Exclude Service Fabric and Emulator from patch release Limit coordinated patch releases to Core, Azure Storage, Service Bus, and Application Insights. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Bernd Verst <beverst@microsoft.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Problem
FailureDetailsis a[Serializable]type that does not implementISerializable.DataContractSerializertherefore treats every field as required. #1237 added thePropertiesauto-property, so payloads persisted before that change no longer deserialize. This affects Netherite commit logs and instance history, for example.This blocks upgrading apps that use a
DataContractSerializer-based backend (for example Netherite) past DurableTask.Core 3.4.0 while existing task hubs contain failed activities, sub-orchestrations or orchestrations. NewerMicrosoft.Azure.Functions.Worker.Extensions.DurableTask.AzureManagedpackages require Core 3.8.0+, so side-by-side DTS + Netherite hosts hit this when upgrading.Fix
Mark the
Propertiesbacking field[field: OptionalField].DataContractSerializerthen deserializes older payloads withProperties = null, and newer payloads still round-trip.ISerializableinstead was avoided because it changes the DCS wire format.[JsonConstructor]already defaults a missingpropertiesto null.I diffed the DCS-relevant types between Core 3.3.0 and 3.11.2. This was the only newly required member. The other additions are optional
[DataMember]s or new rewind-only types and enum values.Tests
Added
FailureDetailsSerializationTests:TaskFailedEventwith nestedFailureDetailsand thePropertieselements removed deserializes. Without the fix this fails with the error above.FailureDetailswithPropertiesround-trips throughDataContractSerializer.Passes on net8.0 and net48 together with the existing
CustomExceptions/ExceptionHandlingtests.