diff --git a/.agents/ARCHITECTURE.md b/.agents/ARCHITECTURE.md index 3d4d24a..3dfa59f 100644 --- a/.agents/ARCHITECTURE.md +++ b/.agents/ARCHITECTURE.md @@ -9,9 +9,11 @@ Metadata/（源数据：XML、图片、Markdown、清单） Output/（发布树：展平后的 metadata.xml + 被引用资源） │ 上传 CDN / Cloudflare Pages / EdgeOne ▼ -客户端：MetadataBuilder.Load(url|path) 或 MetadataClient（带磁盘缓存） +客户端：MetadataReader.Read(url|path).Root 或 MetadataClient（带磁盘缓存） ``` +当前规则见 [集成计划](docs/exec-plans/2026-11-07-composable-metadata.md) / [ID 注册键修正](notes/implemented/bug-fix/2026-11-07-resource-id-keys.md)。只有 Reader；Flat/Tree/Mixed 仅改变组织，不保留 legacy publish、旧读取 API 或格式检测。完整 ID 原样匹配，冒号不构成错误；路径安全约束真实 Source 地址，不约束 ID 分隔符。 + --- ## 一、代码分层 @@ -20,22 +22,24 @@ Output/（发布树：展平后的 metadata.xml + 被引用资源） |---|---|---|---| | 数据契约层 | `Metadata/MetadataSchema.xsd`、`MetadataPublishSchema.xsd` | 约束作者写的源 XML 与最终发布物 | 构建器、编辑器 | | 模型与查询层 | `Ra3.BattleNet.Metadata/Metadata.cs`、`MetadataQueryExtensions.cs`、`MetadataQueryModels.cs`、`MetadataSchema.cs`、`LanguageTag.cs`、`MetadataColor.cs`、`MetadataResourceUri.cs`、`ModCompatibility.cs` | XML 文档的内存模型、LINQ 查询、语言比较、颜色转换、地址拼装 | 构建器、客户端 | -| 构建工具层 | `MetadataBuilder.cs`、`MetadataFlattener.cs`、`MetadataInheritance.cs`、`SchemaValidator.cs`、`VariableResolver.cs`、`MarkdownValidator.cs`、`ImagePostProcessor.cs`、`ImagingInvoker.cs`、`Program.cs` | 展平、继承合并、校验、资源拷贝、调用 Imaging | 只有命令行构建 | +| 读取与组合层 | `Reading/`：Reader、Id、DocumentInfo、SourceOptions、XML sources、SourceManifest | 单一读取、继承、引用解析、文档相对 Source 与捕获图；不写盘 | 构建、导出、缓存、客户端 | +| 导出层 | `Exporting/`：Exporter、Options、SourceExporter、Xml | 单格式 Flat/Tree/Mixed、资源布局和显式短 ID 作者写出 | 构建、工具、Desktop | +| 构建工具层 | `Building/`：Builder、Inheritance、SchemaValidator、VariableResolver、MarkdownValidator、ImagePostProcessor、ImagingInvoker；根 Program | schema/领域校验、变量和可选图片转换；不保留读取 API | 命令行构建 | | 缓存层 | `Cache/MetadataClient.cs`、`MetadataDiskCache.cs`、`IMetadataTransfer.cs`、`HttpMetadataTransfer.cs`、结果记录 | 联网刷新、最后有效缓存、图片复用 | 客户端宿主 | | 发版图片工具 | `Ra3.BattleNet.Metadata.Imaging/` | 把图片转成 WebP | 只有 `--webp` 构建 | -**依赖方向**：缓存层 → 模型与查询层；构建工具层 → 模型与查询层；模型与查询层不引用其他任何一层。`Ra3.BattleNet.Metadata` 是纯托管代码，不依赖 SkiaSharp 之类的图像库；图片转换只在发版时由独立项目执行。 +**依赖方向**：缓存获取 XML 后交给 Reading；Building 使用 Reading 和 Exporting，Exporting 使用捕获图；模型/query/core 不依赖这些层。父在 owner freeze 后按责任目录搬文件，命名空间不变。主库纯托管，不依赖 SkiaSharp；图片转换由独立项目执行。 --- ## 二、构建流水线 -`MetadataBuilder.Build(src, dst, schemaVersion, contentRevision, convertImages)` 按固定顺序执行，任何一步失败都会清空 `Output/` 后抛错，命令行进程返回 1： +`MetadataBuilder.Build(src, dst, contentRevision, convertImages)` 按固定顺序执行，任何一步失败都会清空 `Output/` 后抛错，命令行进程返回 1： 1. **准备**：定位 `Metadata/metadata.xml` 作为唯一入口，删除并重建 `Output/`；找不到入口或源 XSD 直接失败。 2. **源侧校验**：用 `MetadataSchema.xsd` 校验整个源目录；再对独立清单做语义检查（哈希算法、字段组合、`Skudef` 引用）。 -3. **展平**：展开 `Include`、按 `InheritFrom` 合并 `Base`、给资源短 ID 补上 `{路径前缀}:` 前缀、写入 `SchemaVersion` 与 `ContentRevision`。 -4. **拷贝资源**：按登记节点的 `Source` 把图片、Markdown、独立清单复制到输出目录。独立清单不是原样照抄，它同样走一遍展平与变量解析。 +3. **读取**：Reader 展开 Include、合并继承、将短资源 ID 限定为 `{scope}/{local}`；Source 相对定义文档解释，SourceRoot 只负责限定/containment。读取不覆盖 schema/revision。 +4. **导出**：普通 Export 写平面发布根和默认外部清单/资源，不追加任何版本或修订属性（`SchemaVersion` 取自源根声明）；不存在 ExportLegacy 或第二套资源拷贝。 5. **写盘与变量**：保存展平后的 `metadata.xml`，替换 `${TIMESTAMP}`、`${ENV:NAME}`，再检查是否残留 `${…}`。 6. **可选图片转换**：`--webp` 时调用 Imaging，把图片转成 WebP 并改写 `Source`。 7. **发布侧校验**：用 `MetadataPublishSchema.xsd` 校验产物，再做硬校验：ID 断链、资源缺失、显示名 `zh`/`en`、`Skudef` 与 `File` 的规则冲突、挂载角色、哈希格式等。 @@ -46,7 +50,8 @@ Output/（发布树：展平后的 metadata.xml + 被引用资源） ## 三、客户端读取路径 -- **一次性读取**：`MetadataBuilder.Load(url|path)` 把整棵树读进内存，支持 LINQ 查询；适合命令行、测试和小工具。 +- **一次性读取**：`MetadataReader.Read(url|path).Root` 是唯一文档读取入口。Builder.Load/LoadSource/Open、Flattener 和兼容 Detector 不保留；Parser 只投影 XML。 +- **源树与发布物同一条路**：入口 `metadata.xml` 上写了 `SchemaVersion` 就能直接读，不必先构建；`build`/`export` 只负责展平、聚合与解析变量（`Tags/Commit` 里的时间戳），供网络传输。 - **带缓存的常驻读取**：`MetadataClient` 管理根清单、独立清单、图片三类资源，磁盘布局固定： ``` @@ -58,7 +63,7 @@ Output/（发布树：展平后的 metadata.xml + 被引用资源） ``` - **下载边界**：`IMetadataTransfer` 只负责把正文写到调用方给的本地路径；校验、落盘、替换缓存都由库自己完成。默认实现是 `HttpMetadataTransfer`，宿主可注入自己的实现（代理、平台网络栈）。 -- **快照语义**：独立清单的身份只按调用时捕获的那份根快照解析，不会去翻更早或更晚的根。`GetCachedLeafAsync` 只读已有缓存、不发请求；`Refresh*` 才会联网。 +- **快照语义**：包清单只按捕获根快照和 PackageEntry 精确解析，不翻更早/更晚根，也不从 URI 猜 source-root。Get/GetCached/RefreshPackageAsync 保留；URI-only Get/GetCached/RefreshLeafAsync 及其匹配 glue 删除。外部 GetCachedPackageAsync 只读缓存，内联 body 继承快照状态；完整 normalized snapshot 与 raw XML/validators 分开保存。 具体 API 与字段含义见 [references/library-api.md](docs/references/library-api.md)。 @@ -68,10 +73,10 @@ Output/（发布树：展平后的 metadata.xml + 被引用资源） | 不变量 | 说明 | 谁在守 | |---|---|---| -| 发布物只有静态文件 | 根 `metadata.xml` 里不允许残留 `Include` | 发布 XSD + 构建校验 | -| 完整 ID 由构建器生成 | 作者只写短名；手写冒号前缀直接报错 | 构建校验 | -| 资源路径只来自 `Source` | 客户端不推导文件名，也不猜扩展名 | 解析库 | -| `SchemaVersion` 保持 `1.0` | 只有客户端无法兼容的破坏性改动才提高版本 | 构建器 + 评审 | +| 发布物只有静态文件 | Build 发布根没有 Include；Tree/Mixed export 保留明确 Include 组织 | 导出 + 发布校验 | +| 单一资源身份 | 短资源 ID 补 scope，已有含 `/` 或 `:` 的完整 ID 原样保留；引用优先精确匹配；业务 ID 不变 | Reading/MetadataId | +| 文档相对资源地址 | Source 相对定义文档，Root 中相对入口；SourceRoot 不提供回退；Image.Url 优先 | Reader + 路径安全 | +| 读取不伪造版本 | schema/revision 保留原声明，Build/export 显式设置；无 IdFormat/SourceBase markers | Reader + Exporter | | 构建失败不留半成品 | 任何校验失败都清空输出目录 | `MetadataBuilder.Build` | | 缓存只按发布树布局 | 地址映射是纯函数，同一路径不会被两个地址互相覆盖 | `MetadataDiskCache` + `.uri` | @@ -83,7 +88,7 @@ Output/（发布树：展平后的 metadata.xml + 被引用资源） Metadata/ 源数据（入口 metadata.xml，按 mods/ 与 apps/ 分目录） Ra3.BattleNet.Metadata/ 解析库 + 命令行构建工具 Ra3.BattleNet.Metadata.Imaging/ 发版用的 WebP 转换工具 -Ra3.BattleNet.Metadata.Tests/ MSTest 测试（构建、模型、缓存、兼容性） +Ra3.BattleNet.Metadata.Tests/ MSTest：Reading/、Exporting/、Building/、Cache/、Queries/；TestData 保持位置 scripts/verify-docs.mjs 知识库守卫（笔记格式、生成物新鲜度、链接） build.sh 线上构建入口，安装 .NET 后调用上面的命令行 .agents/ 全部研发文档与决策笔记 diff --git a/.agents/docs/core-beliefs.md b/.agents/docs/core-beliefs.md index 14aa2a2..179ea31 100644 --- a/.agents/docs/core-beliefs.md +++ b/.agents/docs/core-beliefs.md @@ -25,7 +25,7 @@ Agent 在运行时看不到群聊、在线文档和口头的约定。凡是会 `SchemaVersion` 是给客户端的承诺。改样式、加可选字段、改默认值都不提升版本；只有旧客户端确实无法继续工作才提升，并且要同时安排客户端升级路径。 -数据作者写起来顺手，不能以让客户端猜路径、猜格式为代价。 +数据作者写起来顺手，不能以让客户端猜路径、猜格式为代价。作者侧只守一条硬约束：入口 `metadata.xml` 的根节点写清 `SchemaVersion`；其余拆分方式、Include 层级、ID 短写都由读取器统一补齐，展平只是传输与聚合，不是读取前提。 ## 6. 说人话 diff --git a/.agents/docs/exec-plans/2026-11-07-composable-metadata.md b/.agents/docs/exec-plans/2026-11-07-composable-metadata.md new file mode 100644 index 0000000..73524bd --- /dev/null +++ b/.agents/docs/exec-plans/2026-11-07-composable-metadata.md @@ -0,0 +1,82 @@ +# Single-format composable Metadata implementation plan + +Status: Metadata single-format implementation verified; Desktop dependency integration and delivery pending. + +## Scope and decision + +The user explicitly removes all legacy compatibility. This plan follows the [single-format decision](../../notes/implemented/simplification/2026-11-07-single-metadata-format.md), which partially supersedes the [earlier composable decision](../../notes/implemented/architecture/2026-11-07-composable-metadata.md). Unified semantics with storage/organization/acquisition choices is the governing rule, not separate input dialects. Earlier test counts belong to the previous contract, not current acceptance. Original author Metadata/ XML/XSDs and unrelated code remain unchanged. + +One Reader and one path-ID format serve all inputs and outputs. Flat/Tree/Mixed change organization only. Do not retain wrappers, format enums, marker detection, legacy publication or old cache migration paths. Keep real resource/layout processing and domain behavior. Delivery updates existing [Metadata PR #65](https://github.com/RA3BattleNet/Metadata/pull/65) and dependent [Desktop PR #182](https://github.com/RA3BattleNet/Desktop/pull/182); do not create replacement PRs or merge them. + +## Shared interfaces and semantics + +- `MetadataReader.Read(pathOrUrl, options).Root` is the document-reading entry point. The result also captures normalized Document, EntryPath, SourceRoot and Documents with raw/normalized own-node XML and Include edges. Result-owned XML is immutable by convention; export uses captured provenance, not a disposed HTTP source. +- Options retain SourceRoot, ValidateReferences, CustomSource and InlineManifests. Remove ForceSourceTree and LegacySourcePaths. Reader does not write, replace variables, convert images or invent schema/revision values. Parser is only low-level XML projection. +- Generate short resource IDs as escaped `scope/local`; preserve existing full keys containing slash or colon without syntax policing or separator conversion. References match exact registered keys before scoped aliases. Keep business entity IDs unchanged. See the superseding [ID-key correction](../../notes/implemented/bug-fix/2026-11-07-resource-id-keys.md). No dual codec or format switch is restored. +- Interpret Source relative to its defining document and project it entry-directory-relative in Root. SourceRoot is containment/qualification only, never a resource-address fallback. Explicit URLs remain supported, Image.Url is authoritative, and HTTP documents cannot reference local files. Preserve URI/path security, cycle/dedup checks, own/descendant/global reference precedence and permanent ambiguity. +- Include contributes actual contents, including full Manifest bodies. External Manifest Source stays external unless InlineManifests is requested. A short stub ID is scoped to its target definition. External-body reads share SourceRoot and resolve exact canonical ID; no suffix/LocalId match or alias rewrite. +- `MetadataExporter.Export(input, outputDirectory, options)` uses one layout algorithm and returns the written XML paths. Layout remains Flat/Tree/Mixed; Mixed selects retained edges with Inline. InlineManifests, ResolveVariables and ConvertImages remain explicit. No IdFormat enum, root IdFormat/SourceBase marker or `--id-format` exists. +- Build calls normal Export and explicitly sets schema/revision while retaining XSD, variables, WebP and domain validation. Remove ExportLegacy and Builder.Load/LoadSource/Open with their separate HTTP/temp/retry pipeline. SourceExporter still explicitly creates short-ID local author declarations, including ExportManifestElement, without changing frozen installed bytes. +- Cache reads every acquired XML through Reader and persists one normalized complete snapshot contract. Raw validators remain paired with raw bytes, never transformed XML. Preserve root-304 child revalidation, last-complete fallback, zero-network offline reads, current `.uri`/`.etag` safety and Fresh/Stale semantics; no old-layout migration fallback or strict raw-flat branch. +- Snapshot/package Get/GetCached/Refresh take captured RootSnapshotResult and PackageEntry. Inline results inherit snapshot status and have no synthetic URI; external lookup requires exact registration and snapshot binding. Desktop keeps per-path authorization, cancellation and image cadence. +- Final public read-surface reduction also removes Parser.LoadFromFile and URI-only cache GetLeafAsync/GetCachedLeafAsync/RefreshLeafAsync with Match/TrySelect/RequireLeaf glue. Retain Parser.Load(XDocument)/ParseManifest as projection, package-context cache operations and PreloadLeavesAsync; SourceManifest has only Resolve(MetadataReadResult declaration, Metadata package). +- A Manifest with nonempty Source is always external; supplying a body at the same time fails. Full included manifests without Source remain full. This is one shape rule for all inputs, not a compatibility branch. + +## Ownership and integration + +Owners edit current paths without checks, commits, submodule edits or directory moves. Reader owns composition/ID/source and relevant tests; Exporter owns exporter/Builder/CLI/SourceExporter and build/export tests; Cache owns cache/SourceManifest and cache tests; Desktop owns consumers/tests. Parent integrates contracts, handles remaining references, moves files after integration, verifies once, reviews/simplifies touched code, records actual results and promotes the proposals only after evidence. Generated docs remain parent-owned. + +## Requested production directory simplification + +Namespaces stay unchanged. Parent completed production/test responsibility moves, including Reading/SourceManifest and Cache tests; Metadata integration verification is recorded below. Desktop dependency integration remains pending. + +| Target under `Ra3.BattleNet.Metadata/` | Files/responsibility | +|---|---| +| `Reading/` | MetadataReader, MetadataId, MetadataDocumentInfo, MetadataSourceOptions, IMetadataXmlSource, PhysicalFileXmlSource, HttpXmlSource, MetadataSourceManifest, MetadataParser (low-level projection), MetadataValidator (reference validation) | +| `Exporting/` | MetadataExporter, MetadataExportOptions, MetadataSourceExporter, MetadataXml | +| `Building/` | MetadataBuilder, MetadataInheritance, SchemaValidator, VariableResolver, MarkdownValidator, ImagePostProcessor, ImagingInvoker | +| `Cache/` | Existing client, disk cache, transfer and snapshot/result files; no new general framework | +| Root | Existing models, query/core contracts and Program entry point; no unrelated model movement | +| Removed | MetadataFlattener, compatibility-only MetadataDetector, old mode/codec/read API branches and obsolete wrappers | + +## Requested unit-test simplification + +Keep shared TestData in its existing copied location, namespaces and AppContext/RepoRoot calculation valid. Remove obsolete-API assertions and duplicate format-choice rows; migrate meaningful cases to current behavior instead of weakening coverage to hide failures. + +| Target under `Ra3.BattleNet.Metadata.Tests/` | Coverage and consolidation | +|---|---| +| `Reading/` | Reader; consolidated source/http/path/ID/composition cases. Merge meaningful LoadSource/Open/Include/IdScope coverage into Reader or focused source/path tests. Remove fixed-stopwatch PerformanceTests rather than replacing it with another timing threshold. | +| `Exporting/` | MetadataExport and SourceFormat author-write tests; Flat/Tree/Mixed and inline/external equivalence using one path format | +| `Building/` | BuildTests (replace StageA naming), Schema, ManifestValidation, Imaging and Inheritance; preserve unrelated Skudef/Mount behavior | +| `Cache/` | Client, Transfer, Image, Composition and JSON contract; preserve ETag/URI safety/offline/last-complete behavior, remove obsolete cache-shape compatibility tests | +| `Queries/` | Query, Style, Language, Compatibility and other model/query behavior; no legacy reader shims | +| `TestData/` | Shared fixtures unchanged in placement; update only fixtures needed by the single-format contract | + +Desktop retains service-boundary tests while migrating removed APIs and colon resource-ID assumptions; no submodule source edits or duplicate resolver. + +## Acceptance matrix + +All rows require new parent evidence; none is a recorded pass. + +| Concern | Required observation | +|---|---| +| API removal | All affected callers migrated; no old Builder read API, Flattener, dual codec, marker/mode option, ExportLegacy or CLI id-format shim remains | +| Reading/identity | Short IDs qualify; existing slash/colon keys retain exact identity; exact registration wins before aliases; ambiguity/type rules, cycles and dedup hold | +| Sources/manifests | Defining-document-relative Source, containment and HTTP/local boundary hold; Image.Url wins; included full bodies stay full; external stubs/bodies share exact identity | +| Actual Build | Build immutable repository author tree to external temporary output with existing schema/domain/variable/resource checks and no author XML/XSD edits | +| Layouts | Actual Reader -> Flat/Tree/Mixed -> Reader for both external and inlined manifests; equivalent model, references and real resources without format markers | +| Output safety | Traversal, encoded separators, source/output overlap and collisions fail before unsafe writes | +| Cache/network | Real loopback root 304 + changed child; child failure preserves prior complete Stale snapshot; cold restart/offline uses zero network; validators match raw bytes; no binary-resource prefetch | +| Tests/structure | Current full suite plus production/test responsibility moves; no lost security, domain, image, cancellation or offline cases; deterministic tests, no stopwatch gates | +| Desktop delivery | Push Metadata PR #65 commit, pin it in normal Desktop submodule, run default-reference tests/build and actual service smoke before PR #182 update; no external-project override | +| Documentation/review | Parent refreshes generated docs/checks, performs touched-code simplify/review, records exact evidence and limitations, and only then promotes proposals | + +## Evidence and delivery boundary + +No check is run by the documentation owner. The parent reports the following final Metadata evidence. Desktop integration and dependency delivery remain pending; no GUI/game execution is inferred. Generated documentation is parent-owned. Push Metadata first, then pin and validate Desktop's normal submodule; no merge is authorized. + +- Final full Metadata suite: 429/429 passed, 0 warnings and 0 failures. This is one complete suite, not a sum of targeted runs. The later source change only removed an obsolete API comment, not behavior. +- Actual Build and Tree export from unchanged repository data succeeded with 15 Mods and 3 Applications. Actual six Flat/Tree/Mixed × InlineManifests false/true roundtrips retained canonical path IDs without markers, inheritance, full manifest semantics and image bytes. +- Final real HTTP/offline smoke passed: root 304 plus child 200, child 404 with last-complete fallback, unchanged old captured snapshot and new-client offline reopening after stopping the server. +- Review findings for empty/default manifests, HTTP double decoding and contextual normalized disk identity were corrected and covered. The security finding around Image-only Url handling was corrected; parent reports no remaining issue in that review. +- Final tracked C# comparison against dcaa1d5, excluding bin/obj and including blank/comment lines: production 43 files/7617 lines -> 41/6941 (−676); tests 36/11167 -> 32/9504 (−1663); combined −2339 lines, excluding docs. These are observed reductions, not acceptance thresholds. diff --git a/.agents/docs/generated/notes-index.md b/.agents/docs/generated/notes-index.md index 5c3af13..0c1c82b 100644 --- a/.agents/docs/generated/notes-index.md +++ b/.agents/docs/generated/notes-index.md @@ -2,13 +2,20 @@ > 自动生成，勿手改。重新生成：`node scripts/verify-docs.mjs --write`。 -共 15 篇。按状态排序：提案 → 已实施 → 已否决。 +共 22 篇。按状态排序：提案 → 已实施 → 已否决。 | 日期 | 状态 | 分类 | 标题 | 问题 | |---|---|---|---|---| +| 2026-11-09 | implemented | architecture | [SchemaVersion 由源根声明](../../notes/implemented/architecture/2026-11-09-authored-schema-version.md) | 此前 SchemaVersion 只由构建注入：MetadataBuilder.Build 展平时给每份输出 XML 的根节点写 SchemaVersion 和 ContentRevision，源树自己不带这个属性。 | +| 2026-11-09 | implemented | architecture | [移除构建期修订标记 ContentRevision](../../notes/implemented/architecture/2026-11-09-drop-content-revision.md) | ContentRevision 是构建时写进发布物根节点的提交标识，文档里一直写成“客户端据此判断内容是否更新”。 | +| 2026-11-09 | implemented | bug-fix | [叶子清单不再追自己的存根 Source](../../notes/implemented/bug-fix/2026-11-09-leaf-stub-source-loop.md) | MetadataClient.FetchLeafDocumentAsync 读叶子清单时打开了 InlineManifests，CaptureGraphAsync 会把文档里所有带 ID + Source 的 Manifest 当成待内联的正文继续抓取。 | +| 2026-11-07 | implemented | architecture | [One composition core with layout-only exports](../../notes/implemented/architecture/2026-11-07-composable-metadata.md) | The earlier unified-reader entry point treated Include detec… | +| 2026-11-07 | implemented | bug-fix | [Preserve resource registration keys without colon policing](../../notes/implemented/bug-fix/2026-11-07-resource-id-keys.md) | The single-format decision(../simplification/2026-11-07-sing… | +| 2026-11-07 | implemented | simplification | [One metadata format without compatibility modes](../../notes/implemented/simplification/2026-11-07-single-metadata-format.md) | The user explicitly removes the old compatibility requiremen… | | 2026-11-06 | implemented | feature | [WebView2 zstd 包采用双层内容指纹](../../notes/implemented/feature/2026-11-06-webview2-raw-fingerprint.md) | 旧 WebView2 清单以 Compression="zip"、HashAlgorithm="None" 描述解包目录，不能验证下载物和待安装 ZIP 的内容。 | | 2026-11-04 | implemented | process | [Cloudflare 部署统一使用 Workers 静态资源](../../notes/implemented/process/2026-11-04-cloudflare-workers-deployment.md) | Cloudflare 后台项目是 Workers，但 npm 发布和本地服务仍调用 Pages 命令。 | | 2026-11-04 | implemented | process | [分支预览迁移到 Worker Previews](../../notes/implemented/process/2026-11-04-worker-previews.md) | 此前部署决定使用 wrangler versions upload 创建版本预览 URL，用户现在选择 Cloudflare 新的 Worker Previews 分支预览机制。 | +| 2026-11-01 | implemented | feature | [统一元数据读取入口](../../notes/implemented/feature/2026-11-01-unified-metadata-reader.md) | 元数据读取此前存在两条独立路径：MetadataBuilder.Load 只接受已展平的发布物，遇到 Include 直接报错； | | 2026-10-31 | implemented | feature | [源格式的导出与清单解析收进库](../../notes/implemented/feature/2026-10-31-source-format-host-api.md) | LoadSource 让宿主能读作者源文件之后，Desktop 仍自己实现了三类源格式逻辑：把在线展平树反向导出成单 Mod 作者声明（ModSourceDeclaration，315 行）、按 stub 找叶子清单并与 LocalId 对齐（ResolveManifestNode）、把独立 Manifest 节点包一层假 Metadata 根再解析。 | | 2026-10-28 | implemented | feature | [本地作者源文件内存读取](../../notes/implemented/feature/2026-10-28-local-mod-source-load.md) | Desktop 需要直接读作者磁盘上的 Mod 源文件（corona.xml、独立清单、管理目录里的 mod.xml），但库只提供两条路：MetadataBuilder.Build 要求整棵源树、XSD 与输出目录，MetadataParser.Load 又只接受已展平产物。 | | 2026-10-25 | implemented | process | [文档拆分为人类入口与 Agent 地图](../../notes/implemented/process/2026-10-25-agent-first-docs.md) | README.md 303 行、AGENTS.md 226 行，两个文件都在同时承担四类内容：作者写数据的教程、解析库 API、缓存行为、构建与发布流程。 | diff --git a/.agents/docs/generated/paths.md b/.agents/docs/generated/paths.md index 81b4ded..d764cac 100644 --- a/.agents/docs/generated/paths.md +++ b/.agents/docs/generated/paths.md @@ -22,69 +22,84 @@ | 文件 | 行数 | |---|---| -| `Ra3.BattleNet.Metadata/Cache/HttpMetadataTransfer.cs` | 84 | +| `Ra3.BattleNet.Metadata/Building/ImagePostProcessor.cs` | 80 | +| `Ra3.BattleNet.Metadata/Building/ImagingInvoker.cs` | 129 | +| `Ra3.BattleNet.Metadata/Building/MarkdownValidator.cs` | 88 | +| `Ra3.BattleNet.Metadata/Building/MetadataBuilder.cs` | 543 | +| `Ra3.BattleNet.Metadata/Building/MetadataInheritance.cs` | 304 | +| `Ra3.BattleNet.Metadata/Building/SchemaValidator.cs` | 132 | +| `Ra3.BattleNet.Metadata/Building/VariableResolver.cs` | 81 | +| `Ra3.BattleNet.Metadata/Cache/HttpMetadataTransfer.cs` | 87 | | `Ra3.BattleNet.Metadata/Cache/ImageResult.cs` | 24 | | `Ra3.BattleNet.Metadata/Cache/IMetadataTransfer.cs` | 41 | | `Ra3.BattleNet.Metadata/Cache/LeafResult.cs` | 34 | | `Ra3.BattleNet.Metadata/Cache/MetadataCacheJsonContext.cs` | 16 | -| `Ra3.BattleNet.Metadata/Cache/MetadataClient.cs` | 1702 | -| `Ra3.BattleNet.Metadata/Cache/MetadataDiskCache.cs` | 734 | +| `Ra3.BattleNet.Metadata/Cache/MetadataClient.cs` | 1679 | +| `Ra3.BattleNet.Metadata/Cache/MetadataDiskCache.cs` | 837 | | `Ra3.BattleNet.Metadata/Cache/MetadataFreshness.cs` | 18 | | `Ra3.BattleNet.Metadata/Cache/RootSnapshotResult.cs` | 23 | -| `Ra3.BattleNet.Metadata/ImagePostProcessor.cs` | 72 | -| `Ra3.BattleNet.Metadata/ImagingInvoker.cs` | 129 | +| `Ra3.BattleNet.Metadata/Exporting/MetadataExporter.cs` | 378 | +| `Ra3.BattleNet.Metadata/Exporting/MetadataExportOptions.cs` | 18 | +| `Ra3.BattleNet.Metadata/Exporting/MetadataSourceExporter.cs` | 310 | +| `Ra3.BattleNet.Metadata/Exporting/MetadataXml.cs` | 34 | | `Ra3.BattleNet.Metadata/LanguageTag.cs` | 29 | | `Ra3.BattleNet.Metadata/ManifestMountPlanner.cs` | 156 | | `Ra3.BattleNet.Metadata/ManifestRawFingerprint.cs` | 33 | -| `Ra3.BattleNet.Metadata/MarkdownValidator.cs` | 88 | -| `Ra3.BattleNet.Metadata/Metadata.cs` | 120 | -| `Ra3.BattleNet.Metadata/MetadataBuilder.cs` | 842 | +| `Ra3.BattleNet.Metadata/Metadata.cs` | 108 | | `Ra3.BattleNet.Metadata/MetadataColor.cs` | 43 | -| `Ra3.BattleNet.Metadata/MetadataFlattener.cs` | 400 | -| `Ra3.BattleNet.Metadata/MetadataInheritance.cs` | 304 | -| `Ra3.BattleNet.Metadata/MetadataParser.cs` | 106 | | `Ra3.BattleNet.Metadata/MetadataQueryExtensions.cs` | 339 | | `Ra3.BattleNet.Metadata/MetadataQueryModels.cs` | 199 | | `Ra3.BattleNet.Metadata/MetadataResourceUri.cs` | 31 | | `Ra3.BattleNet.Metadata/MetadataSchema.cs` | 21 | -| `Ra3.BattleNet.Metadata/MetadataSourceExporter.cs` | 302 | -| `Ra3.BattleNet.Metadata/MetadataSourceManifest.cs` | 52 | -| `Ra3.BattleNet.Metadata/MetadataXml.cs` | 34 | | `Ra3.BattleNet.Metadata/ModCompatibility.cs` | 61 | -| `Ra3.BattleNet.Metadata/Program.cs` | 94 | -| `Ra3.BattleNet.Metadata/SchemaValidator.cs` | 132 | -| `Ra3.BattleNet.Metadata/VariableResolver.cs` | 81 | +| `Ra3.BattleNet.Metadata/Program.cs` | 95 | +| `Ra3.BattleNet.Metadata/Reading/HttpXmlSource.cs` | 208 | +| `Ra3.BattleNet.Metadata/Reading/IMetadataXmlSource.cs` | 35 | +| `Ra3.BattleNet.Metadata/Reading/MetadataDocumentInfo.cs` | 52 | +| `Ra3.BattleNet.Metadata/Reading/MetadataId.cs` | 34 | +| `Ra3.BattleNet.Metadata/Reading/MetadataParser.cs` | 75 | +| `Ra3.BattleNet.Metadata/Reading/MetadataReader.cs` | 296 | +| `Ra3.BattleNet.Metadata/Reading/MetadataSourceManifest.cs` | 32 | +| `Ra3.BattleNet.Metadata/Reading/MetadataSourceOptions.cs` | 27 | +| `Ra3.BattleNet.Metadata/Reading/MetadataValidator.cs` | 114 | +| `Ra3.BattleNet.Metadata/Reading/PhysicalFileXmlSource.cs` | 76 | ## 测试源码 | 文件 | 说明 | |---|---| -| `Ra3.BattleNet.Metadata.Tests/CacheJsonContractTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/IdScopeTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/ImagingTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/IncludeTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/InheritanceTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/LanguageTagTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/ManifestMountPlannerTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/ManifestParseTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/ManifestRawFingerprintTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/ManifestValidationTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/MetadataClientImageTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/MetadataClientTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/MetadataClientTransferTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/MetadataTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/ModCompatibilityTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/PostHeadImageTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/PostTypeTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/PublishSurfaceTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/QueryApiTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/SchemaTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/SourceFormatApiTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/SourceLoadTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/StageATests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/StyleResourceTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/StyleTests.cs` | MSTest 测试 | -| `Ra3.BattleNet.Metadata.Tests/UpdaterEndpointTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/BuildTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/ImagingTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/InheritanceTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/ManifestParseTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/ManifestRawFingerprintTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/ManifestValidationTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/PublishSurfaceTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Building/SchemaTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Cache/CacheJsonContractTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientImageTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTransferTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Cache/MetadataCompositionCacheTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Exporting/MetadataExportTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Exporting/SourceFormatApiTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/LanguageTagTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/ManifestMountPlannerTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/ModCompatibilityTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/PostHeadImageTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/PostTypeTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/QueryApiTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/StyleResourceTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/StyleTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Queries/UpdaterEndpointTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/HttpSourceTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/IdScopeTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/MetadataReaderTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/MetadataTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/PathSecurityTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/RegistrationValidationTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/SourceLoadTests.cs` | MSTest 测试 | +| `Ra3.BattleNet.Metadata.Tests/Reading/ValidationTests.cs` | MSTest 测试 | ## 原始数据树 `Metadata/` diff --git a/.agents/docs/glossary.md b/.agents/docs/glossary.md index 184b8c4..a93dd71 100644 --- a/.agents/docs/glossary.md +++ b/.agents/docs/glossary.md @@ -5,12 +5,12 @@ | 术语 | 准确含义 | |---|---| | **源数据** | `Metadata/` 下作者手写的 XML、图片、Markdown、清单。可以带 `Include`，可以有短 ID。 | -| **发布树 / 发布面** | `Output/` 里的最终产物，客户端只认这一份。不含 `Include`、XSD、`Templates/`。 | -| **大总表** | 展平后的根文件 `metadata.xml`。所有实体和资源登记节点都在里面，独立清单的正文不在。 | -| **展平** | 构建时把 `Include` 展开、`Base` 继承合并、短 ID 补全为完整 ID 的过程。执行者：`MetadataFlattener` 与 `MetadataInheritance`。 | -| **登记节点** | 带 `@ID` 的 `Image`、`Markdown`、`Manifest` 标签。它声明“有这个资源”，正文在 `Source` 指向的文件里，或在 `Url` 指向的外部地址。 | -| **短 ID** | 作者在文件内写的资源 ID，例如 `icon-64px`。只在当前文件作用域内需要唯一。 | -| **完整 ID** | 展平后带路径前缀的 ID，形如 `corona:icon-64px`。前缀由构建器生成，作者手写会直接报错。 | +| **发布树 / 发布面** | Build 输出的平面 metadata.xml 与外部资源，不含 XSD、Templates；通用 export 还支持 Tree/Mixed Include 组织，读取语义相同。 | +| **大总表** | Build 输出的平面 metadata.xml，包含实体与资源登记；默认外部清单正文另存，显式内联时正文可在图内。 | +| **展平** | Reader 展开 Include、合并 Base 继承并限定短资源 ID；Exporter 的 Flat 将捕获图写成平面 XML。不是第二种读取模式。 | +| **登记节点** | 带 ID 的 Image、Markdown、Manifest。Source 相对定义文档，Image.Url 优先；Manifest 有 Source 即外部引用，不允许同时带 body。 | +| **短 ID** | 作者文件中的资源局部名，如 icon-64px；Reader 总按定义 scope 限定，即使输入只有一个文件。 | +| **完整 ID** | 资源登记键；新生成形式如 mods/corona/corona/icon-64px，scope 来自定义文件相对 SourceRoot 去 .xml。已有含 `/` 或 `:` 的键原样保留，引用优先精确匹配，不把 ID 当地址校验。 | | **实体** | `Mod` 或 `Application`。`@ID` 全仓库唯一，不分大小写。 | | **版本包** | 实体下的 `Package`，只有 `Version`、可选 `ReleaseDate`、可选 `Manifest`。不携带更新说明。 | | **独立清单** | 某个版本的文件表文件，通过 `Manifest/Source` 单独发布，不塞进大总表。旧文档里叫“叶子清单”。 | @@ -19,11 +19,10 @@ | **硬失败** | 校验不通过就清空输出、抛错、进程返回 1，不产出带病产物。 | | **主发布基** | 调用方第一次打开时传入的那个地址。写进 `origin.json` 后不再改变，其余地址视为其他发布基。 | | **最后有效缓存** | 上一次成功下载并校验通过的正文。网络失败时仍可离线读取，但状态不是 `Fresh`。 | -| **快照** | 一次成功读取的根清单加上它的来源地址和摘要。独立清单的身份只在这份快照里解析。 | +| **快照** | 完整组合根及来源/SourceRoot/摘要；只有整图成功才发布。包清单身份绑定捕获快照，raw XML/HTTP validators 与 normalized record 分开。 | | **暂存文件** | 下载方写到磁盘的临时正文。`.tmp` 与 `.stage` 下的内容不当作缓存读取。 | | **语言族比较** | `LanguageTag.Matches` 的规则：查询带地区时按语言族命中（`zh-CN` 命中 `zh`），查询不带地区时精确匹配，比较不分大小写。 | | **契约版本** | 根节点的 `SchemaVersion`，当前是 `1.0`。它约束客户端的兼容判断，与内容更新无关。 | -| **内容修订** | 根节点的 `ContentRevision`，构建时写入的提交标识，用来判断要不要重新拉取。 | ## 不要用的说法 diff --git a/.agents/docs/index.md b/.agents/docs/index.md index 918e3b3..15e505a 100644 --- a/.agents/docs/index.md +++ b/.agents/docs/index.md @@ -8,8 +8,10 @@ | 写版本清单、`Skudef`、下载来源 | [references/manifest-and-skudef.md](references/manifest-and-skudef.md) | 文件表不进大总表；写了 `Skudef` 就不许在 `File` 上写挂载属性 | | 改 XML 模型、样式、语言、启动兼容性 | [references/metadata-model.md](references/metadata-model.md) | `SchemaVersion` 保持 `1.0`；显示名 `zh`/`en` 必填 | | 改解析库、客户端查询、磁盘缓存 | [references/library-api.md](references/library-api.md) | 发布树布局是纯函数；下载边界只在 `IMetadataTransfer` | +| 单格式 Metadata、目录/测试精简与集成验收 | [计划](exec-plans/2026-11-07-composable-metadata.md) / [single-format 决定](../notes/implemented/simplification/2026-11-07-single-metadata-format.md) | Metadata 已验证；Desktop 依赖交付与验证待完成；只留 Reader/path ID，无 legacy API/mode | | 改构建流水线、CI、发版 | [references/build-and-release.md](references/build-and-release.md) | 失败即清空输出；Imaging 只在发版跑 | | 了解代码分层与数据流 | [../ARCHITECTURE.md](../ARCHITECTURE.md) | 依赖只能单向 | +| 看元数据书写与读取模型的可视化说明 | [visuals/metadata-authoring.html](visuals/metadata-authoring.html) | 只认入口 metadata.xml；源树自由拆分；展平只是传输与聚合 | | 查某个文件在哪、多少行 | [generated/paths.md](generated/paths.md) | 自动生成，以表为准 | | 追溯以前为什么这么定 | [generated/notes-index.md](generated/notes-index.md) | 自动索引；原文在 `.agents/notes/` | | 写决策笔记、改文档 | [../CONTRIBUTING.md](../CONTRIBUTING.md) | 非平凡改动必写；文档必须挂到本索引 | diff --git a/.agents/docs/references/build-and-release.md b/.agents/docs/references/build-and-release.md index e62fa83..9f270d0 100644 --- a/.agents/docs/references/build-and-release.md +++ b/.agents/docs/references/build-and-release.md @@ -13,7 +13,9 @@ npm run deploy # 构建后发布到 Cloudflare Workers npm run preview # 本地用 wrangler 起静态服务看产物 ``` -命令行参数：`--src=`、`--dst=`、`--webp`、`--schema-version=`、`--content-revision=`。不带 `--dst` 时默认 `./Output`。 +命令行参数：`--src=`、`--dst=`、`--webp`。不带 `--dst` 时默认 `./Output`。构建没有任何版本参数：`SchemaVersion` 由作者写在源根 `metadata.xml` 的根节点上，构建也不追加修订标记，只解析 `${TIMESTAMP}` 等变量。 + +单格式保留 `build` 与 `export`。export 保留真实的 `--layout=flat|tree|mixed`、Mixed 选择性 Include 内联、显式 manifest 内联及资源/变量/图片处理；删除 `--id-format`，没有 legacy flavor。所有布局写同一 canonical path ID，不写格式 markers。无效选项清楚报错，不回退旧模式。Metadata 单格式与目录迁移已验证，Desktop 依赖集成待完成；证据见 [集成计划](../exec-plans/2026-11-07-composable-metadata.md)。 本地开发和调试默认不跑图片转换；只有线上发版才带 `--webp`。**不要在代码里再加 `RUN_STAGE_B` / `StageB` 之类的阶段开关**，流水线只有一条。 @@ -23,7 +25,7 @@ npm run preview # 本地用 wrangler 起静态服务看产物 - 入口固定是 `<源目录>/metadata.xml`；找不到直接失败。 - 先校验源树（`MetadataSchema.xsd`），展平后再校验发布物（`MetadataPublishSchema.xsd`）。 -- 独立清单会被再展平一次并替换宏变量；否则发布物里会残留客户端拒绝的 `Include` 与 `${…}`。 +- 所有 XML 读取共用 Reader；外部清单默认保持外部，构建导出时处理变量。没有严格发布读取/作者读取两条兼容流水线。 - `--webp` 在展平之后执行：`ImagePostProcessor` 调 Imaging 转图，再改写 XML 里的 `Source`。 - 失败一律清空 `Output/` 并返回 1，不留半成品。 @@ -70,12 +72,11 @@ npm run preview # 本地用 wrangler 起静态服务看产物 `Ra3.BattleNet.Metadata.Tests` 使用 MSTest，禁止混入 xunit。覆盖范围： -- 展平与 `Include`、`Base` 继承合并、实体 ID 冲突检测 -- 源模式与发布模式的硬失败拦截、ID 与资源断链 -- 发布树结构断言、Catalog 与 LINQ 查询 API -- 独立清单解析、`Skudef` 与历史 `Mount` 的挂载计划 -- 缓存 JSON 契约、缓存读写节奏、下载边界（`MetadataClient` 三组测试） -- 语言标签、颜色转换、样式继承、启动兼容性、图片后处理 +- Reading/：Include、继承、路径 ID、引用优先级/永久歧义、HTTP/source-root 安全；SourceLoad/OpenMethod 本质用例归并，删除专测旧 API/mode 与固定 stopwatch PerformanceTests。 +- Exporting/：Flat/Tree/Mixed、内外部清单和短 ID 作者写出；删除旧双 encoding 矩阵的一维，不机械改写所有 obsolete fixtures。 +- Building/：Build（不再 StageA 命名）、Schema、ManifestValidation、Imaging、Inheritance；保留真实 Skudef/Mount 与领域校验。 +- Cache/：完整 snapshot、JSON、ETag/原始字节配对、URI 安全、offline 和图片节奏；删除旧 flat-cache migration 测试，不重钉旧布局。 +- Queries/：Catalog/LINQ、语言、颜色、样式和游戏兼容性。共享 TestData 原位置不变，不新增通用测试框架。父在集成 freeze 后移动目录，完整表见计划。 改构建器或缓存行为时，先补测试再改实现；测试失败不允许用重跑掩盖。 diff --git a/.agents/docs/references/data-authoring.md b/.agents/docs/references/data-authoring.md index 1c24aaf..1764b31 100644 --- a/.agents/docs/references/data-authoring.md +++ b/.agents/docs/references/data-authoring.md @@ -22,7 +22,7 @@ 2. **文件内的资源短 ID 只要本文件内不重名**。`Image`、`Markdown`、`Manifest` 的 `@ID` 不需要全仓库唯一。 3. **顶层实体的 `@ID` 全仓库唯一**。大小写不敏感，跨 Mod 与 Application 一起查重。 -构建器自动做的事：补全 `{路径前缀}:{localId}`、把引用处的短名换成完整 ID、复制被引用资源、做全量校验。**不要自己手写冒号前缀**，写了会被直接拒绝。 +Reader 把短资源 ID 限定为 `{scope}/{localId}`，scope 是定义 XML 相对 SourceRoot 的路径去掉 `.xml`；单文件也不例外。已有含 `/` 或 `:` 的完整 ID 原样保留，引用优先精确匹配，不禁止冒号。作者建议写本文件短名；Build 复制资源并做全量校验。Source 相对定义文档，SourceRoot 不提供第二路径回退；Image.Url 优先。Manifest 有 Source 就是外部引用，不允许同时写正文；无 Source 的完整 Manifest 经 Include 保持完整。 ## 三、目录结构 @@ -128,8 +128,6 @@ Metadata/ - mods/corona/corona.xml (Mod) - mods/fake/fake.xml (Application) -错误: 登记 ID "a/b:icon" 包含 ':'——前缀由构建器自动生成，请直接写短名 - 错误: Image 资源不存在: images/nope.png (ID: ...) 错误: Mod 'Demo': 缺少 en DisplayName diff --git a/.agents/docs/references/library-api.md b/.agents/docs/references/library-api.md index fbd2649..cb8521a 100644 --- a/.agents/docs/references/library-api.md +++ b/.agents/docs/references/library-api.md @@ -14,90 +14,66 @@ 客户端拿到的是静态文件，没有接口调用。真实地址 = 发布基地址 + `Source`，用 `MetadataResourceUri.Resolve` 拼装，不要手写字符串拼接。 -## 二、一次性读取 +## 二、单一读取与布局导出 + +当前契约见 [single-format 决定](../../notes/implemented/simplification/2026-11-07-single-metadata-format.md) 和 [集成计划](../exec-plans/2026-11-07-composable-metadata.md)。Metadata 已完成单格式验证；Desktop 依赖交付与验证待完成。 ```csharp -var doc = MetadataBuilder.Load("https://metadata.ra3battle.net/metadata.xml"); -// 也可以是本地路径：MetadataBuilder.Load("./Output/metadata.xml") +var input = MetadataReader.Read("https://metadata.ra3battle.net/metadata.xml"); +var root = input.Root; +var local = MetadataReader.Read("./Metadata/mods/corona/corona.xml", + new MetadataSourceOptions { SourceRoot = "./Metadata" }).Root; +var corona = root.Catalog().Mod("Corona"); +var writtenXml = MetadataExporter.Export(input, "./ComposedOutput", + new MetadataExportOptions { Layout = MetadataExportLayout.Tree }); ``` -内部按 `http`/`https`/`file` 分流；HTTP 路径带连接预算与有限重试，读不到就抛异常，不返回空文档。 +### 读取、ID 与 Source -```csharp -foreach (var mod in doc.Mods().Where(m => m.Version is not null).OrderBy(m => m.Id)) - Console.WriteLine($"{mod.Id}@{mod.Version}"); - -var app = doc.Catalog().Application("RA3BattleNet"); // 不区分大小写 -var corona = doc.Catalog().Mod("Corona")!; - -// 按版本取独立清单地址，再单独拉取 -var leafUrl = MetadataResourceUri.Resolve("https://metadata.ra3battle.net/metadata.xml", corona.ManifestSource()); -var manifest = MetadataBuilder.Load(leafUrl.AbsoluteUri).Find("Manifest")!.ToManifestEntry(); - -// 新闻与友情链接是原始节点，库不做强类型查询，也不替你筛语言 -var posts = app!.Raw.Find("Posts"); -var firstPost = posts?.Children.FirstOrDefault(c => c.Name == "Post"); -var zhTitle = firstPost?.Find("Titles")?.Children - .FirstOrDefault(c => c.Name == "Title" && LanguageTag.Matches(c.Get("Language"), "zh")) - ?.Value; - -var links = corona.Raw.Find("Links"); - -// 挂载计划：写了 Skudef 就按声明顺序和条件来，没写就走旧的 File@Mount 角色 -var plan = ManifestMountPlanner.Build( - manifest.Skudef, - manifest.Files.Select(f => new ManifestMountFile(f.FileName, f.Mount, f.Language, f.Package)).ToList(), - language: "en", - packages: ["hd-shadow"], - localConfigExists: _ => false); -``` +`MetadataReader.Read(pathOrUrl, options)` 返回 MetadataReadResult：Root、规范化 Document、EntryPath、SourceRoot 和 Documents。每份 MetadataDocumentInfo 捕获 Path、RelativePath、原始 Document、自己的 NormalizedDocument 和 Includes（SourcePath/TargetPath）。结果拥有的 XML 按不可变使用；导出使用捕获图，不依赖已释放的内部 HTTP source。 -### 读本地作者源文件（未发布） +源树和展平发布物走同一条读取路径：只要入口根节点写了 `SchemaVersion`，`MetadataBuilder.Build` 就不是读取的前置条件；构建只负责把分散文档合成一份、解析变量并产出发布面。 -```csharp -// 入口可以是带 Includes / Base / InheritFrom / 短 ID 的源文件；源根缺省为入口所在目录 -var source = MetadataBuilder.LoadSource("./Metadata/mods/corona/corona.xml"); -// 也可以显式给源根：ID 前缀与资源路径都相对它 -var leaf = MetadataBuilder.LoadSource("./Metadata/mods/corona/manifests/3.258.xml", "./Metadata/mods/corona"); -``` +MetadataSourceOptions 保留 SourceRoot、ValidateReferences、CustomSource 和 InlineManifests。CustomSource 允许缓存异步获取 XML 后交给同一 Reader。读取只在内存中合并 Include/继承与资源引用，不写盘、不替换变量、不转换图片，不覆盖声明的 SchemaVersion。 -`LoadSource` 在内存里展平 Include、合并继承、限定登记 ID，返回普通 `Metadata` 树：不写任何文件，不替换 `${TIMESTAMP}` / `${ENV:...}`，不做 XSD 与跨实体 `Compatibility` 校验（单模组没有整棵目录），也不要求其他 Mod 存在。 +- 新生成的资源 ID 使用可逆转义的 `scope/local`，scope 是定义文件相对 SourceRoot 的路径去掉 `.xml`。短 ID 总是限定；已有含 `/` 或 `:` 的完整登记 ID 原样保留，引用先按完整字符串精确匹配，不转换分隔符，也不把 ID 当文件路径校验。Mod/Application 业务 ID 不变。 +- MetadataId 统一提供 PathPrefix、Qualify、LocalId；LocalId 取最后一个 `/` 段并反转义。没有双 codec 或按输入猜格式。 +- Source 始终相对定义文档解释，Root 中投影为相对入口目录。SourceRoot 只负责限定和 containment，不是找不到文件时的第二基址。HTTP 文档不能引用本地文件；显式资源 URL 保留。Image.Url 优先于信息性 Source，不触发本地探测。 +- Include 贡献实际内容，无 Source 的完整 Manifest 不转换为 stub。Manifest 只要声明非空 Source 就是外部引用，同时携带 body 直接失败，不按输入来源猜语义；默认保持外部，显式 InlineManifests 才内联外部 body。 +- 短 Manifest stub 按目标定义 scope 限定。外部 body 使用共享 SourceRoot 并精确匹配 canonical ID，不按后缀/LocalId 凑身份，不重写别名。 +- 保留 own/descendant/global 引用优先级、永久歧义、字段类型规则（Icon 可指 Image/Markdown）、循环/去重和路径/URI 安全。资源 ID 比较为 OrdinalIgnoreCase，文档身份按 HTTP/文件系统规则。 -- 源根缺省是入口文件所在目录；登记 ID 的前缀同样是文件相对源根、去 `.xml` 的路径，与构建一致。 -- 资源 `Source` 改写为相对源根的路径：先按定义文件目录解释，不存在再按源根解释；两项都必须落在源根内，越界直接抛 `InvalidOperationException`，不会去探测、读取目标。 -- 入口、Include 与资源路径越出源根同样在读取前拒绝；不跟随跨根链接。 -- Include 进来的独立清单只留下登记 stub（ID 已限定、`Source` 相对源根）。`` 手写 stub 保留入口文件前缀，单独读叶子时前缀来自文件路径，两者可能不同，按 `ManifestRegistration(ID)` 精确查找前先用 `MetadataFlattener.LocalId` 对齐局部名。 -- 需要发布物的严格校验（XSD、Compatibility、变量残留）时仍用 `Load` / `LoadFromFile`；它们拒绝任何含 `Include` 的源树。 +目标 API 删除 MetadataBuilder.Load/LoadSource/Open、MetadataFlattener、ForceSourceTree、LegacySourcePaths、MetadataExportIdFormat、IdFormat/SourceBase markers 和 EncodePath/DecodePath，不留 shim。MetadataParser 只做底层 XML 到模型投影，不是第二种文档读取入口。 -已展平的内存文档可直接解析，不需要写临时文件： +Parser 的公开投影保留 `Load(XDocument)` 与 `ParseManifest`；文件读取 `LoadFromFile` 移除，内部 stream 投影不构成公开 IO 入口。XML/外部 leaf 必须在共享 SourceRoot 内，不通过 URI 猜 scope；资源 Source 可用显式 HTTP 外链。HTTP 文档拒绝 file、Windows drive 和 UNC 本地地址；同源 root-relative 路径仍须 containment 校验。 -```csharp -var doc = new XDocument(new XElement("Metadata", savedManifestElement)); -var root = MetadataParser.Load(doc); // 根必须是 Metadata，且不得含 Include / Includes / Module -``` +### 导出与构建 + +MetadataExporter.Export 写 metadata.xml、必要 XML 与资源，返回写出的 XML 路径列表。Layout 保留 Flat（默认）、Tree、Mixed；Mixed 用 `Inline: Func` 选择目标 RelativePath。InlineManifests、ConvertImages、ResolveVariables 默认 false。所有布局使用同一图和算法，输出同一 path ID，不写格式 markers，也没有 IdFormat 选项。 -### 导回作者源格式，以及单独解析清单 +Flat 只改变 XML 组织；外部清单仍是外部资源，真正单一 XML 需显式内联清单。平面输入无法恢复丢失的注释/原始拆分。保留 Source、Markdown、Image/Manifest 及 URL 语义；写盘前拒绝源/输出重叠、越界和碰撞。失败只清理目标输出。Build 调普通 Export，显式设置 schema/revision，继续做 schema、变量、WebP 和领域校验，不走 ExportLegacy 或另一套读取/资源拷贝。 -`LoadSource` 的反向操作和清单读取也在库里，宿主不再自己压短 ID、改 `Source` 或对齐前缀。 +### 作者声明写出与清单投影 ```csharp -// 整份 Mod：保留所有版本与 CurrentVersion；发布地址为空时保留相对 Source -var whole = MetadataSourceExporter.ExportWhole(root, "Corona", "https://metadata.ra3battle.net/metadata.xml"); -// 单版本：只留这一版，并把它设成 CurrentVersion +var whole = MetadataSourceExporter.ExportWhole(root, "Corona", + "https://metadata.ra3battle.net/metadata.xml"); var one = MetadataSourceExporter.ExportVersion(root, "Corona", "3.259"); - -// 本地声明里的版本清单：内联直接返回，stub 则按 Source 读叶子并按 LocalId 对齐 -var manifest = MetadataSourceManifest.Resolve(localRoot, declarationPath, package.Raw); - -// 独立清单节点或文本，不必再套假的 Metadata 根 +var authorManifest = MetadataSourceExporter.ExportManifestElement(manifestElement); +var localInput = MetadataReader.Read(declarationPath, sourceOptions); +var manifest = MetadataSourceManifest.Resolve(localInput, package.Raw); var entry = MetadataParser.ParseManifest(manifestElement); -var same = MetadataParser.ParseManifest(""); - -// 反方向：把解析出的节点还原成 XML，抄清单原文时用 var xml = MetadataXml.Write(manifest!); ``` -`ExportWhole` / `ExportVersion` 只做纯转换：不写盘、不发请求、不改动传入的树。带 `Url` 的图片只保留外链，不猜 `Source`；返回的是语义等价的源格式，不恢复发布时丢掉的注释与拆分。 +SourceExporter 显式生成短 ID 作者声明，是 Desktop 的实际写出功能，不是读取兼容模式。转换不写盘、不请求、不改输入树，不声称恢复原始拓扑；带 Url 的图片不猜 Source。Desktop 对独立本地副本使用 ExportManifestElement，不改变冻结的原始 installed Manifest 字节。SourceManifest 接受 MetadataReadResult 以共享 source-root 上下文，按精确身份读取外部清单，不再 LocalId 对齐。 + +实现位于 [Reading/MetadataSourceManifest.cs](../../../Ra3.BattleNet.Metadata/Reading/MetadataSourceManifest.cs)；只消费 read result 与 package，不提供旧 URI/LocalId 读取入口。 + +### 文件与测试组织 + +生产代码按 Reading/、Exporting/、Building/ 和现有 Cache/ 分责，模型/query/core 与 Program 留 root，命名空间不变。测试按 Reading/、Exporting/、Building/、Cache/、Queries/ 分责，TestData 位置不变。优先删除专测旧 API/mode 和重复 encoding 矩阵，将 SourceLoad/OpenMethod 本质用例归并到 Reader，删除固定 stopwatch PerformanceTests；不是机械改写所有旧测试 ID，也不新增通用测试框架。完整目录表与父负责的迁移状态见计划。 + ## 三、查询 API @@ -111,7 +87,7 @@ var xml = MetadataXml.Write(manifest!); | `ModEntry.ManifestSource(version)` | 取独立清单的相对路径；缺版本、缺清单、缺 `Source` 都抛出带原因的异常 | | `entry.Raw.Find("Posts")` / `mod.Raw.Find("Links")` | 读实体自己的新闻或友情链接 | | `node.ToManifestEntry()` | XML 节点转强类型 `ManifestEntry`（文件表、依赖、`Skudef`） | -| `MetadataSchema.Current` / `IsCompatible(version)` | 契约版本与兼容判断；`Load` 不会自动调用 | +| `MetadataSchema.Current` / `IsCompatible(version)` | 契约版本与兼容判断；底层 XML 投影不自动执行发布校验 | | `MetadataResourceUri.Resolve(baseUrl, source)` | 相对 `Source` 拼成绝对地址 | | `MetadataColor.ToCss(value, format)` | 颜色转 CSS；ARGB 的 Alpha 移到末尾，非法值抛 `FormatException` | | `ManifestMountPlanner.Build(...)` | 算出挂载指令序列 | @@ -124,9 +100,10 @@ var xml = MetadataXml.Write(manifest!); ```csharp var client = new MetadataClient(cacheDirectory, TimeSpan.FromSeconds(30)); var root = await client.RefreshRootAsync(metadataUrl, ct); -var leafUri = MetadataResourceUri.Resolve(root.OriginUri!.AbsoluteUri, relativeSource); -var leaf = await client.GetLeafAsync(root, version, leafUri, ct); -var cached = await client.GetCachedLeafAsync(root, version, leafUri, ct); +// package 来自 root 捕获的那份目录，不通过外部 URI 或后缀猜身份。 +var leaf = await client.GetPackageAsync(root, package, ct); +var cached = await client.GetCachedPackageAsync(root, package, ct); +var refreshed = await client.RefreshPackageAsync(root, package, ct); var leaves = await client.PreloadLeavesAsync(root, ct); // Total / Loaded / Failed var offline = await client.OpenSnapshotAsync(metadataUrl, ct); var image = await client.GetImageAsync(root, imageId, ct); @@ -148,12 +125,24 @@ var revision = client.ImageRevision(root, imageId); - `OpenSnapshotAsync` 只读磁盘，状态只会是 `Stale` 或 `Unavailable`，不会是 `Fresh`。 - `RefreshRootAsync` 对 http(s) 发条件 GET，对本地路径与 `file://` 读当前文件，成功才是 `Fresh`。 -- `GetCachedLeafAsync` 只取已有缓存，不发请求、不等待别人正在进行中的请求；缺失返回 `Unavailable`。 +- `GetCachedPackageAsync` 不发请求、不等待在途请求；外部 body 缺失返回 `Unavailable`，内联 body 来自捕获快照。 - `PreloadLeavesAsync` 的可用文档包含缓存回退，不代表本轮联网确认成功。 - 已登记图片命中完整缓存时直接返回，不发 HTTP，也不做后台校验。冷缓存只共享一次下载，失败后十分钟内普通读取不再请求；`RefreshImageAsync` 不受这个限制。 - `ImageRevision` 用旁挂摘要算虚拟地址，只有 32 位，碰撞时可能得到相同地址，不能当成绝不复用的证明；Markdown 正文里没有登记的图片不进这套机制。 - 关闭磁盘缓存时，`file://` 每次读当前源文件并返回 `Stale`；http(s) 不主动请求。 +### Snapshot/package access for composed roots + +The client exposes `Task GetPackageAsync/GetCachedPackageAsync/RefreshPackageAsync(RootSnapshotResult snapshot, PackageEntry package, CancellationToken ct = default)` and PreloadLeavesAsync. Package access requires captured snapshot/package context, not a mandatory ManifestSource URI. GetLeafAsync/GetCachedLeafAsync/RefreshLeafAsync and their Match/TrySelect/RequireLeaf glue are removed without wrappers or URI-based source-context inference. + +Inline results use the snapshot's ManifestNode/ManifestEntry, have null SourceUri, and inherit snapshot.Status in all three operations. A complete Fresh root therefore permits Fresh inline data without a fake HTTP request; Stale root never becomes Fresh. External package access verifies Raw membership and Source binding; external cached-only reads remain Stale. Preload handles both forms and does not claim independent network freshness for inline data. + +Runtime refresh acquires every XML through IMetadataTransfer and composes it through Reader CustomSource, including leaves. Root 304 still revalidates children. Only a complete graph publishes the normalized snapshot; failures retain the last complete graph, and offline open uses no network. One private normalized snapshot record is separate from URL-mapped raw bodies and their validators. Never attach a raw HTTP ETag to transformed bytes. There is no raw-flat preservation, strict-parser input split or old-cache migration fallback. Current `.uri`/`.etag` ownership and integrity remain safety functionality. The parent verifies these behaviors against the new contract before recording evidence. + +The single private `.compositions` record carries Origin, SourceRoot, normalized Root/Sha256 and captured raw-document digests. Every root entry point uses that same complete-record contract; raw publication-tree bodies are inputs for XML conditional revalidation, not an alternate normalized snapshot. External bodies require exact canonical ID, and SourceManifest's sole resolver is `Resolve(MetadataReadResult declaration, Metadata package)`, preserving the declaration context instead of inferring it from a URI or local-name suffix. + +Normalized snapshot keys include both SourceRoot context and document URI, identically in memory and on disk. A shared external leaf under two roots therefore retains each root's canonical identity across cold offline reopening. Raw URI-mapped body/ETag reuse remains separate from contextual normalized identity. HTTP relative document paths retain percent escapes until PathPrefix decodes segments once, keeping direct HTTP and cache qualification identical. A Manifest without Source is full even when its body is empty/default; full/empty is not inferred from child count. + ## 五、下载边界 `Cache/IMetadataTransfer.cs` 是唯一允许发网络请求的接口： @@ -167,6 +156,8 @@ var revision = client.ImageRevision(root, imageId); 传入 `transfer` 之后，构造函数里的 `httpClient` 不再参与任何请求。 +Composition transport safety does not add a public transfer option: the default HttpMetadataTransfer disables all automatic redirects, including image requests; register final Source/Url addresses. An injected HttpClient must also be configured with AllowAutoRedirect=false. The library rejects a response whose RequestMessage.RequestUri differs, but cannot undo a redirect request already sent by that client. A custom IMetadataTransfer is likewise responsible for not following redirects. + ## 六、地址与安全约束 只接受静态发布文件树。以下情况在写盘前抛 `ArgumentException`：带查询串、含跳转段或空段、含编码的斜杠或反斜杠、Windows 保留设备名、结尾空格或点、正文文件名带 `.etag`/`.tmp`/`.uri` 后缀、带用户信息或非 DNS／IPv4 主机、占用 `origin.json` 或 `.sources` 保留名。 diff --git a/.agents/docs/references/metadata-model.md b/.agents/docs/references/metadata-model.md index 6b03068..2ff83b8 100644 --- a/.agents/docs/references/metadata-model.md +++ b/.agents/docs/references/metadata-model.md @@ -8,8 +8,9 @@ XML 的字段、默认值和校验规则。字段顺序以 [MetadataSchema.xsd]( | 属性 | 含义 | |---|---| -| `SchemaVersion` | 数据契约版本，当前固定 `1.0`。客户端用 `MetadataSchema.IsCompatible` 判断能否读取 | -| `ContentRevision` | 构建时写入的提交标识，客户端据此判断内容是否更新 | +| `SchemaVersion` | 数据契约版本，写在源根 `metadata.xml` 的根节点上，由作者声明，当前固定 `1.0`。客户端用 `MetadataSchema.IsCompatible` 判断能否读取 | + +根节点只有 `SchemaVersion` 一个构建无关的契约属性：构建不注入版本，也不写修订标记，只解析 `${TIMESTAMP}` 等变量。因此指定入口 `metadata.xml` 就能直接读，展平只是把分散的文档合成一份、方便网络传输。 `SchemaVersion` 提高意味着旧客户端可能无法工作。改样式、加可选字段、改默认值都不提高版本。协议版本是 `1.0` 不等于所有旧字段都还在：`Package.Changelogs` 和 `ChangelogSource` 已经删除，没有兼容层，调用方改读实体自己的 `Posts`。 diff --git a/.agents/docs/visuals/metadata-authoring.html b/.agents/docs/visuals/metadata-authoring.html new file mode 100644 index 0000000..b040e87 --- /dev/null +++ b/.agents/docs/visuals/metadata-authoring.html @@ -0,0 +1,774 @@ + + + + + + + 元数据书写与读取模型 + + + + + +
+
+

元数据 · 书写与读取

+

元数据书写与读取模型

+

作者想怎么写就怎么写，读取只认一个入口 metadata.xml。展平与导出只服务于网络传输和把模组信息放在一起，不是读取的前提。

+
+ +
+

三条结论

+
    +
  1. +

    只认一个入口

    +

    MetadataReader.Read 接收入口 metadata.xml，其余文件由 Include + 递归发现。入口可以是本地路径、HTTP(S) 地址或自定义来源，没有第二个读取入口。

    +
    2. +
  2. +

    源树写法自由

    +

    可以按 Mod、应用任意拆分，支持嵌套 Include、Base 继承、资源短 ID 和完整内联清单。读取器在内存里合成一棵规范树。

    +
    3. +
  3. +

    展平不是前提

    +

    build 与 export 把树合成单文件发布物，解析 ${TIMESTAMP} 等变量，便于下载与聚合；源树本身同样能被 + Reader + 直接读取。

    +
    4. +
+
+ +
+

读取路径

+

同一个 Reader 消费两种形态：作者维护的源树，以及构建导出的单文件发布物。展平只是把多份文件变成一份，不改变读取入口，也不改变读取语义。

+
+ + 源树、展平产物与读取器的关系 + 左侧源树通过 Include 组织文件；中间是展平后的单文件发布物；右侧是同一个 MetadataReader 和 + MetadataClient。源树可以直接读取，展平产物也可以直接读取。 + + + + + + + + + + 源树（作者怎么写） + 展平（build / export） + 同一套 Reader / 客户端 + + + metadata.xml（入口） + + + mods/mods.xml + + + apps/apps.xml + + + corona/corona.xml + + + manifests/3258.xml + + + manifests/3300.xml + + + ra3battlenet/app.xml + + + + + + + + + Include + Source + + + Output/metadata.xml + 单文件，无 Include + 展开变量 + 资源 Source 保留 + 网络传输与聚合 + 不是读取前提 + + + MetadataReader + MetadataClient + 规范化 Root + Documents + 磁盘缓存 / 离线可用 + 客户端与桌面端消费 + 两条路径同一模型 + + + build / export + 展平 + + + Output → Reader + 同样可用 + + + 源树 → Reader：直接可用 + +
+

+ Include 关系 + 资源 Source 引用 + 读取 / 构建路径 +

+
+
+
    +
  • 源树 → Reader：MetadataReader.Read("./Metadata/metadata.xml")，入口直接指向源根，不需要先构建。 +
    • +
  • Output → Reader：MetadataReader.Read("./Output/metadata.xml")，或由 + MetadataClient 拉取发布物；展平后的单文件同样由一个入口读取。 +
    • +
  • 两条路径产出同一个规范化模型，最终都由 MetadataClient 与桌面端消费。
    • +
+
+ +
+

三种写法，同一个入口

+

下面的写法都能被同一个入口读取。示例 ID 使用短名（Corona、icon、manifest-3258）：资源的 scope 前缀由 + Reader 与构建统一补齐，实体 ID 保持全仓库唯一的原名。子文件根节点同样写 <Metadata>；SchemaVersion 只要求写在入口根节点上。 +

+
+
+ + + +
+ +
+

入口只负责列出 Include；每个 Mod 或应用独占目录，内部文件继续按需拆分。

+ 
<!-- Metadata/metadata.xml -->
+              <Metadata SchemaVersion="1.0">
+              <Includes>
+              <Include Source="mods/mods.xml" />
+              <Include Source="apps/apps.xml" />
+              </Includes>
+              </Metadata>
+
+              <!-- Metadata/mods/mods.xml -->
+              <Metadata>
+              <Includes>
+              <Include Source="corona/corona.xml" />
+              </Includes>
+              </Metadata>
+
+              <!-- Metadata/mods/corona/corona.xml -->
+              <Metadata>
+              <Image ID="icon" Source="images/icon.png" />
+              <Manifest ID="manifest-3258" Source="manifests/3258.xml" />
+
+              <Mod ID="Corona">
+              <CurrentVersion>3.258</CurrentVersion>
+              <Icon>icon</Icon>
+              <DisplayName Language="zh">日冕</DisplayName>
+              <DisplayName Language="en">Corona</DisplayName>
+              <Packages>
+              <Package Version="3.258">
+              <Manifest>manifest-3258</Manifest>
+              </Package>
+              </Packages>
+              </Mod>
+              </Metadata>
+
+ +
+

全部内容放在入口文件里。资源 Source 相对当前文档解释，因此这里从源根算起。

+ 
<!-- Metadata/metadata.xml：没有 Include，所有节点平铺 -->
+              <Metadata SchemaVersion="1.0">
+              <Image ID="icon" Source="corona/images/icon.png" />
+              <Manifest ID="manifest-3258" Source="corona/manifests/3258.xml" />
+
+              <Mod ID="Corona">
+              <CurrentVersion>3.258</CurrentVersion>
+              <Icon>icon</Icon>
+              <DisplayName Language="zh">日冕</DisplayName>
+              <DisplayName Language="en">Corona</DisplayName>
+              <Packages>
+              <Package Version="3.258">
+              <Manifest>manifest-3258</Manifest>
+              </Package>
+              </Packages>
+              </Mod>
+              </Metadata>
+
+ +
+

完整清单直接写在 Manifest 里（无 Source），外部清单只写 + Source。两种形式可以共存，Package 里都只写短名。 +

+ 
<!-- Metadata/metadata.xml -->
+              <Metadata SchemaVersion="1.0">
+              <Includes>
+              <Include Source="mods/corona/corona.xml" />
+              </Includes>
+              </Metadata>
+
+              <!-- Metadata/mods/corona/corona.xml -->
+              <Metadata>
+              <Image ID="icon" Source="images/icon.png" />
+
+              <!-- 完整内联清单：没有 Source，随 Include 一起保留 -->
+              <Manifest ID="manifest-3258" HashAlgorithm="None">
+              <File Mount="base">
+              <FileName>Corona.big</FileName>
+              <RelativePath>Corona</RelativePath>
+              <KindOf>big</KindOf>
+              </File>
+              </Manifest>
+
+              <!-- 外部清单：有 Source，只是引用 -->
+              <Manifest ID="manifest-3300" Source="manifests/3300.xml" />
+
+              <Mod ID="Corona">
+              <CurrentVersion>3.258</CurrentVersion>
+              <Icon>icon</Icon>
+              <DisplayName Language="zh">日冕</DisplayName>
+              <DisplayName Language="en">Corona</DisplayName>
+              <Packages>
+              <Package Version="3.258">
+              <Manifest>manifest-3258</Manifest>
+              </Package>
+              <Package Version="3.300">
+              <Manifest>manifest-3300</Manifest>
+              </Package>
+              </Packages>
+              </Mod>
+              </Metadata>
+
+
+
+ +
+

版本契约

+

SchemaVersion 是作者声明的数据契约，构建不再替你注入，也不写任何修订标记。

+
+ + + + + + + + + + + + + + + + + +
字段写在哪谁写入缺失或版本不符
SchemaVersion="1.0"源根入口 metadata.xml 的根节点作者手写；构建不再注入构建直接失败；MetadataClient 拒读该根清单
+
+

MetadataSchema.IsCompatible 只接受当前版本 + 1.0。版本提高意味着旧客户端可能无法工作；改样式、加可选字段、改默认值都不提高版本。缺 SchemaVersion 时构建的报错文本是「源根 metadata.xml + 的根节点缺少 SchemaVersion 属性」。 +

+
+ +
+

读取细节

+
    +
  • 递归捕获 Include。入口加载后按 Include@Source 继续递归；路径相对当前文件。入口与 Include 都可以是 + HTTP(S) 地址，但 HTTP 文档不能引用本地文件。
    • +
  • 菱形 Include 只展开一次。同一份文档只捕获一次，内容只贡献一次；循环引用直接报错，不做静默跳过。
    • +
  • 路径限制在源根内。入口、Include 与资源 Source 都必须落在 + SourceRoot（缺省为入口所在目录）内，越界抛 InvalidOperationException。 +
    • +
  • 缺 SchemaVersion 或入口不存在就直接报错。构建在源根硬失败；客户端对缺 SchemaVersion + 的根清单报「根清单缺少 SchemaVersion」并拒读；入口文件不存在抛 FileNotFoundException。没有默认版本注入，也没有兜底路径。
    • +
  • 读取只在内存中合并。Reader 合并 Include、继承与资源引用，不写盘、不替换变量、不转换图片，也不改写已声明的 + SchemaVersion。 +
    • +
  • 引用校验是显式选项。MetadataSourceOptions.ValidateReferences 开启后才检查断引用；默认关闭。
    • +
+
+ +
+

为什么这样设计

+

作者侧不规定文件数量、嵌套层数、短名还是完整 ID。拆成多少文件、要不要内联清单，都是作者的编辑选择，不构成读取契约的一部分。

+

客户端与构建共用同一个 + MetadataReader：构建读源树，客户端读的是同一个入口——源树或展平产物都行，展平只是同一张图的另一种布局。这样就不存在「源树一套语义、发布物另一套语义」的分叉，也不会出现必须先构建才能读取的隐含规则。 +

+
+ +
+

+ 静态说明页，无外部依赖。实现位置：Ra3.BattleNet.Metadata/Reading/MetadataReader.cs、Ra3.BattleNet.Metadata/Cache/MetadataClient.cs。 +

+
+
+ + + + diff --git a/.agents/notes/implemented/architecture/2026-10-04-metadata-xml-runtime.md b/.agents/notes/implemented/architecture/2026-10-04-metadata-xml-runtime.md index f0c4ad0..f5c3166 100644 --- a/.agents/notes/implemented/architecture/2026-10-04-metadata-xml-runtime.md +++ b/.agents/notes/implemented/architecture/2026-10-04-metadata-xml-runtime.md @@ -2,6 +2,8 @@ Status: implemented +Related decision: [one composition core and layout-only exports](../../implemented/architecture/2026-11-07-composable-metadata.md) partially overlaps the runtime root boundary: complete composed graphs and inline package access reuse transfer/cache ownership. This historical strict-parser Decision and its evidence are preserved, not silently rewritten. + ## Problem `MetadataBuilder.Load` 的 HTTP 路径每次下载并解析临时文件，不保存运行时缓存。Desktop 的应用、模组和更新检查各自读取同一份根目录，重复下载；断网时也没有根目录可供展示。包清单登记存根不是完整文件表，Updater 的增量清单也不是包清单，不能把三者当成同一种模型。 diff --git a/.agents/notes/implemented/architecture/2026-10-09-registered-resource-cache-cadence.md b/.agents/notes/implemented/architecture/2026-10-09-registered-resource-cache-cadence.md index 86d74d2..83f1658 100644 --- a/.agents/notes/implemented/architecture/2026-10-09-registered-resource-cache-cadence.md +++ b/.agents/notes/implemented/architecture/2026-10-09-registered-resource-cache-cadence.md @@ -2,6 +2,8 @@ Status: implemented +Related decision: [one composition core and layout-only exports](../../implemented/architecture/2026-11-07-composable-metadata.md) adds snapshot/package access for inline manifests while preserving cache-only display, explicit refresh and image cadence. This Decision is not reversed. + ## Problem 普通展示如果在每次读取时发 HTTP，或在命中图片缓存后再做后台条件校验，页面轮询就会放大请求。图片虚拟地址只带登记 ID 和 revision；进程重启后若 revision 回到相同起点，或同一 ID 更换 Source 后仍得到相同 revision，网页会复用旧图。 diff --git a/.agents/notes/implemented/architecture/2026-10-12-metadata-transfer-to-local-file.md b/.agents/notes/implemented/architecture/2026-10-12-metadata-transfer-to-local-file.md index 2b139d8..1f9fea8 100644 --- a/.agents/notes/implemented/architecture/2026-10-12-metadata-transfer-to-local-file.md +++ b/.agents/notes/implemented/architecture/2026-10-12-metadata-transfer-to-local-file.md @@ -2,6 +2,8 @@ Status: implemented +Related decision: [one composition core and layout-only exports](../../implemented/architecture/2026-11-07-composable-metadata.md) acquires composed graphs through the same IMetadataTransfer/staging boundary. Transfer ownership, validators and cancellation in this Decision remain applicable. + ## Problem 库原先自己内联 `HttpClient`，把响应正文读成字节再写缓存：传输不可替换（宿主无法插手代理、重试或平台网络栈），也没法“先把正文落到本地再解析”。服务端不稳时，宿主想要的是可替换的传输边界与一条“完整正文才发布”的路径，而不是库内部硬编码的字节中转。 diff --git a/.agents/notes/implemented/architecture/2026-11-07-composable-metadata.md b/.agents/notes/implemented/architecture/2026-11-07-composable-metadata.md new file mode 100644 index 0000000..02779e1 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-11-07-composable-metadata.md @@ -0,0 +1,75 @@ +# Agent Note: One composition core with layout-only exports + +Status: implemented + +Partially superseded by [single metadata format](../../implemented/simplification/2026-11-07-single-metadata-format.md): the user removes dual-format detection, legacy read APIs/source behavior and default legacy publication. The shared composition/layout/cache decision remains relevant. The Decision and verification below describe the earlier contract and are not evidence for the new cutover. + +## Problem + +The earlier unified-reader entry point treated Include detection as a choice between author and published semantics. A single author document can contain inheritance or short resource IDs without Include; a generated split document can already contain canonical IDs. Guessing from Include, a short ID or SchemaVersion alone cannot distinguish these cases. Export and runtime consumers must not each reinvent qualification, inherited reference resolution or Source rebasing. + +The refactor must simplify the existing implementation, not preserve an old flattener under a new reader and then add another exporter/cache resolver. Flattened and split documents express the same metadata; their difference is organization. Existing author files, schemas, business IDs, public model/query/image APIs and default legacy publication remain compatible. + +## Decision + +One Reader composition core serves Open, explicit legacy LoadSource, the public Flattener compatibility methods, and composed runtime roots. Load/Parser retain strict published-document boundaries. Required legacy public methods are thin delegates; superseded internal composition branches are removed. + +The Reader returns a normalized legacy-ID model/document plus captured raw documents, Include edges and normalized own-node provenance. It owns composition, inheritance, typed reference resolution and Source normalization, but never writes output, evaluates variables or converts images. The cache fetches XML asynchronously through the existing transfer boundary and supplies a captured memory source; it does not implement ID resolution or inheritance. + +Exporter consumes this result. One layout algorithm selects which Include edges remain external: Flat retains none, Tree retains the document organization, Mixed retains the selected subset. It does not parse and compose each layout independently. SourceExporter shares author-manifest element normalization without serialize/parse roundtrips. Default legacy Build uses the same output core through ExportLegacy without generated format markers; new tree/mixed export is a separate operation. + +The old Flattener body and Open's separate IO branches are replaced by delegates to Reader. Build no longer owns a duplicate resource-copy/layout path. Composition, captured provenance and complete-result caching share the semantic core rather than adding independent parsers. Desktop's dependent removal of URI glue remains under its separate proposed adaptation until default-submodule verification. + +The full file responsibilities, interfaces, six phases and acceptance matrix remain in [the implementation plan](../../../docs/exec-plans/2026-11-07-composable-metadata.md). Metadata implementation is verified; Desktop integration and dependency delivery remain pending and are not implied by this note's status. + +## Identity and address contract + +- Resource identity is `(logical document scope, local ID)`, compared OrdinalIgnoreCase. A single codec supports legacy `scope:local` and reversible path `scope/local` encodings. Keep existing `QualifyId`, `LocalId` and `PathPrefix` signatures/legacy results. Physical relocation does not redefine encoded scope. +- Document identity is separate: HTTP paths remain case-sensitive; local paths follow filesystem platform rules. Repeated documents deduplicate; cycles and distinct duplicate canonical registrations fail, and ambiguous aliases remain ambiguous. Reference resolution retains existing field-specific kind rules, including Icon support for Image or Markdown. +- Generated exports explicitly declare `IdFormat="path"` or `IdFormat="legacy"` and `SourceBase="document"`. Reject unknown declared formats. Do not modify author XML/XSDs. Path-format IDs decode to legacy canonical IDs in the ordinary model. +- Preserve unmarked old flat IDs and Source when no composition construct exists. Qualify author composition once, with explicit compatibility options where the input is otherwise ambiguous. Do not infer asset addresses from ID strings. +- Legacy unmarked flat HTTP documents retain existing file Source values for images/Markdown; author/normalized composition containment checks do not retroactively constrain that published-document contract. Local no-disk cached-leaf access uses Reader with an offline XML source to decode path IDs and expand local Includes, rather than strict Parser. Missing cached HTTP XML returns Unavailable instead of starting network access. These compatibility corrections remain within this same composition decision. +- The unified result exposes Source relative to the entry URI, while explicit legacy LoadSource preserves SourceRoot-relative Source and legacy manifest stub semantics. Default unmarked author-included full manifests become stubs; marked normalized full manifests remain inline regardless of organization. SourceRoot is a containment boundary, not a license to rewrite a published document's logical scope. +- Preserve Image/Markdown/Manifest Source and URL-only semantics across layouts. Check output collisions and traversal before writing. A flat input lacks original topology/comments; deterministic regrouping is not lossless restoration. + +## Complete snapshots and package access + +Snapshot/package Get/GetCached/Refresh operations coexist with compatible URI-oriented APIs. Inline manifests come from the captured root with null SourceUri and Status inherited from the snapshot in all three operations: a complete Fresh root permits Fresh inline data without a separate request, but Stale is never elevated. External cached-only access stays Stale. External registration resolution uses the same frozen snapshot and Source identity. Preload handles both forms without independently refreshing inline data or inventing URIs. + +A successful runtime root represents a complete composed graph. Root 304 still revalidates included documents. A normalized root and minimal private graph metadata publish only after all composition inputs succeed; failure preserves the previous complete root and offline reopening uses no network. Public root/leaf/image URI mappings and image cadence remain unchanged. Raw validators stay associated with raw bytes, not normalized serialization. First flat-to-composition migration preflights graph paths and preserves the previous complete materialized snapshot before raw replacement. Default transfer disables redirects; injected handlers/custom transfers must do likewise because response checks cannot undo an already-sent redirect. No public transfer option or general store/lease/multiprocess framework is added. + +## Related decision audit + +The active implemented/rejected notes were searched for unified reading, local source, SourceExporter and cache behavior before proposing this note. No prior Decision is reversed in place; historical verification remains evidence for that earlier change, not this refactor. + +| Prior note | Classification | Preserved boundary and change | +|---|---|---| +| [Unified reader](../../implemented/feature/2026-11-01-unified-metadata-reader.md) | Partial overlap; crosslinked | Keep unified Open, XML source abstraction, strict Load and containment. Replace Include-only semantic branching with explicit format-aware single composition. Preserve its historical decision and verification. | +| [Local source loading](../../implemented/feature/2026-10-28-local-mod-source-load.md) | Partial overlap; crosslinked | Keep pure in-memory author reading, explicit SourceRoot-relative compatibility and stubs. Share the core rather than a second LoadSource pipeline. | +| [Source-format host API](../../implemented/feature/2026-10-31-source-format-host-api.md) | Partial overlap; crosslinked | Keep library-owned short-ID author declarations, standalone manifest parsing and no claimed topology recovery. Reuse applicable core in the broader layout exporter. | +| [XML runtime cache](../../implemented/architecture/2026-10-04-metadata-xml-runtime.md) | Partial overlap; crosslinked | Keep transfer ownership, last-valid publication and snapshot-bound identity; extend roots to captured composed graphs and add package access. The old Include prohibition remains the historical strict-parser decision, not an instruction to reject Reader composition. | +| [Registered-resource cadence](../../implemented/architecture/2026-10-09-registered-resource-cache-cadence.md) | Partial overlap; crosslinked | Keep cache-only display, explicit refresh and independent image behavior. Add equivalent package access for inline manifests without network-freshness fiction. | +| [Transfer to local file](../../implemented/architecture/2026-10-12-metadata-transfer-to-local-file.md) | Partial overlap; crosslinked | Keep IMetadataTransfer, validators, staging, ownership and cancellation; graph capture uses this boundary rather than synchronous HTTP in the cache. | +| [Rejected runtime-cache framework](../../rejected/architecture/2026-10-02-runtime-metadata-cache.md) | Related rejection remains valid; crosslinked | Do not restore leases, multiversion object storage, multiprocess pointers or a MetadataRuntime framework. Minimal complete-result persistence is not that framework. | + +Other notes found through shared words concern data model/language/style or deployment, not the composition decision. They are unchanged. None of the audited notes is fully absorbed or suitable for archival/deletion by this decision. + +## Alternatives considered + +- **Keep Include-only detection and extend each consumer**: smallest apparent reader change and no new captured-result contract. It cannot distinguish single-file author semantics from normalized split output, and spreads composition into exporter/cache/Desktop. Reject in favor of one semantic core. +- **Add Reader while retaining the old flattener**: initially minimizes risk to legacy calls. It creates two implementations whose ID, inheritance and Source behavior must remain synchronized indefinitely. Replace the implementation and delegate only required legacy public entry points. +- **Use three independent Flat/Tree/Mixed pipelines**: each layout is easy to reason about locally. This repeats parsing/resolution, makes Mixed a separate dialect, and contradicts organization-only flattening. Use retained-edge selection over one captured graph. +- **Rewrite all publication to the new tree/path format**: one output dialect would reduce long-term exporter branches. Existing binaries and published layout are required compatibility boundaries; retain legacy Build and make new composition export explicit. +- **Restore the rejected general cache framework**: could model graph generations and multi-process readers. The task needs the last complete result, not leases, stores, cleanup policy or another state abstraction; keep persistence minimal. + +## Consequences + +Flattening is an organization choice over one model rather than a second interpretation. Shared composition/layout eliminates independent ID and Source rules; captured provenance supports deterministic export without rereading disposed HTTP sources. Explicit format markers introduce an output dialect old binaries do not understand, so legacy Build stays unchanged and new Tree/Mixed artifacts require the Reader. Captured XML/provenance costs memory; flat input still cannot recover original comments/topology. The cache retains only the last complete result, not leases or multiversion storage. Broader storage/format options require a new demonstrated consumer need rather than extension for its own sake. + +## Verification + +The parent ran the full Metadata suite before final simplification: 442 passed, 0 failed, with one existing test CS8602 warning and no production warning. After simplification, the affected Reader/Exporter/Cache/Transfer checks passed 86/86 with 0 failures and 0 warnings. Following Desktop integration's compatibility corrections, the affected Reader/CompositionCache/MetadataClient checks passed 68/68 with 0 failures and 0 warnings, and the actual Metadata smoke passed again. These are three distinct runs, not counts to add together or a claim of a later full-suite run. + +Actual legacy CLI Build succeeded from immutable repository data. Actual repository-tree export produced 53 XML documents; Reader reread confirmed 15 Mods, 3 Applications, 50 Image identities and all resource files. The external real smoke exercised six Flat/Tree/Mixed and inline false/true combinations plus HTTP root 304/child update, retained old snapshot, child-failure Stale fallback and cold-client offline package access. The parent reran the actual smoke and legacy CLI after simplification; both passed. + +Security source re-review reported no remaining findings for the two identified vulnerabilities. Executed regression coverage includes redirect rejection and migration-preflight/cold-offline; it does not simulate a crash or mid-write IO failure. The parent previously ran docs generation/checks successfully and will refresh after lifecycle/link changes. Desktop build/service acceptance and pushed-submodule dependency delivery remain pending in the implementation plan. diff --git a/.agents/notes/implemented/architecture/2026-11-09-authored-schema-version.md b/.agents/notes/implemented/architecture/2026-11-09-authored-schema-version.md new file mode 100644 index 0000000..6637359 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-11-09-authored-schema-version.md @@ -0,0 +1,39 @@ +# Agent Note: SchemaVersion 由源根声明 + +Status: implemented + +## Problem + +此前 `SchemaVersion` 只由构建注入：`MetadataBuilder.Build` 展平时给每份输出 XML 的根节点写 `SchemaVersion` 和 `ContentRevision`，源树自己不带这个属性。客户端读根清单时用 `CatalogError` 检查 `SchemaVersion`，缺了直接判 `Unavailable`。 + +结果是“指定入口就能读”这条规则只对发布物成立：作者写好的源树 `Metadata/metadata.xml` 必须先跑一遍构建才能被客户端消费，而构建只是把分散文档合成一份、方便网络传输的步骤。作者想直接把自己的源树指给客户端调试，或者想让源树本身就是可读的单一入口，都做不到。同时命令行还要单独传 `--schema-version=` 决定输出写什么版本，契约版本与作者看到的数据是两处。 + +## Decision + +- `SchemaVersion` 改为作者写在源根 `metadata.xml` 的根节点上（``），作为数据契约版本的唯一来源。 +- `MetadataBuilder.Build` 不再注入 `SchemaVersion`；导出复制根节点属性，源上写的版本原样进发布物。构建期标记（`ContentRevision`）随后被 [移除构建期修订标记](2026-11-09-drop-content-revision.md) 一并删除。 +- 删除 `Build` 的 `schemaVersion` 参数和命令行 `--schema-version=`。`MetadataBuilder.DefaultSchemaVersion` 保留，`MetadataSchema.Current` 继续引用它，作为库的期望值。 +- 构建在读入口后立即检查 `SchemaVersion` 属性，缺失时报「源根 metadata.xml 的根节点缺少 SchemaVersion 属性」，构建失败并清空 `Output/`。 +- `MetadataSchema.xsd` 的根类型显式声明可选 `SchemaVersion` 属性，保留 `anyAttribute`，这样被 `Include` 的片段不必带版本。 +- 源树与展平产物共用 `MetadataReader` / `MetadataClient` 同一条读取路径；构建不再是读取前置条件。 + +## Alternatives considered + +- **继续由构建注入，只给客户端加“源树宽松模式”**：客户端不用改，源树也能读；但等于让客户端接受没有契约声明的数据，版本兼容判断失去依据，还会把“发布物严格、源树宽松”两套语义带回来。 +- **把 `SchemaVersion` 放进 `Tags` 子元素**：能复用现成变量结构；但发布物 schema 已要求它是根属性，同语义两处放法，客户端还得两种都认。 +- **命令行显式覆盖，源文件可选**：保留 `--schema-version=` 可以临时发别的版本；但契约版本属于作者声明，构建期覆盖会让同一份源产出不同契约，且与“源树可直接读”互相矛盾。 +- **要求每份被 Include 的片段都写 `SchemaVersion`**：校验更一致；但源树拆分本就是为了自由书写，片段带不带版本对读取没有意义，没有必要加这条约束。 + +## Consequences + +- 作者侧新增一条硬约束：入口 `metadata.xml` 根节点必须写 `SchemaVersion`，否则构建失败；被 Include 的片段不需要写。 +- 直接把 `RA3BN_METADATA_URL` 指向源树 `Metadata/metadata.xml` 即可被桌面端读取（Debug），不必先构建；源树里的外部清单按需从同一源根懒加载。 +- 发布物根节点的 `SchemaVersion` 来自源文件，改写源属性会直接改掉对外契约。 +- 旧的构建命令带 `--schema-version=` 会报未知参数，需要删掉该参数；`MetadataBuilder.Build` 的调用方按新签名 `Build(src, dst, convertImages)` 调整。 + +## Verification + +- `dotnet test Metadata.sln` 全绿。 +- 新增回归测试：源根缺少 `SchemaVersion` 时 `Build` 抛出含「缺少 SchemaVersion」的 `InvalidOperationException`；等价性测试改为断言源声明被原样保留到发布物。 +- 真实 CLI 构建到 `Output/`，产物根节点 `SchemaVersion=1.0`，源 XML、XSD 不进输出。 +- 临时探针用 `MetadataClient.RefreshRootAsync` 直接读源树 `Metadata/metadata.xml` 返回 `Fresh`，与读 `Output/metadata.xml` 结果一致。 diff --git a/.agents/notes/implemented/architecture/2026-11-09-drop-content-revision.md b/.agents/notes/implemented/architecture/2026-11-09-drop-content-revision.md new file mode 100644 index 0000000..fba5205 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-11-09-drop-content-revision.md @@ -0,0 +1,43 @@ +# Agent Note: 移除构建期修订标记 ContentRevision + +Status: implemented + +## Problem + +`ContentRevision` 是构建时写进发布物根节点的提交标识，文档里一直写成“客户端据此判断内容是否更新”。核对代码后没有这个消费者： + +- `MetadataBuilder` 写它，`ValidateHard` 只检查它非空，CLI 构建完打印一次。 +- 缓存与更新判断走的是 ETag、Last-Modified、整图 SHA-256（`GraphDigest`）和 composition 清单；`UpdateService` 比的是 `Application` / `Updater` 的 `Version`。 +- 桌面端 `AppMetadataReader` 把它塞进 `AppMetadataSnapshot.ContentRevision`，这个字段全仓没有读取点；`MainWindow` 只用版本、公告和状态。 +- `ModLibraryStore` 写本地 `mod.xml` 时反而把它删掉。 +- 部署流水线把 `github.sha` 传进构建，产物里那一行没有任何下游校验。 + +结果是发布契约里多了一个必填属性、命令行多一个参数、CI 多一段传递，读的人还以为它能判断更新。 + +## Decision + +- 删除 `ContentRevision`：`MetadataBuilder.Build` 不再接收 `contentRevision`，也不再写这个属性；签名变成 `Build(src, dst, convertImages)`。 +- 删除 `ValidateHard` 对它的必填检查、`Program` 的 `--content-revision=` 参数与打印、`build.sh` 的参数转发、`deploy.yml` 里的 `--content-revision=${{ github.sha }}`。 +- 发布物 `MetadataPublishSchema.xsd` 不再要求该属性；源树 XSD 注释同步说明构建不追加任何版本或修订属性。 +- 桌面端删掉 `AppMetadataSnapshot.ContentRevision` 字段与读取；`ModLibraryStore.StripRuntimeNodes` 只清理 `SchemaVersion`。 +- `Tags/Commit` 的 `${TIMESTAMP}` 保留：那是作者写在源里的时间戳字段，构建负责解析，属于数据而不是标记。 +- 编辑器判断“这是发布物”的探针从 `ContentRevision` 改成 `Tags/Commit` 是否已被替换成时间戳。 + +## Alternatives considered + +- **保留为可选属性，文档改成“仅供人工排查”**：改动最小，也能定位发布物来自哪次提交；但没有人读、没人校验，留着仍然会被误认为契约的一部分，而且部署那一段传递链也没有意义。 +- **改成 HTTP 响应头或构建日志输出**：能保住提交追溯，又不污染数据契约；但需要服务端配合，现阶段排查直接看 CI 记录就够，没必要新增通道。 +- **连 `Tags/Commit` 一起删**：时间戳同样没有代码读取；但它是作者可写的数据字段、构建只在解析 `${TIMESTAMP}`，删它超出这次“删标记”的范围，留下。 + +## Consequences + +- 发布物根节点只剩 `SchemaVersion` 一个版本属性；源树与发布物在这点完全同构。 +- 旧的构建命令带 `--content-revision=` 会报未知参数；`MetadataBuilder.Build` 调用方按 `Build(src, dst, convertImages)` 调整。 +- 桌面端不再有 `ContentRevision` 字段；`AppMetadataSnapshot` 构造参数少一项。 +- 需要追溯发版时看 CI 的运行记录，而不是拉 `metadata.xml`。 + +## Verification + +- `dotnet test Metadata.sln` 全绿；等价性测试断言发布物根节点没有 `ContentRevision` 且 `SchemaVersion` 原样保留。 +- 真实 CLI 构建到 `Output/`：根节点为 ``，无修订属性。 +- 桌面端测试全绿，子模块固定到该提交后仍能直接读取源树 `Metadata/metadata.xml`（`Fresh`）。 diff --git a/.agents/notes/implemented/bug-fix/2026-11-07-resource-id-keys.md b/.agents/notes/implemented/bug-fix/2026-11-07-resource-id-keys.md new file mode 100644 index 0000000..4a56bf6 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-11-07-resource-id-keys.md @@ -0,0 +1,34 @@ +# Agent Note: Preserve resource registration keys without colon policing + +Status: implemented + +## Problem + +The [single-format decision](../simplification/2026-11-07-single-metadata-format.md) incorrectly equated a single read protocol with a mandatory path-shaped registration key. The current production document failed before lookup on `mods/corona/manifests/3.258:manifest-3258`, even though the registration, reference and external body share that exact key. The user explicitly requires removing the colon ban and unnecessary ID checks. + +## Decision + +Remove resource/entity colon rejection and path-segment validation from ID qualification. Existing full keys containing slash or colon are preserved verbatim. Newly scoped short IDs still use escaped scope/local to prevent collisions across definitions. Reference rewriting checks the complete registration set before scoped aliases. Keep duplicate/ambiguous-registration checks and exact external Manifest identity. ID strings are not Source addresses: actual file/URI containment and HTTP/local restrictions remain unchanged. No colon decoder, separator conversion, format detector or compatibility API is added. + +The follow-up defensive-code review found LocalId still manually rejected non-hex percent sequences while building aliases, preventing exact-key lookup. Remove that hand-written escape validator; Uri.UnescapeDataString still decodes generated short-name escapes without rejecting literal percent text. Also remove OfflineSource's unused disk parameter and unreachable remote raw-cache fallback. The source remains local-only and uses the existing shared Reader path containment; no network read or complete-snapshot fallback behavior changes. + +## Alternatives considered + +- **Require republishing production with path IDs**: produces uniform-looking keys but rejects valid exact registrations for no consumer benefit. The user rejects this restriction. +- **Convert colon keys to slash keys**: makes existing keys look generated but mutates identity and requires conversion across references and external bodies. Preserve exact strings instead. + +## Consequences + +Current production is readable without republishing XML. There is still one read algorithm and exact key lookup, not an old-format mode. The earlier colon-rejection decision and its associated rejection-only tests are superseded; path validation remains on actual addresses. This change does not modify source metadata, XSDs or the installed RisingStorm declaration. + +## Verification + +- Regression exercises colon-only Image, slash/colon Markdown, colon entity ID and an external Manifest with exact registration/body identity, including inline resolution and reference validation. +- Focused Reader/ID suite: 34 passed. Full Metadata suite: 426 passed, zero failures. Four obsolete invalid-ID rows were removed and one behavior regression added; totals are not added across runs. +- Actual `MetadataReader.Read` against `https://metadata.ra3battle.cn/metadata.xml` with reference validation succeeded: 15 Mods, 3 Applications, 50 image projections and 20 Markdowns. The failing production key remains unchanged. +- Actual MetadataClient root refresh and package preload loaded all 22 of 22 production Manifest bodies, zero failures. +- Actual installed RisingStorm mod.xml read with reference validation and MetadataSourceManifest.Resolve succeeded: current version 077R, full Manifest with RisingStorm_077R.big. +- No BIG download/hash validation, GUI or game launch is claimed. +- Follow-up full suite after both review fixes: 428/428 passed. Existing full-ID regression now covers literal percent and escaped-percent suffixes as well as colon registrations and exact external body identity. +- Actual probe: scope:icon%raw, scope/icon%raw and scope:icon%20 all retain exact registration/reference identity. Production root/reference validation and 22/22 external manifests passed again; installed RisingStorm declaration/full Manifest passed again. +- Actual no-disk-cache local package read expanded a leaf Include and resolved a scope:manifest%raw body successfully. Existing cache regressions also cover local file changes and remote Include rejection without network access. diff --git a/.agents/notes/implemented/bug-fix/2026-11-09-leaf-stub-source-loop.md b/.agents/notes/implemented/bug-fix/2026-11-09-leaf-stub-source-loop.md new file mode 100644 index 0000000..d5e6679 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-11-09-leaf-stub-source-loop.md @@ -0,0 +1,37 @@ +# Agent Note: 叶子清单不再追自己的存根 Source + +Status: implemented + +## Problem + +`MetadataClient.FetchLeafDocumentAsync` 读叶子清单时打开了 `InlineManifests`，`CaptureGraphAsync` 会把文档里所有带 `ID` + `Source` 的 `Manifest` 当成待内联的正文继续抓取。团队为此写的缓存测试 `Leaf_RejectsStubCycleAndMissingFile_AcceptsEmptyManifest` 用一个“叶子清单本身只有 ``”的存根来验证拒绝路径： + +- 存根被当成待内联清单，`Source` 按叶子自身地址解析，得到 `apps/apps/leaf.xml`； +- 那个地址返回同一份存根，再解析成 `apps/apps/apps/leaf.xml`……地址无限增长，请求无限打。 + +Windows 上缓存目录路径很快超过 MAX_PATH，抛 `IOException` 被兜住，测试表现为“快速失败”；Linux 没有路径长度限制，于是在 CI 里跑到超时。CI 只挂在 Ubuntu，本地 Windows 全绿，正是这个原因。 + +## Decision + +- 删掉 `CaptureGraphAsync` 的 `inlineManifests` 参数与 Manifest 追踪分支；它只保留 `Include` 递归。 +- 读叶子清单不再打开 `InlineManifests`：叶子本身就是正文目标，里面出现带 `Source` 的 `Manifest` 只可能是存根，交给 `TryAcceptLeaf` 直接报「叶子清单是登记存根」。 +- 根清单读取、`MetadataReader` 的 `InlineManifests` 选项与导出流程不变。 +- 补回归测试 `Leaf_StubSource_IsRejectedWithoutFollowingItself`：断言只发根清单 + 目标叶子两次请求，状态为 `Unavailable`，错误含「存根」。 + +## Alternatives considered + +- **在追踪处判断目标地址与当前文档相同就停下**：能挡住自引用；但地址每层都会变（`apps/apps/...`），单点比较挡不住增长链。 +- **按 Manifest ID 去重**：自引用场景能停；但存根本身就该拒绝，靠去重只是把它变成“第二次遇到才停”，语义仍然别扭。 +- **只限制最大抓取深度**：能防死循环；但把错误变成“深度超限”，不如直接按存根拒绝，且仍会发出多余请求。 + +## Consequences + +- 存根叶子现在一次请求就判失败，不再放大成无限请求链。 +- 客户端不再把叶子里的 `Manifest@Source` 当正文追踪；发布契约本来也不允许叶子同时带 body 与 Source。 +- Linux 与 Windows 行为一致；缓存测试在两条平台上都在秒级完成。 + +## Verification + +- WSL Ubuntu + .NET 10 全量测试：430 通过，10 秒完成（修复前 3 分钟无进展、被 hang 探针中止）。 +- Windows 全量测试：430 通过。 +- 回归测试确认请求次数为 2，错误为存根拒绝。 diff --git a/.agents/notes/implemented/feature/2026-10-28-local-mod-source-load.md b/.agents/notes/implemented/feature/2026-10-28-local-mod-source-load.md index c97af18..e23f879 100644 --- a/.agents/notes/implemented/feature/2026-10-28-local-mod-source-load.md +++ b/.agents/notes/implemented/feature/2026-10-28-local-mod-source-load.md @@ -2,6 +2,8 @@ Status: implemented +Related decision: [one composition core and layout-only exports](../../implemented/architecture/2026-11-07-composable-metadata.md) shares this local-source boundary. Explicit legacy LoadSource keeps SourceRoot-relative Source and stub compatibility; the shared core does not reverse this historical Decision. + ## Problem Desktop 需要直接读作者磁盘上的 Mod 源文件（`corona.xml`、独立清单、管理目录里的 `mod.xml`），但库只提供两条路：`MetadataBuilder.Build` 要求整棵源树、XSD 与输出目录，`MetadataParser.Load` 又只接受已展平产物。宿主只能先跑一次构建或自己拼 XML，既写盘也绕开既有展平规则。 @@ -17,7 +19,7 @@ Desktop 需要直接读作者磁盘上的 Mod 源文件（`corona.xml`、独立 - 新增 `MetadataParser.Load(XDocument)`，内存文档无需临时文件即可解析，仍然拒绝 `Include`/`Includes`/`Module` 与非法根节点。 - `MetadataBuilder.Load` / `LoadFromFile` 的严格语义不变：发布物入口依旧拒绝源树。 -规则写进 [library-api.md](../../../docs/references/library-api.md)，实现在 [MetadataBuilder.cs](../../../../Ra3.BattleNet.Metadata/MetadataBuilder.cs) 与 [MetadataFlattener.cs](../../../../Ra3.BattleNet.Metadata/MetadataFlattener.cs)。 +当时的规则记录于 [library-api.md](../../../docs/references/library-api.md)。当前 [MetadataBuilder.cs](../../../../Ra3.BattleNet.Metadata/Building/MetadataBuilder.cs) 仅负责构建；本篇当时的 LoadSource/MetadataFlattener 读取实现已移除，迁移边界见 [single-format 决定](../../implemented/simplification/2026-11-07-single-metadata-format.md)。上文保留原决定的历史语义，不表示旧 API 仍存在。 ## Alternatives considered diff --git a/.agents/notes/implemented/feature/2026-10-31-source-format-host-api.md b/.agents/notes/implemented/feature/2026-10-31-source-format-host-api.md index 07bd3f3..668ac2b 100644 --- a/.agents/notes/implemented/feature/2026-10-31-source-format-host-api.md +++ b/.agents/notes/implemented/feature/2026-10-31-source-format-host-api.md @@ -2,6 +2,8 @@ Status: implemented +Related decision: [one composition core and layout-only exports](../../implemented/architecture/2026-11-07-composable-metadata.md) extends library-owned export without duplicating ID/Source rules. This single-Mod author-format Decision and its topology-recovery limit remain independent. + ## Problem `LoadSource` 让宿主能读作者源文件之后，Desktop 仍自己实现了三类源格式逻辑：把在线展平树反向导出成单 Mod 作者声明（`ModSourceDeclaration`，315 行）、按 stub 找叶子清单并与 `LocalId` 对齐（`ResolveManifestNode`）、把独立 `Manifest` 节点包一层假 `Metadata` 根再解析。 diff --git a/.agents/notes/implemented/feature/2026-11-01-unified-metadata-reader.md b/.agents/notes/implemented/feature/2026-11-01-unified-metadata-reader.md new file mode 100644 index 0000000..57cd983 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-11-01-unified-metadata-reader.md @@ -0,0 +1,46 @@ +# Agent Note: 统一元数据读取入口 + +Status: implemented + +Related decision: [one composition core and layout-only exports](../../implemented/architecture/2026-11-07-composable-metadata.md) partially overlaps this decision. It replaces Include-only branching while preserving strict Load, source abstraction and historical evidence; this Decision is not rewritten. + +## Problem + +元数据读取此前存在两条独立路径：`MetadataBuilder.Load` 只接受已展平的发布物，遇到 `Include` 直接报错；`MetadataBuilder.LoadSource` 只支持本地作者源树，而完整构建 `Build` 又要求整棵源树与输出目录。宿主若想在同一个调用点兼顾作者源文件与线上发布物，必须自行预读并判断文件类型；同时 `MetadataFlattener` 内部直接调用 `System.IO`，远端 HTTP 源树无法复用展平逻辑；此外还存在菱形 Include 重复展开、单文件内同名登记 ID 静默覆盖、以及越界路径检查口径不统一等边界问题。 + +## Decision + +新增统一入口 `MetadataBuilder.Open(pathOrUrl, options)` 与配套抽象，将源树展平与发布物直接解析整合在一起： + +- **自动识别与选项**：由 `MetadataDetector.HasInclude` 检查入口 XML 是否包含 `Include`/`Includes` 元素，有则走展平，无则走发布物解析。通过 `MetadataSourceOptions` 提供 `SourceRoot`（源根目录）、`ForceSourceTree`（无 Include 也强制展平）、`ValidateReferences`（开启断引用校验）与 `CustomSource`（注入自定义 XML 源）。 +- **只读 XML 源抽象**：定义 `IMetadataXmlSource` 接口，把存在性检查、文档流读取与路径运算（`Combine`、`GetRelativePath` 等）从 `MetadataFlattener` 抽离。提供针对本地磁盘的 `PhysicalFileXmlSource` 与针对远端地址的 `HttpXmlSource`。远端路径以绝对 URL 表达，跨主机或越出源根的引用直接拒绝。 +- **既有入口契约保持稳定**：`MetadataBuilder.Load` 仍只认发布物；`MetadataBuilder.LoadSource` 保持签名不变，内部委托 `Open` 并开启 `ForceSourceTree = true`；`MetadataParser.Load` 继续严格拒绝 `Include`/`Includes`/`Module`。 +- **引用完整性校验**：新增 `MetadataValidator.EnsureReferences` 与 `FindMissingReferences`，以登记节点（带 `ID` 的 `Image`/`Markdown`/`Manifest`）为合法候选集合，检查实体与样式的引用字段，并在发现断引用时抛出明确异常。 +- **四个边界缺陷收紧**： + 1. 菱形 Include 去重：会话内记录已展开文件，再次遇到跳过展开，避免重复并入实体和登记节点。 + 2. 同文件登记 ID 查重：同文件内声明的登记短名查重（含仅大小写差异），冲突抛出 `InvalidOperationException`。 + 3. 路径越界前置拦截：入口、`Include` 与资源 `Source` 经 `..`、绝对路径或 UNC 越出源根时，在读取或探测目标前直接拒绝。 + 4. 大小写口径统一：同一文件内登记短名查重与 ID 唯一性判定统一按 `OrdinalIgnoreCase`，仅大小写不同的重复登记也抛 `InvalidOperationException`。 + +规则同步记录于 [library-api.md](../../../docs/references/library-api.md)，此前本地源树机制见 [2026-10-28-local-mod-source-load.md](./2026-10-28-local-mod-source-load.md)。 +当前 [MetadataBuilder.cs](../../../../Ra3.BattleNet.Metadata/Building/MetadataBuilder.cs) 仅负责构建；本篇当时的 Open/MetadataDetector/MetadataFlattener 实现已移除，见 [single-format 决定](../../implemented/simplification/2026-11-07-single-metadata-format.md)。低层 XML 投影与引用校验分别位于 [Reading/MetadataParser.cs](../../../../Ra3.BattleNet.Metadata/Reading/MetadataParser.cs)、[Reading/MetadataValidator.cs](../../../../Ra3.BattleNet.Metadata/Reading/MetadataValidator.cs)，当前单一读取与目录责任见 [集成计划](../../../docs/exec-plans/2026-11-07-composable-metadata.md)。上文历史 Decision 不据此改写。 + +当前读取实现位于 [Reading/MetadataReader.cs](../../../../Ra3.BattleNet.Metadata/Reading/MetadataReader.cs)，选项与 XML 来源边界分别见 [MetadataSourceOptions.cs](../../../../Ra3.BattleNet.Metadata/Reading/MetadataSourceOptions.cs)、[IMetadataXmlSource.cs](../../../../Ra3.BattleNet.Metadata/Reading/IMetadataXmlSource.cs)、[PhysicalFileXmlSource.cs](../../../../Ra3.BattleNet.Metadata/Reading/PhysicalFileXmlSource.cs) 和 [HttpXmlSource.cs](../../../../Ra3.BattleNet.Metadata/Reading/HttpXmlSource.cs)。这些是现行实现路径，不表示旧模式选项继续存在。 + +## Alternatives considered + +- **放宽 `MetadataBuilder.Load` 直接支持源树**：调用点看似最少，但这会让发布物严格校验失去防线，客户端可能静默载入未展平的源树半成品。 +- **将 HTTP 读取与下载完全留给宿主**：外部宿主可通过下载整包或挂载本地目录再调用，但在线调试作者源树是常见场景，标准库引入轻量只读流式源即可支持，无需宿主编写大量胶水代码。 +- **源树模式下求值构建期变量**：如果在源树模式下替换 `${TIMESTAMP}` 等变量，每次读取得到的时间戳与指纹都不一致，破坏了只读解析的幂等性与可复现性。 + +## Consequences + +- **等价性保证**：现有源树构建产物与源树直接读取在语义上完全一致。基于真实仓库数据（源树 661 节点 / 82 个登记 ID / 17 个实体），源树展平与完整构建产物逐字对齐，仅有的差异是预期的构建期字段（根 `ContentRevision` 与 `Tags/Commit`）。 +- **HTTP 源行为与开销特征**：`HttpXmlSource` 的 XML 正文按规范化 URL 缓存，因此同一 XML 先 `Exists` 再 `LoadDocument` 只会发起一次 HTTP GET 请求；配置 60 秒超时；自建处理器时关闭自动重定向，遇到 3xx 抛错；仍然没有多地址重试和断点续传，那部分由发布物专用的 `Load` / `MetadataClient` 承担。 +- **资源生命周期**：`HttpXmlSource` 实现 `IDisposable`，仅释放自身创建的 `HttpClient`，外部注入的 handler 归调用方所有；`MetadataBuilder.Open` 内部创建的源实例在用完后立即以 `using` 释放，宿主传入的 `CustomSource` 则归调用方管理。 + +## Verification + +- `dotnet test Metadata.sln` 372 个测试全绿，涵盖 `OpenMethodTests`、`EquivalenceBaselineTests`、`HttpSourceTests`、`ValidationTests`、`PathSecurityTests` 与 `PerformanceTests`。 +- `EquivalenceBaselineTests` 验证源树读取与发布物读取逐字等价，整仓源树读取本机实测约 4 ms，测试断言上限 500 ms。 +- `dotnet run --project Ra3.BattleNet.Metadata -- build --src=./Metadata --dst=<临时目录>` 冒烟成功，产物 `metadata.xml` 绝无 `Include` 节点。 diff --git a/.agents/notes/implemented/simplification/2026-11-07-single-metadata-format.md b/.agents/notes/implemented/simplification/2026-11-07-single-metadata-format.md new file mode 100644 index 0000000..31d03a2 --- /dev/null +++ b/.agents/notes/implemented/simplification/2026-11-07-single-metadata-format.md @@ -0,0 +1,54 @@ +# Agent Note: One metadata format without compatibility modes + +Status: implemented + +The colon-ID rejection in this historical decision is superseded by [resource ID keys](../bug-fix/2026-11-07-resource-id-keys.md). Existing complete keys are retained and matched verbatim without restoring compatibility modes. + +## Problem + +The user explicitly removes the old compatibility requirement. Keeping author/published detection, colon/path codecs, source-root fallback and old read entry points after that decision leaves two protocols inside the nominally shared Reader. It also duplicates tests that protect behavior no longer requested. Flat, Tree and Mixed must describe storage organization only. + +## Decision + +Use only `MetadataReader.Read(...).Root` for document reading and only canonical reversible path resource IDs (`scope/local`) in the normalized model and exports. Qualify every short resource ID, including a single document without Includes; preserve validated canonical path IDs and reject colon-qualified IDs. Keep Mod/Application business IDs unchanged. Remove `MetadataBuilder.Load`, `LoadSource`, `Open`, `MetadataFlattener`, compatibility detection, dual codecs, `ForceSourceTree`, `LegacySourcePaths`, `MetadataExportIdFormat`, `ExportLegacy`, `IdFormat`/`SourceBase` markers and `--id-format`. `MetadataId` owns PathPrefix, Qualify and LocalId; Parser remains the low-level XML-to-model projection, not an alternative reader. + +Interpret Source relative to its defining document and expose it relative to the entry directory. SourceRoot supplies containment and qualification, never a second address fallback. HTTP documents cannot reference local files. Explicit resource URLs remain supported; Image.Url takes priority over informational Source. Include contributes actual full contents, including Manifest bodies. External Manifest Source stays external unless explicitly inlined. A short external stub uses the target definition scope; body resolution shares SourceRoot and requires exact canonical identity, never suffix/LocalId matching or alias rewriting. Preserve scoped reference precedence, permanent ambiguity, URI/path checks, cycles and deduplication. Reading preserves the declared schema/revision; Build/export set them explicitly. + +Build uses normal Export and its existing validation/variable/image processing. Flat/Tree/Mixed, selective Include inlining and explicit manifest/resource processing remain real features. SourceExporter still creates short-ID local author declarations explicitly; that write operation is not a read compatibility mode. Cache uses Reader for every acquired XML and one normalized snapshot record, with no old-cache migration fallback. Preserve last-complete graphs, cache-only/Fresh/Stale behavior and raw-byte/validator pairing. + +## File and test organization + +Library files use Reading, Exporting, Building and existing Cache responsibility folders; models/query contracts remain at root. Tests use Reading, Exporting, Building, Cache and Queries. Namespaces and shared TestData placement stay unchanged. Meaningful old-entry-point cases are consolidated into current tests; obsolete-mode tests and fixed-stopwatch PerformanceTests are removed without dropping security/offline/domain coverage. The [active plan](../../../docs/exec-plans/2026-11-07-composable-metadata.md) owns the directory table and integration evidence. + +## Related decision audit + +The active proposed/implemented/rejected notes were searched for composition, local source, unified reading, source export and cache behavior before this proposal. No historical Decision is rewritten or archived. + +| Prior note | Classification | Retained and superseded boundary | +|---|---|---| +| [Composable metadata](../../implemented/architecture/2026-11-07-composable-metadata.md) | Partially superseded; crosslinked | Keep one composition/layout core, captured graph and package APIs; overturn dual ID formats, marker/detection branches, compatibility read APIs and default legacy publication. Its verification applies only to that earlier contract. | +| [Unified reader](../../implemented/feature/2026-11-01-unified-metadata-reader.md) | Partial overlap | Keep unified reading and XML source abstraction; the old Open/strict Load split is removed. Historical rationale remains accessible through the composable decision. | +| [Local source loading](../../implemented/feature/2026-10-28-local-mod-source-load.md) | Partial overlap | Keep in-memory source reads and containment; remove LoadSource, source-root fallback and stub conversion. | +| [Source-format host API](../../implemented/feature/2026-10-31-source-format-host-api.md) | Partial overlap | Keep explicit short-ID author writes and manifest projection; replace LocalId matching with exact canonical identity. | +| [XML runtime](../../implemented/architecture/2026-10-04-metadata-xml-runtime.md), [cadence](../../implemented/architecture/2026-10-09-registered-resource-cache-cadence.md), [transfer](../../implemented/architecture/2026-10-12-metadata-transfer-to-local-file.md) | Partial overlap | Keep transfer ownership, cache-only display and last complete graph; no strict raw-flat reader branch or old snapshot migration protocol. | +| [Rejected cache framework](../../rejected/architecture/2026-10-02-runtime-metadata-cache.md) | Rejection remains valid | Do not add leases, stores or a migration framework. | + +Model/style/language and deployment decisions are unrelated. No note is fully absorbed: earlier unique rationale and evidence remain historical, not current acceptance evidence. + +## Alternatives considered + +- **Keep thin compatibility delegates and two codecs**: reduces immediate caller churn and preserves old consumers. The user explicitly rejects that requirement; it retains obsolete surface and tests rather than simplifying the contract. +- **Infer author/published mode from Include or markers**: accommodates ambiguous old inputs. A single format removes that ambiguity instead of maintaining heuristics. +- **Delete layout or local author export together with legacy modes**: removes more code, but erases genuine requested storage and Desktop write features. Remove protocol choices, not those capabilities. + +## Verification + +The parent reports the final Metadata full suite at 429/429 passed with 0 warnings and 0 failures. Actual single-format Build, Tree export, all six Flat/Tree/Mixed × external/inline smoke combinations, and HTTP root 304/child update, complete-graph fallback and offline reopening passed. Security review found no remaining Image-only Url issue, and the semantic review findings were corrected with targeted coverage. This evidence is parent-reported and does not claim Desktop verification or GUI/game execution. + +The final Exporter shape simplification uses only absent/empty Source to identify a full Manifest, matching Reader and cache even for empty bodies; the affected export checks passed 9/9 with no warnings. This targeted result is separate from, not added to, the full 429-test run. + +## Consequences + +This intentionally breaks colon-format and old API consumers; caches may start new. Exact external Manifest identity requires shared SourceRoot context, not a forgiving fallback. Flat/Tree/Mixed remain storage organization choices, while local author export remains an explicit write feature. Desktop dependency delivery and its own validation remain pending. + +Against prior PR HEAD dcaa1d5, final tracked C# comparison excluding bin/obj (including blank/comment lines) observed production 43 files/7617 lines to 41/6941 (−676) and tests 36/11167 to 32/9504 (−1663), for 2339 fewer lines total. This is an observed consequence, not a target or acceptance threshold, and excludes documentation. diff --git a/.agents/notes/rejected/architecture/2026-10-02-runtime-metadata-cache.md b/.agents/notes/rejected/architecture/2026-10-02-runtime-metadata-cache.md index 28e5375..76a3d14 100644 --- a/.agents/notes/rejected/architecture/2026-10-02-runtime-metadata-cache.md +++ b/.agents/notes/rejected/architecture/2026-10-02-runtime-metadata-cache.md @@ -2,6 +2,8 @@ Status: rejected — replaced by the implemented XML-only runtime cache in [2026-10-04-metadata-xml-runtime.md](../../implemented/architecture/2026-10-04-metadata-xml-runtime.md) +Related decision: [one composition core and layout-only exports](../../implemented/architecture/2026-11-07-composable-metadata.md) explicitly retains this rejection. Minimal last-complete graph persistence does not revive the multiversion, lease or multiprocess runtime framework. + ## Problem 本提案原本想为根清单、包清单、媒体资源和跨进程快照建立统一缓存，并计划接入一个包含租约和回收机制的 `MetadataRuntime`。这比本次实际需求更宽：用户只要求 Desktop 与 Metadata 统一加载和解析 XML，图片、Markdown、大安装包以及 Updater 增量清单都不属于预加载范围。 diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index 43ad883..3199f14 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -20,7 +20,7 @@ jobs: persist-credentials: false - name: Build Output - run: bash build.sh --webp --content-revision=${{ github.sha }} + run: bash build.sh --webp - name: Deploy to EdgeOne run: npx edgeone makers deploy ./Output -n metadata -t ${{ secrets.EDGEONE_API_TOKEN }} -e production diff --git a/AGENTS.md b/AGENTS.md index b2996a1..0a2279e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -45,7 +45,7 @@ node scripts/verify-docs.mjs --write # 文档内容确实改了时，重写生 - 发布树只有静态文件，根 `metadata.xml` 里不能有 `Include`。 - 完整 ID 由构建器生成；`Manifest`、`Image`、`Markdown` 的登记与引用只用短名。 - 资源地址只来自登记节点的 `Source`，客户端不猜路径、不猜扩展名。 -- `SchemaVersion` 保持 `1.0`；改样式、加可选字段不提高版本。 +- `SchemaVersion` 写死在源根 `metadata.xml` 的根节点，保持 `1.0`；构建不注入版本、不写修订标记，改样式、加可选字段不提高版本。 - `Package` 只带 `Version`、`ReleaseDate`、`Manifest`；更新说明写在实体的 `Posts`。 - `Mod` 与 `Application` 必须各有 `zh`、`en` 两条 `DisplayName`；显示名、`Posts`、`Links` 都不从 `Base` 继承。 - 写了 `Skudef` 就不许在 `File` 上写 `Mount`、`Language`、`Package`。 diff --git a/Metadata/MetadataPublishSchema.xsd b/Metadata/MetadataPublishSchema.xsd index 20184d0..f03aa50 100644 --- a/Metadata/MetadataPublishSchema.xsd +++ b/Metadata/MetadataPublishSchema.xsd @@ -14,7 +14,6 @@ - diff --git a/Metadata/MetadataSchema.xsd b/Metadata/MetadataSchema.xsd index 3ecc811..61c9cae 100644 --- a/Metadata/MetadataSchema.xsd +++ b/Metadata/MetadataSchema.xsd @@ -15,6 +15,8 @@ + + diff --git a/Metadata/editor.html b/Metadata/editor.html index 189f156..eadc4da 100644 --- a/Metadata/editor.html +++ b/Metadata/editor.html @@ -2409,8 +2409,10 @@

第三方组件

if (mod.getAttribute('InheritFrom')) { warn(`Mod 声明了 InheritFrom="${mod.getAttribute('InheritFrom')}"；本工具不解析 Base，预览只显示本文件里写出的字段。`); } - if (state.doc.documentElement.getAttribute('SchemaVersion')) { - warn('这个文件带 SchemaVersion，看起来是构建产物而不是源文件；导出后不能直接提交，请导入 mods//.xml。'); + const commitTag = direct(state.doc.documentElement, 'Tags'); + const commitStamp = commitTag ? direct(commitTag, 'Commit') : null; + if (commitStamp && /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$/.test(String(commitStamp.textContent || '').trim())) { + warn('这个文件的 Commit 已被构建替换成时间戳，是发布物而不是源文件；导出后不能直接提交，请导入 mods//.xml。'); } const hasIncludes = Boolean(direct(state.doc.documentElement, 'Includes')); const missingRef = (text) => { diff --git a/Metadata/metadata.xml b/Metadata/metadata.xml index f262584..ad37eff 100644 --- a/Metadata/metadata.xml +++ b/Metadata/metadata.xml @@ -1,5 +1,5 @@  - + ${TIMESTAMP} diff --git a/README.md b/README.md index a3c3cd1..e5b474d 100644 --- a/README.md +++ b/README.md @@ -20,6 +20,8 @@ dotnet run --project Ra3.BattleNet.Metadata -- build --src=./Metadata --dst=./Ou 构建成功会打印 `✓ 构建完成` 和实体列表，产物在 `Output/`。任何校验不通过都会清空输出并说明改哪个文件。 +一次性读取统一使用 `MetadataReader.Read(pathOrUrl).Root`，不再提供 Builder.Load/LoadSource/Open。新生成资源 ID 使用 `scope/local`，已有含 `/` 或 `:` 的完整 ID 原样匹配；Source 相对定义文档。Flat/Tree/Mixed 只是导出组织，没有 `--id-format` 或旧格式兼容模式。完整 API 见 [解析库与缓存](.agents/docs/references/library-api.md)。 + ## 加一个 Mod 1. 复制 `Metadata/Templates/Mod.xml` 到 `Metadata/mods/<英文名>/<英文名>.xml`。 @@ -46,9 +48,9 @@ dotnet run --project Ra3.BattleNet.Metadata -- build --src=./Metadata --dst=./Ou ```text Metadata/ 源数据；metadata.xml 是总入口 -Ra3.BattleNet.Metadata/ 解析库 + 命令行构建工具 +Ra3.BattleNet.Metadata/ Reading/、Exporting/、Building/、Cache/；模型/query 留根 Ra3.BattleNet.Metadata.Imaging/ 发版用的 WebP 转换工具 -Ra3.BattleNet.Metadata.Tests/ MSTest 测试 +Ra3.BattleNet.Metadata.Tests/ Reading/、Exporting/、Building/、Cache/、Queries/；TestData 不移 scripts/verify-docs.mjs 知识库检查 .agents/ 研发文档、决策笔记、自动生成的索引 ``` diff --git a/Ra3.BattleNet.Metadata.Tests/StageATests.cs b/Ra3.BattleNet.Metadata.Tests/Building/BuildTests.cs similarity index 74% rename from Ra3.BattleNet.Metadata.Tests/StageATests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/BuildTests.cs index bcf4ce2..9ffac51 100644 --- a/Ra3.BattleNet.Metadata.Tests/StageATests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Building/BuildTests.cs @@ -5,7 +5,7 @@ namespace Ra3.BattleNet.Metadata.Tests; [TestClass] -public class StageATests +public class BuildTests { private static string RepoMetadataDir => Path.GetFullPath(Path.Combine(AppContext.BaseDirectory, "..", "..", "..", "..", "Metadata")); @@ -27,7 +27,7 @@ public void Build_RepoSample_FlattensWithoutInclude_AndResolvesIds() try { - MetadataBuilder.Build(src, dst, schemaVersion: "1.0", contentRevision: "test-rev"); + MetadataBuilder.Build(src, dst); var flatPath = Path.Combine(dst, "metadata.xml"); File.Exists(flatPath).Should().BeTrue(); @@ -36,33 +36,29 @@ public void Build_RepoSample_FlattensWithoutInclude_AndResolvesIds() text.Should().NotContain(" a.Id == "RA3BattleNet"); loaded.Mods().Should().Contain(m => m.Id == "Corona"); - // 发布 ID = {路径前缀}:{localId} + // 发布 ID 使用定义文档 scope/local。 var registries = loaded.GetAllElements("Manifest") .Where(m => !string.IsNullOrEmpty(m.Get("ID"))).ToList(); var manifest = registries - .First(m => MetadataFlattener.LocalId(m.Get("ID")) == "manifest-1.9.9.22"); - manifest.Get("ID")!.Should().Contain(":"); - manifest.Get("ID")!.Should().EndWith(":manifest-1.9.9.22"); + .First(m => MetadataId.LocalId(m.Get("ID")) == "manifest-1.9.9.22"); + manifest.Get("ID")!.Should().Contain("/"); + manifest.Get("ID")!.Should().EndWith("/manifest-1.9.9.22"); var source = manifest.Get("Source"); source.Should().NotBeNullOrWhiteSpace(); - source!.Replace('\\', '/').Should().EndWith("manifests/1.9.9.22.xml"); - source.Should().NotContain("apps.xml"); - var manifestPath = Path.Combine(dst, source.Replace('/', Path.DirectorySeparatorChar)); + var manifestPath = MetadataResourceUri.Resolve(flatPath, source!).LocalPath; File.Exists(manifestPath).Should().BeTrue(); File.ReadAllText(manifestPath).Should().Contain(" MetadataFlattener.LocalId(m.Get("ID")!) == "manifest-3259"); - coronaManifest.Get("Source")!.Replace('\\', '/').Should().EndWith("manifests/3.259.xml"); + .First(m => MetadataId.LocalId(m.Get("ID")!) == "manifest-3259"); + File.Exists(MetadataResourceUri.Resolve(flatPath, coronaManifest.Get("Source")!).LocalPath).Should().BeTrue(); // 引用已改写为限定 ID（示例数据里只剩 Mod 带新闻） var app = loaded.Applications().Single(a => a.Id == "RA3BattleNet"); @@ -108,7 +104,7 @@ public void Build_RepoSample_FlattensWithoutInclude_AndResolvesIds() .GetAllElements("Content") .First(c => string.Equals(c.Get("Language"), "zh", StringComparison.OrdinalIgnoreCase)) .Value!; - MetadataFlattener.LocalId(publishedContentId).Should().Be(localContentId); + MetadataId.LocalId(publishedContentId).Should().Be(localContentId); var markdown = loaded.Markdowns().Single(m => m.Id == publishedContentId); File.ReadAllText(Path.Combine(dst, markdown.Source!.Replace('/', Path.DirectorySeparatorChar))) .Should().Be(sourceBody); @@ -118,7 +114,7 @@ public void Build_RepoSample_FlattensWithoutInclude_AndResolvesIds() var icon = loaded.Images().Single(image => image.Id == corona.Icon); var sourceDoc = XDocument.Load(Path.Combine(src, "mods", "corona", "corona.xml")); var localIconId = sourceDoc.Descendants("Mod").Single().Element("Icon")!.Value; - icon.Id.Should().Be(MetadataFlattener.QualifyId("mods/corona/corona", localIconId)); + icon.Id.Should().Be(MetadataId.Qualify("mods/corona/corona", localIconId)); } finally { @@ -135,7 +131,7 @@ public void Build_RepoSample_TransferAdFlattensForDeclaringAppOnly() try { - MetadataBuilder.Build(src, dst, schemaVersion: "1.0", contentRevision: "transfer-ad"); + MetadataBuilder.Build(src, dst); var doc = XDocument.Load(Path.Combine(dst, "metadata.xml")); var apps = doc.Root!.Elements("Application").ToList(); @@ -166,6 +162,31 @@ public void Build_RepoSample_TransferAdFlattensForDeclaringAppOnly() } } + [TestMethod] + public void Build_MissingSourceSchemaVersion_FailsBeforeExport() + { + var temp = Path.Combine(Path.GetTempPath(), $"stage-a-version-{Guid.NewGuid():N}"); + var src = Path.Combine(temp, "src"); + var dst = Path.Combine(temp, "out"); + SeedSchemas(src); + try + { + File.WriteAllText(Path.Combine(src, "metadata.xml"), """ + + +"""); + + var act = () => MetadataBuilder.Build(src, dst); + act.Should().Throw().WithMessage("*缺少 SchemaVersion*"); + Directory.Exists(dst).Should().BeFalse("半残输出应被清理"); + } + finally + { + if (Directory.Exists(temp)) + Directory.Delete(temp, recursive: true); + } + } + [TestMethod] public void Build_MissingMarkdown_HardFails_AndCleansOutput() { @@ -177,13 +198,13 @@ public void Build_MissingMarkdown_HardFails_AndCleansOutput() { File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + """); - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "x"); - act.Should().Throw(); + var act = () => MetadataBuilder.Build(src, dst); + act.Should().Throw(); Directory.Exists(dst).Should().BeFalse("半残输出应被清理"); } finally @@ -205,7 +226,7 @@ public void Build_BadIdReference_HardFails() File.WriteAllText(Path.Combine(src, "note.md"), "# hello sample markdown body\n"); File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + 1.0 @@ -220,7 +241,7 @@ public void Build_BadIdReference_HardFails() """); - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "x"); + var act = () => MetadataBuilder.Build(src, dst); act.Should().Throw().WithMessage("*no-such-manifest*"); } finally @@ -234,6 +255,7 @@ public void Build_BadIdReference_HardFails() public void Flatten_CircularInclude_Throws() { var temp = Path.Combine(Path.GetTempPath(), $"stage-a-circ-{Guid.NewGuid():N}"); + var output = temp + "-output"; SeedSchemas(temp); try { @@ -255,20 +277,22 @@ public void Flatten_CircularInclude_Throws() """); File.WriteAllText(Path.Combine(temp, "metadata.xml"), """ - + """); - var act = () => MetadataBuilder.Build(temp, Path.Combine(temp, "out"), contentRevision: "x"); - act.Should().Throw().WithMessage("*循环引用*"); + var act = () => MetadataBuilder.Build(temp, output); + act.Should().Throw(); } finally { if (Directory.Exists(temp)) Directory.Delete(temp, recursive: true); + if (Directory.Exists(output)) + Directory.Delete(output, recursive: true); } } @@ -282,13 +306,13 @@ public void Build_LeftoverVariable_HardFails() { File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + ${ENV:THIS_ENV_SHOULD_NOT_EXIST_ZZZ} """); - var act = () => MetadataBuilder.Build(src, Path.Combine(temp, "out"), contentRevision: "x"); + var act = () => MetadataBuilder.Build(src, Path.Combine(temp, "out")); act.Should().Throw().WithMessage("*残留*"); } finally @@ -299,12 +323,32 @@ public void Build_LeftoverVariable_HardFails() } [TestMethod] - public void Flatten_Document_HasNoIncludeNodes() + public void Build_PreservesSourceSemanticsExceptDeclaredBuildFields() { - var entry = Path.Combine(RepoMetadataDir, "metadata.xml"); - var doc = MetadataFlattener.Flatten(entry, RepoMetadataDir, "1.0", "rev"); - doc.Descendants().Any(e => e.Name.LocalName is "Include" or "Includes").Should().BeFalse(); - doc.Root!.Attribute("SchemaVersion")!.Value.Should().Be("1.0"); - XDocument.Parse(doc.ToString()).Root!.Attribute("ContentRevision")!.Value.Should().Be("rev"); + var output = Path.Combine(Path.GetTempPath(), $"build-equivalence-{Guid.NewGuid():N}"); + try + { + var options = new MetadataSourceOptions { InlineManifests = true }; + var source = MetadataReader.Read(Path.Combine(RepoMetadataDir, "metadata.xml"), options).Document; + MetadataBuilder.Build(RepoMetadataDir, output); + var published = MetadataReader.Read(Path.Combine(output, "metadata.xml"), options).Document; + // 源声明的 SchemaVersion 原样保留，构建不追加版本或修订属性 + source.Root!.Attribute("SchemaVersion")!.Value.Should().Be("1.0"); + published.Root!.Attribute("SchemaVersion")!.Value.Should().Be("1.0"); + published.Root.Attribute("ContentRevision").Should().BeNull(); + source.Root.Element("Tags")!.Element("Commit")!.Value.Should().Be("${TIMESTAMP}"); + published.Root.Element("Tags")!.Element("Commit")!.Value.Should() + .MatchRegex(@"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$"); + foreach (var document in new[] { source, published }) + { + document.Root!.Element("Tags")!.Element("Commit")!.Value = "build-stamp"; + } + MetadataXml.Write(MetadataParser.Load(published)).Should().Be(MetadataXml.Write(MetadataParser.Load(source))); + } + finally + { + if (Directory.Exists(output)) Directory.Delete(output, recursive: true); + } } + } diff --git a/Ra3.BattleNet.Metadata.Tests/ImagingTests.cs b/Ra3.BattleNet.Metadata.Tests/Building/ImagingTests.cs similarity index 93% rename from Ra3.BattleNet.Metadata.Tests/ImagingTests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/ImagingTests.cs index 4ad38a7..ecde01f 100644 --- a/Ra3.BattleNet.Metadata.Tests/ImagingTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Building/ImagingTests.cs @@ -16,7 +16,7 @@ public void Build_WithConvertImages_UpdatesXmlViaImagingCli() var dst = Path.Combine(Path.GetTempPath(), $"build-webp-{Guid.NewGuid():N}"); try { - MetadataBuilder.Build(RepoMetadataDir, dst, contentRevision: "webp", convertImages: true); + MetadataBuilder.Build(RepoMetadataDir, dst, convertImages: true); var flat = Path.Combine(dst, "metadata.xml"); var doc = XDocument.Load(flat); var iconId = doc.Descendants("Mod").Single(e => e.Attribute("ID")!.Value == "Corona").Element("Icon")!.Value; diff --git a/Ra3.BattleNet.Metadata.Tests/InheritanceTests.cs b/Ra3.BattleNet.Metadata.Tests/Building/InheritanceTests.cs similarity index 99% rename from Ra3.BattleNet.Metadata.Tests/InheritanceTests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/InheritanceTests.cs index a0591b5..5351ec9 100644 --- a/Ra3.BattleNet.Metadata.Tests/InheritanceTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Building/InheritanceTests.cs @@ -266,11 +266,8 @@ public void Flatten_Publish_NoBaseNoInheritFrom() """); - var doc = MetadataFlattener.Flatten( - Path.Combine(src, "metadata.xml"), - src, - "1.0", - "rev"); + var doc = MetadataReader.Read(Path.Combine(src, "metadata.xml"), + new MetadataSourceOptions { SourceRoot = src }).Document; doc.Descendants("Base").Should().BeEmpty(); doc.Descendants("Include").Should().BeEmpty(); @@ -303,7 +300,7 @@ public void Build_WithPrivateBase_SucceedsAndPublishClean() SeedSchemas(src); File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + @@ -350,9 +347,9 @@ public void Build_WithPrivateBase_SucceedsAndPublishClean() """); - MetadataBuilder.Build(src, dst, schemaVersion: "1.0", contentRevision: "inh-test"); + MetadataBuilder.Build(src, dst); - var loaded = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")); + var loaded = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; loaded.GetAllElements("Base").Should().BeEmpty(); loaded.GetAllElements("Include").Should().BeEmpty(); loaded.GetAllElements("Includes").Should().BeEmpty(); diff --git a/Ra3.BattleNet.Metadata.Tests/ManifestParseTests.cs b/Ra3.BattleNet.Metadata.Tests/Building/ManifestParseTests.cs similarity index 91% rename from Ra3.BattleNet.Metadata.Tests/ManifestParseTests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/ManifestParseTests.cs index 375155e..a145602 100644 --- a/Ra3.BattleNet.Metadata.Tests/ManifestParseTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Building/ManifestParseTests.cs @@ -13,7 +13,7 @@ public class ManifestParseTests private static Metadata LoadManifestNode(string fileName) { - var node = Metadata.LoadFromFile(Path.Combine(TestDataPath, fileName)).Find("Manifest"); + var node = MetadataReader.Read(Path.Combine(TestDataPath, fileName)).Root.Find("Manifest"); node.Should().NotBeNull(); return node!; } @@ -23,7 +23,7 @@ public void ToManifestEntry_ExplicitAlgorithm_ParsesFilesSourcesAndDependencies( { var entry = LoadManifestNode("manifest-full.xml").ToManifestEntry(); - entry.Id.Should().Be("manifest-full"); + entry.Id.Should().Be("manifest-full/manifest-full"); entry.HashAlgorithm.Should().Be("SHA256"); entry.Files.Should().HaveCount(2); @@ -105,7 +105,7 @@ public void ToManifestEntry_ExplicitMount_ReadsLanguageAndPackage() """); - var entry = Metadata.LoadFromFile(temp).Find("Manifest")!.ToManifestEntry(); + var entry = MetadataReader.Read(temp).Root.Find("Manifest")!.ToManifestEntry(); entry.Files[0].Mount.Should().Be("language"); entry.Files[0].Language.Should().Be("chs"); entry.Files[0].Package.Should().BeNull(); @@ -169,7 +169,7 @@ public void ToManifestEntry_LowercaseSourceType_NormalizedToUpper() """); - var entry = Metadata.LoadFromFile(temp).Find("Manifest")!.ToManifestEntry(); + var entry = MetadataReader.Read(temp).Root.Find("Manifest")!.ToManifestEntry(); entry.Files[0].Sources.Select(s => s.Type).Should().Equal("HTTP", "BT"); } finally @@ -196,7 +196,7 @@ public void ToManifestEntry_UnknownAlgorithm_Throws() """); - var node = Metadata.LoadFromFile(temp).Find("Manifest")!; + var node = MetadataReader.Read(temp).Root.Find("Manifest")!; var act = () => node.ToManifestEntry(); act.Should().Throw().WithMessage("*HashAlgorithm*"); } @@ -227,7 +227,7 @@ public void ToManifestEntry_NoneAlgorithmZipCompression_ParsesEmptyHash() """); - var entry = Metadata.LoadFromFile(temp).Find("Manifest")!.ToManifestEntry(); + var entry = MetadataReader.Read(temp).Root.Find("Manifest")!.ToManifestEntry(); entry.HashAlgorithm.Should().Be("NONE"); var file = entry.Files.Should().ContainSingle().Subject; @@ -250,14 +250,16 @@ public void ManifestRegistration_ResolvesRegisteredId_AndNullWhenMissing() var dst = Path.Combine(Path.GetTempPath(), $"manifest-reg-{Guid.NewGuid():N}"); try { - MetadataBuilder.Build(src, dst, contentRevision: "manifest-reg"); - var root = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")); + MetadataBuilder.Build(src, dst); + var root = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; var registered = root.GetAllElements("Manifest") - .First(m => MetadataFlattener.LocalId(m.Get("ID")) == "manifest-3259"); + .First(m => MetadataId.LocalId(m.Get("ID")) == "manifest-3259"); var found = root.ManifestRegistration(registered.Get("ID")!); found.Should().BeSameAs(registered); - found!.Get("Source").Should().EndWith("manifests/3.259.xml"); + var source = found!.Get("Source")!; + var leaf = MetadataReader.Read(MetadataResourceUri.Resolve(Path.Combine(dst, "metadata.xml"), source).LocalPath).Root; + leaf.ManifestRegistration(registered.Get("ID")!)!.ToManifestEntry().Files.Should().NotBeEmpty(); root.ManifestRegistration("no-such-manifest").Should().BeNull(); } @@ -274,7 +276,7 @@ private static Metadata LoadInlineManifest(string xml) try { File.WriteAllText(path, xml); - var node = Metadata.LoadFromFile(path).Find("Manifest"); + var node = MetadataReader.Read(path).Root.Find("Manifest"); node.Should().NotBeNull(); return node!; } diff --git a/Ra3.BattleNet.Metadata.Tests/ManifestRawFingerprintTests.cs b/Ra3.BattleNet.Metadata.Tests/Building/ManifestRawFingerprintTests.cs similarity index 100% rename from Ra3.BattleNet.Metadata.Tests/ManifestRawFingerprintTests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/ManifestRawFingerprintTests.cs diff --git a/Ra3.BattleNet.Metadata.Tests/ManifestValidationTests.cs b/Ra3.BattleNet.Metadata.Tests/Building/ManifestValidationTests.cs similarity index 96% rename from Ra3.BattleNet.Metadata.Tests/ManifestValidationTests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/ManifestValidationTests.cs index f2de682..710b623 100644 --- a/Ra3.BattleNet.Metadata.Tests/ManifestValidationTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Building/ManifestValidationTests.cs @@ -91,10 +91,10 @@ private static void AssertBuildFails(string manifestXml, params string[] fragmen try { SeedSchemas(src); - File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); + File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); File.WriteAllText(Path.Combine(src, "manifest.xml"), manifestXml); - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "x"); + var act = () => MetadataBuilder.Build(src, dst); var ex = act.Should().Throw().Which; ex.Message.Should().Contain("manifest.xml", "消息必须带来源文件路径"); ex.Message.Should().Contain("manifest-bad", "消息必须带 Manifest ID"); @@ -118,10 +118,10 @@ private static void AssertBuildSucceeds(string manifestXml) try { SeedSchemas(src); - File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); + File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); File.WriteAllText(Path.Combine(src, "manifest.xml"), manifestXml); - MetadataBuilder.Build(src, dst, contentRevision: "x"); + MetadataBuilder.Build(src, dst); File.Exists(Path.Combine(dst, "metadata.xml")).Should().BeTrue(); } finally @@ -227,11 +227,11 @@ public void Build_UnknownCompression_HardFails() try { SeedSchemas(src); - File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); + File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); File.WriteAllText(Path.Combine(src, "manifest.xml"), ManifestXml(CompressedFileBlock("a.zst", "lz4"))); // Compression 的取值由 XSD 枚举兜住，构建在 schema 阶段就失败 - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "x"); + var act = () => MetadataBuilder.Build(src, dst); act.Should().Throw().WithMessage("*Compression*"); Directory.Exists(dst).Should().BeFalse("半残输出应被清理"); } @@ -306,12 +306,12 @@ public void Build_CompressedFile_Succeeds() try { SeedSchemas(src); - File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); + File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); File.WriteAllText( Path.Combine(src, "manifest.xml"), ManifestXml(CompressedFileBlock("a.zst", "zstd", fileName: "a.lyi"))); - MetadataBuilder.Build(src, dst, contentRevision: "x"); + MetadataBuilder.Build(src, dst); File.Exists(Path.Combine(dst, "metadata.xml")).Should().BeTrue(); } finally @@ -421,7 +421,7 @@ public void Build_InvalidExplicitRawAlgorithm_IsRejectedBySchema(string algorith var body = FileBlock("9B623C7C", HttpSource("https://example.com/a.bin"), sizeAttribute: $" RawHash=\"3507CD54\" RawHashAlgorithm=\"{algorithm}\""); File.WriteAllText(Path.Combine(src, "metadata.xml"), ManifestXml(body)); - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "x"); + var act = () => MetadataBuilder.Build(src, dst); act.Should().Throw().WithMessage("*RawHashAlgorithm*"); Directory.Exists(dst).Should().BeFalse(); } @@ -497,12 +497,12 @@ public void Build_NewFormatManifest_Succeeds() try { SeedSchemas(src); - File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); + File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); File.Copy( Path.Combine(AppContext.BaseDirectory, "TestData", "manifest-full.xml"), Path.Combine(src, "manifest.xml")); - MetadataBuilder.Build(src, dst, contentRevision: "x"); + MetadataBuilder.Build(src, dst); File.Exists(Path.Combine(dst, "metadata.xml")).Should().BeTrue(); } finally @@ -521,12 +521,12 @@ public void Build_LegacyManifest_StillSucceeds() try { SeedSchemas(src); - File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); + File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); File.Copy( Path.Combine(AppContext.BaseDirectory, "TestData", "manifest-legacy.xml"), Path.Combine(src, "manifest.xml")); - MetadataBuilder.Build(src, dst, contentRevision: "x"); + MetadataBuilder.Build(src, dst); File.Exists(Path.Combine(dst, "metadata.xml")).Should().BeTrue(); } finally @@ -608,14 +608,14 @@ public void Build_SkudefManifest_Succeeds() try { SeedSchemas(src); - File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); + File.WriteAllText(Path.Combine(src, "metadata.xml"), ""); File.WriteAllText( Path.Combine(src, "manifest.xml"), ManifestXml(SkudefBlock( "" + "") + SingleFile())); - MetadataBuilder.Build(src, dst, contentRevision: "x"); + MetadataBuilder.Build(src, dst); File.Exists(Path.Combine(dst, "metadata.xml")).Should().BeTrue(); } finally diff --git a/Ra3.BattleNet.Metadata.Tests/PublishSurfaceTests.cs b/Ra3.BattleNet.Metadata.Tests/Building/PublishSurfaceTests.cs similarity index 69% rename from Ra3.BattleNet.Metadata.Tests/PublishSurfaceTests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/PublishSurfaceTests.cs index eb62a88..6af283b 100644 --- a/Ra3.BattleNet.Metadata.Tests/PublishSurfaceTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Building/PublishSurfaceTests.cs @@ -1,3 +1,4 @@ +using System.Xml.Linq; using FluentAssertions; using Microsoft.VisualStudio.TestTools.UnitTesting; @@ -30,22 +31,27 @@ public void Build_RepoSample_OutputHasOnlyFlattenedAndReferencedResources() try { - MetadataBuilder.Build(src, dst, schemaVersion: "1.0", contentRevision: "publish-test"); + MetadataBuilder.Build(src, dst); // 展平数据 + 入口 File.Exists(Path.Combine(dst, "metadata.xml")).Should().BeTrue(); File.ReadAllText(Path.Combine(dst, "metadata.xml")).Should().NotContain(" image.Source is not null)) { var relative = image.Source!.Replace('/', Path.DirectorySeparatorChar); File.ReadAllBytes(Path.Combine(dst, relative)) .Should().Equal(File.ReadAllBytes(Path.Combine(src, relative))); } - File.Exists(Path.Combine(dst, "mods", "corona", "manifests", "3.259.xml")).Should().BeTrue(); - File.Exists(Path.Combine(dst, "apps", "ra3battlenet", "manifests", "1.9.9.22.xml")).Should().BeTrue(); + foreach (var manifest in loaded.GetAllElements("Manifest").Where(node => node.Get("ID") is not null)) + { + var leaf = MetadataResourceUri.Resolve(Path.Combine(dst, "metadata.xml"), manifest.Get("Source")!); + File.Exists(leaf.LocalPath).Should().BeTrue(); + MetadataReader.Read(leaf.LocalPath).Root.ManifestRegistration(manifest.Get("ID")!) + .Should().NotBeNull(); + } // _redirects 自动带出 File.Exists(Path.Combine(dst, "_redirects")).Should().BeTrue(); @@ -83,13 +89,13 @@ public void Build_UrlOnlyImage_CopiesNothingForIt() File.WriteAllText(Path.Combine(src, "img.png"), "fake-image-bytes"); File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + """); - MetadataBuilder.Build(src, dst, contentRevision: "x"); + MetadataBuilder.Build(src, dst); // Url 优先：remote-only 不落盘；with-source 落盘 Directory.GetFiles(dst, "img.png", SearchOption.AllDirectories).Should().HaveCount(1); @@ -103,6 +109,31 @@ public void Build_UrlOnlyImage_CopiesNothingForIt() } } + [TestMethod] + public void Build_ImageUrlWithEscapingSource_DoesNotReadOrCopySource() + { + var temp = Path.Combine(Path.GetTempPath(), $"publish-url-source-{Guid.NewGuid():N}"); + var src = Path.Combine(temp, "src"); + var dst = Path.Combine(temp, "out"); + SeedSchemas(src); + try + { + File.WriteAllText(Path.Combine(src, "metadata.xml"), """ + + """); + MetadataBuilder.Build(src, dst); + var image = XDocument.Load(Path.Combine(dst, "metadata.xml")).Root!.Element("Image")!; + image.Attribute("Url")!.Value.Should().Be("https://example.com/img.png"); + image.Attribute("Source")!.Value.Should().Be("../missing.png"); + Directory.GetFiles(dst).Select(Path.GetFileName).Should().Equal("metadata.xml"); + } + finally + { + if (Directory.Exists(temp)) + Directory.Delete(temp, recursive: true); + } + } + [TestMethod] public void Build_RepoSample_LeafManifestIsFlattenedAndLoadable() { @@ -110,22 +141,27 @@ public void Build_RepoSample_LeafManifestIsFlattenedAndLoadable() try { - MetadataBuilder.Build(RepoMetadataDir, dst, schemaVersion: "1.0", contentRevision: "publish-test"); + MetadataBuilder.Build(RepoMetadataDir, dst); // 叶子清单不能照抄源文件：客户端解析器拒绝 Includes，也不该收到未替换的变量 - var leafPath = Path.Combine(dst, "mods", "corona", "manifests", "3.259.xml"); + var root = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; + var coronaPackage = root.Mods().Single(mod => mod.Id == "Corona").Packages.Single(package => package.Version == "3.259"); + var leafPath = MetadataResourceUri.Resolve(Path.Combine(dst, "metadata.xml"), + root.ManifestRegistration(coronaPackage.ManifestId!)!.Get("Source")!).LocalPath; var text = File.ReadAllText(leafPath); text.Should().NotContain(" f.Sources.Count > 0); entry.Files.Should().Contain(f => f.DownloadName != null); - var runtime = MetadataBuilder.Load(Path.Combine(dst, "apps", "webview2", "manifests", "156.0.4314.58.xml")) - .Find("Manifest")!.ToManifestEntry(); + var runtimePackage = root.Applications().Single(app => app.Id == "webview2").Packages.Single(); + var runtimePath = MetadataResourceUri.Resolve(Path.Combine(dst, "metadata.xml"), + root.ManifestRegistration(runtimePackage.ManifestId!)!.Get("Source")!).LocalPath; + var runtime = MetadataReader.Read(runtimePath).Root.Find("Manifest")!.ToManifestEntry(); runtime.HashAlgorithm.Should().Be("SHA256"); var file = runtime.Files.Should().ContainSingle().Subject; file.FileName.Should().Be("WebView2Runtime.zip"); @@ -154,10 +190,13 @@ public void Build_RepoSample_SkudefSurvivesFlattenInDeclaredOrder() try { - MetadataBuilder.Build(RepoMetadataDir, dst, schemaVersion: "1.0", contentRevision: "publish-test"); + MetadataBuilder.Build(RepoMetadataDir, dst); - var entry = MetadataBuilder.Load(Path.Combine(dst, "mods", "corona", "manifests", "3.259.xml")) - .Find("Manifest")!.ToManifestEntry(); + var root = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; + var package = root.Mods().Single(mod => mod.Id == "Corona").Packages.Single(package => package.Version == "3.259"); + var leafPath = MetadataResourceUri.Resolve(Path.Combine(dst, "metadata.xml"), + root.ManifestRegistration(package.ManifestId!)!.Get("Source")!).LocalPath; + var entry = MetadataReader.Read(leafPath).Root.Find("Manifest")!.ToManifestEntry(); var skudef = entry.Skudef; skudef.Should().NotBeNull("展平后叶子清单要保住 Skudef"); diff --git a/Ra3.BattleNet.Metadata.Tests/SchemaTests.cs b/Ra3.BattleNet.Metadata.Tests/Building/SchemaTests.cs similarity index 93% rename from Ra3.BattleNet.Metadata.Tests/SchemaTests.cs rename to Ra3.BattleNet.Metadata.Tests/Building/SchemaTests.cs index 1f45f87..b0bff02 100644 --- a/Ra3.BattleNet.Metadata.Tests/SchemaTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Building/SchemaTests.cs @@ -16,14 +16,14 @@ public void Build_SampleTree_PassesSourceAndPublishSchema() var dst = Path.Combine(Path.GetTempPath(), $"schema-ok-{Guid.NewGuid():N}"); try { - MetadataBuilder.Build(src, dst, contentRevision: "schema-ok"); + MetadataBuilder.Build(src, dst); var flat = Path.Combine(dst, "metadata.xml"); File.Exists(flat).Should().BeTrue(); var publishSchema = Path.Combine(src, SchemaValidator.PublishSchemaFileName); SchemaValidator.ValidateFile(flat, publishSchema).Should().BeEmpty(); - var loaded = MetadataBuilder.Load(flat); + var loaded = MetadataReader.Read(flat).Root; loaded.Applications().Should().Contain(a => a.Id == "RA3BattleNet"); loaded.Mods().Should().Contain(m => m.Id == "Corona"); loaded.Get("SchemaVersion").Should().NotBeNullOrWhiteSpace(); @@ -109,14 +109,14 @@ public void Build_ApplicationMissingId_FailsSourceSchema() File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + 1.0 """); - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "x"); + var act = () => MetadataBuilder.Build(src, dst); act.Should().Throw() .WithMessage("*XSD*"); Directory.Exists(dst).Should().BeFalse(); @@ -143,7 +143,7 @@ static string WriteTree(string temp, string application) Path.Combine(src, SchemaValidator.PublishSchemaFileName)); File.WriteAllText(Path.Combine(src, "metadata.xml"), $""" - + {application} """); @@ -168,7 +168,7 @@ static string WriteTree(string temp, string application) { if (ok) { - MetadataBuilder.Build(src, dst, contentRevision: "transfer-ad"); + MetadataBuilder.Build(src, dst); SchemaValidator .ValidateFile(Path.Combine(dst, "metadata.xml"), Path.Combine(src, SchemaValidator.PublishSchemaFileName)) @@ -176,7 +176,7 @@ static string WriteTree(string temp, string application) } else { - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "transfer-ad"); + var act = () => MetadataBuilder.Build(src, dst); act.Should().Throw().WithMessage("*XSD*"); Directory.Exists(dst).Should().BeFalse(); } @@ -238,7 +238,7 @@ public void Build_EntityDisplayNames_RequiresExactZhEn_UniqueAndNonBlank() File.WriteAllText(Path.Combine(src, "metadata.xml"), $""" - + {displayNames} @@ -246,13 +246,13 @@ public void Build_EntityDisplayNames_RequiresExactZhEn_UniqueAndNonBlank() if (fragment == null) { - MetadataBuilder.Build(src, dst, contentRevision: "display-name"); - MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")).Mods().Single().DisplayNames + MetadataBuilder.Build(src, dst); + MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root.Mods().Single().DisplayNames .Select(d => d.Language).Should().Equal(expected); } else { - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "display-name"); + var act = () => MetadataBuilder.Build(src, dst); act.Should().Throw().WithMessage($"*{fragment}*"); Directory.Exists(dst).Should().BeFalse("半残输出应被清理"); } @@ -274,7 +274,7 @@ public void PublishSchema_RejectsManifestWithoutSource() { File.WriteAllText(temp, """ - + """); @@ -295,8 +295,8 @@ public void ConsumerParse_CatalogResolvesIdsAndResources() var dst = Path.Combine(Path.GetTempPath(), $"parse-{Guid.NewGuid():N}"); try { - MetadataBuilder.Build(src, dst, contentRevision: "parse"); - var doc = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")); + MetadataBuilder.Build(src, dst); + var doc = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; var app = doc.Catalog().Application("RA3BattleNet"); app.Should().NotBeNull(); diff --git a/Ra3.BattleNet.Metadata.Tests/CacheJsonContractTests.cs b/Ra3.BattleNet.Metadata.Tests/Cache/CacheJsonContractTests.cs similarity index 86% rename from Ra3.BattleNet.Metadata.Tests/CacheJsonContractTests.cs rename to Ra3.BattleNet.Metadata.Tests/Cache/CacheJsonContractTests.cs index 2f8c8a0..eaf6c48 100644 --- a/Ra3.BattleNet.Metadata.Tests/CacheJsonContractTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Cache/CacheJsonContractTests.cs @@ -7,14 +7,13 @@ namespace Ra3.BattleNet.Metadata.Tests; /// -/// 缓存磁盘 JSON（origin.json 与 .etag）是跨版本契约：源生成上下文写出的字节 -/// 必须与旧运行时反射实现逐字节一致——默认属性命名、null 属性照常写出、无缩进、默认转义。 +/// 缓存磁盘 JSON（origin.json 与 .etag）保存来源绑定和原始正文校验器。 /// [TestClass] public class CacheJsonContractTests { [TestMethod] - public void Origin_SerializesToLegacyBytes() + public void Origin_SerializesPublishingBase() { var json = JsonSerializer.Serialize( new CacheOrigin { PublishingBase = "https://metadata.example/" }, @@ -25,7 +24,7 @@ public void Origin_SerializesToLegacyBytes() } [TestMethod] - public void Stamp_FullFields_SerializeToLegacyBytesInDeclarationOrder() + public void Stamp_SerializesBodyValidators() { var json = JsonSerializer.Serialize( new CacheStamp @@ -59,7 +58,7 @@ public void Stamp_XmlBody_KeepsNullFieldsAndFieldOrder() } [TestMethod] - public void Context_DeserializesLegacyFilesCaseInsensitively() + public void Context_ReadsCaseInsensitiveProperties() { var origin = JsonSerializer.Deserialize( """{"publishingBase":"https://metadata.example/"}""", diff --git a/Ra3.BattleNet.Metadata.Tests/MetadataClientImageTests.cs b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientImageTests.cs similarity index 82% rename from Ra3.BattleNet.Metadata.Tests/MetadataClientImageTests.cs rename to Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientImageTests.cs index d128c43..8790e03 100644 --- a/Ra3.BattleNet.Metadata.Tests/MetadataClientImageTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientImageTests.cs @@ -32,7 +32,7 @@ public async Task DisabledDiskCache_NeverCreatesOrReadsCacheDirectory() var root = await fixture.Client.RefreshRootAsync(RootUrl); root.Status.Should().Be(MetadataFreshness.Fresh); - var image = await fixture.Client.GetImageAsync(root, "icon"); + var image = await fixture.Client.GetImageAsync(root, "metadata/icon"); image.Status.Should().Be(MetadataFreshness.Fresh); image.Bytes.Should().Equal(IconBytes); image.ContentType.Should().Be("image/png"); @@ -51,8 +51,8 @@ public async Task DisabledDiskCache_SecondImageRequestReadsSourceAgain() fixture.Handler.Next = (request, _) => Task.FromResult( request.RequestUri!.AbsoluteUri == IconUrl ? Png(body) : Xml(RootXml())); - (await fixture.Client.GetImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Fresh); - (await fixture.Client.GetImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Fresh); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Fresh); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Fresh); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(2, "禁用缓存时不复用上一次的正文"); } @@ -63,7 +63,7 @@ public async Task Image_RelativeSource_IsStoredInReleaseTreeWithContentType() using var fixture = Fixture.Create(); var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); - var image = await fixture.Client.GetImageAsync(root, "icon"); + var image = await fixture.Client.GetImageAsync(root, "metadata/icon"); image.Status.Should().Be(MetadataFreshness.Fresh); var body = Directory.EnumerateFiles(fixture.Cache, "icon.png", SearchOption.AllDirectories).Single(); @@ -77,13 +77,13 @@ public async Task HasCachedImage_OnlyTrueOnceBodyIsUsable() using var fixture = Fixture.Create(); var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); - fixture.Client.HasCachedImage(root, "icon").Should().BeFalse("还没下载过"); + fixture.Client.HasCachedImage(root, "metadata/icon").Should().BeFalse("还没下载过"); fixture.Client.HasCachedImage(root, "没有登记的图").Should().BeFalse(); - await fixture.Client.GetImageAsync(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); - fixture.Client.HasCachedImage(root, "icon").Should().BeTrue(); - fixture.Client.HasCachedImage(root, "ICON").Should().BeTrue("登记 ID 大小写不敏感"); + fixture.Client.HasCachedImage(root, "metadata/icon").Should().BeTrue(); + fixture.Client.HasCachedImage(root, "metadata/ICON").Should().BeTrue("登记 ID 大小写不敏感"); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(1, "探针本身不发请求"); } @@ -94,9 +94,9 @@ public async Task HasCachedImage_WithoutDiskCache_IsAlwaysFalse() fixture.Handler.Next = (request, _) => Task.FromResult( request.RequestUri!.AbsoluteUri == IconUrl ? Png(IconBytes) : Xml(RootXml())); var root = await fixture.Client.RefreshRootAsync(RootUrl); - await fixture.Client.GetImageAsync(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); - fixture.Client.HasCachedImage(root, "icon").Should().BeFalse("非磁盘会话的图片读取永远要联网"); + fixture.Client.HasCachedImage(root, "metadata/icon").Should().BeFalse("非磁盘会话的图片读取永远要联网"); } [TestMethod] @@ -104,14 +104,14 @@ public async Task Image_WarmReads_DoNotValidate() { using var fixture = Fixture.Create(); var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); - await fixture.Client.GetImageAsync(root, "icon"); - var revision = fixture.Client.ImageRevision(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); + var revision = fixture.Client.ImageRevision(root, "metadata/icon"); var calls = fixture.Calls.Count(call => call.Uri == IconUrl); fixture.Handler.Next = (_, _) => throw new HttpRequestException("warm read must not touch the network"); for (var i = 0; i < 5; i++) { - var cached = await fixture.Client.GetImageAsync(root, "icon"); + var cached = await fixture.Client.GetImageAsync(root, "metadata/icon"); cached.Status.Should().Be(MetadataFreshness.Stale); cached.Bytes.Should().Equal(IconBytes); cached.Error.Should().BeNull(); @@ -119,7 +119,7 @@ public async Task Image_WarmReads_DoNotValidate() } fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(calls); - fixture.Client.ImageRevision(root, "icon").Should().Be(revision); + fixture.Client.ImageRevision(root, "metadata/icon").Should().Be(revision); revision.Should().NotBe(0); } @@ -128,8 +128,8 @@ public async Task Image_ExplicitRefresh_NotModified_KeepsBytesAndRevision() { using var fixture = Fixture.Create(); var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); - await fixture.Client.GetImageAsync(root, "icon"); - var revision = fixture.Client.ImageRevision(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); + var revision = fixture.Client.ImageRevision(root, "metadata/icon"); var updated = 0; fixture.Client.ImageUpdated += (_, _) => Interlocked.Increment(ref updated); fixture.Handler.Next = (request, _) => @@ -138,16 +138,16 @@ public async Task Image_ExplicitRefresh_NotModified_KeepsBytesAndRevision() return Task.FromResult(NotModified()); }; - var refreshed = await fixture.Client.RefreshImageAsync(root, "icon"); + var refreshed = await fixture.Client.RefreshImageAsync(root, "metadata/icon"); refreshed.Status.Should().Be(MetadataFreshness.Stale); refreshed.Error.Should().BeNull(); refreshed.Bytes.Should().Equal(IconBytes); fixture.Calls.Last().Conditional.Should().BeTrue(); - fixture.Client.ImageRevision(root, "icon").Should().Be(revision); + fixture.Client.ImageRevision(root, "metadata/icon").Should().Be(revision); Interlocked.CompareExchange(ref updated, 0, 0).Should().Be(0); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(2); - (await fixture.Client.GetImageAsync(root, "icon")).Bytes.Should().Equal(IconBytes); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Bytes.Should().Equal(IconBytes); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(2, "刷新之后的普通读取不再请求"); } @@ -156,8 +156,8 @@ public async Task Image_ExplicitRefresh_ChangedBytes_UpdatesCacheAndRaisesOnce() { using var fixture = Fixture.Create(); var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); - await fixture.Client.GetImageAsync(root, "icon"); - var revision = fixture.Client.ImageRevision(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); + var revision = fixture.Client.ImageRevision(root, "metadata/icon"); var changed = new byte[] { 9, 9, 9, 9 }; var raised = 0; fixture.Client.ImageUpdated += (_, args) => @@ -171,23 +171,23 @@ public async Task Image_ExplicitRefresh_ChangedBytes_UpdatesCacheAndRaisesOnce() return Task.FromResult(Png(changed, "\"icon-2\"")); }; - var refreshed = await fixture.Client.RefreshImageAsync(root, "icon"); + var refreshed = await fixture.Client.RefreshImageAsync(root, "metadata/icon"); refreshed.Status.Should().Be(MetadataFreshness.Fresh); refreshed.Error.Should().BeNull(); refreshed.Bytes.Should().Equal(changed); raised.Should().Be(1); - fixture.Client.ImageRevision(root, "icon").Should().NotBe(revision); - var updatedRevision = fixture.Client.ImageRevision(root, "icon"); + fixture.Client.ImageRevision(root, "metadata/icon").Should().NotBe(revision); + var updatedRevision = fixture.Client.ImageRevision(root, "metadata/icon"); var body = Directory.EnumerateFiles(fixture.Cache, "icon.png", SearchOption.AllDirectories).Single(); File.ReadAllBytes(body).Should().Equal(changed); fixture.Handler.Next = (_, _) => Task.FromResult(NotModified()); - var again = await fixture.Client.RefreshImageAsync(root, "icon"); + var again = await fixture.Client.RefreshImageAsync(root, "metadata/icon"); again.Bytes.Should().Equal(changed); again.Error.Should().BeNull(); raised.Should().Be(1); - fixture.Client.ImageRevision(root, "icon").Should().Be(updatedRevision); + fixture.Client.ImageRevision(root, "metadata/icon").Should().Be(updatedRevision); } [TestMethod] @@ -195,13 +195,13 @@ public async Task Image_RefreshFailure_KeepsOldBytesWithoutAnotherRead() { using var fixture = Fixture.Create(); var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); - await fixture.Client.GetImageAsync(root, "icon"); - var revision = fixture.Client.ImageRevision(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); + var revision = fixture.Client.ImageRevision(root, "metadata/icon"); var raised = 0; fixture.Client.ImageUpdated += (_, _) => Interlocked.Increment(ref raised); fixture.Handler.Next = (_, _) => Task.FromResult(new HttpResponseMessage(HttpStatusCode.ServiceUnavailable)); - var failed = await fixture.Client.RefreshImageAsync(root, "icon"); + var failed = await fixture.Client.RefreshImageAsync(root, "metadata/icon"); var calls = fixture.Calls.Count(call => call.Uri == IconUrl); failed.Ok.Should().BeTrue(); @@ -209,9 +209,9 @@ public async Task Image_RefreshFailure_KeepsOldBytesWithoutAnotherRead() failed.Bytes.Should().Equal(IconBytes); failed.Error.Should().Contain("503"); raised.Should().Be(0); - fixture.Client.ImageRevision(root, "icon").Should().Be(revision); + fixture.Client.ImageRevision(root, "metadata/icon").Should().Be(revision); fixture.Handler.Next = (_, _) => throw new HttpRequestException("ordinary read must not retry"); - (await fixture.Client.GetImageAsync(root, "icon")).Bytes.Should().Equal(IconBytes); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Bytes.Should().Equal(IconBytes); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(calls); } @@ -224,21 +224,21 @@ public async Task Image_ExplicitRefresh_AfterColdFailure_NotifiesWithoutAnotherR ? Xml(RootXml()) : new HttpResponseMessage(HttpStatusCode.NotFound)); var root = await fixture.Client.RefreshRootAsync(RootUrl); - (await fixture.Client.GetImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Unavailable); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Unavailable); var raised = 0; fixture.Client.ImageUpdated += (_, args) => { args.ImageUri.Should().Be(IconUrl); Interlocked.Increment(ref raised); - var during = fixture.Client.GetImageAsync(root, "icon").GetAwaiter().GetResult(); + var during = fixture.Client.GetImageAsync(root, "metadata/icon").GetAwaiter().GetResult(); during.Ok.Should().BeTrue(); during.Bytes.Should().Equal(IconBytes); }; fixture.Handler.Next = (request, _) => Task.FromResult( request.RequestUri!.AbsoluteUri == IconUrl ? Png(IconBytes, "\"icon-1\"") : Xml(RootXml())); - var refreshed = await fixture.Client.RefreshImageAsync(root, "icon"); + var refreshed = await fixture.Client.RefreshImageAsync(root, "metadata/icon"); var calls = fixture.Calls.Count(call => call.Uri == IconUrl); refreshed.Ok.Should().BeTrue(); @@ -246,7 +246,7 @@ public async Task Image_ExplicitRefresh_AfterColdFailure_NotifiesWithoutAnotherR refreshed.Error.Should().BeNull(); raised.Should().Be(1); calls.Should().Be(2, "失败一次，显式成功一次；通知回调里的读取不能再发请求"); - (await fixture.Client.GetImageAsync(root, "icon")).Bytes.Should().Equal(IconBytes); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Bytes.Should().Equal(IconBytes); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(calls); } @@ -261,22 +261,22 @@ public async Task Image_ColdFailure_BlocksOrdinaryReadsUntilExplicitRefreshInter : new HttpResponseMessage(HttpStatusCode.NotFound)); var root = await fixture.Client.RefreshRootAsync(RootUrl); - (await fixture.Client.GetImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Unavailable); - (await fixture.Client.GetImageAsync(root, "icon")).Error.Should().Contain("显式刷新"); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Unavailable); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Error.Should().Contain("显式刷新"); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(1); clock.Now += TimeSpan.FromMinutes(9); - await fixture.Client.GetImageAsync(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(1); - var refreshed = await fixture.Client.RefreshImageAsync(root, "icon"); + var refreshed = await fixture.Client.RefreshImageAsync(root, "metadata/icon"); refreshed.Status.Should().Be(MetadataFreshness.Unavailable); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(2, "显式刷新不受冷失败抑制"); - await fixture.Client.GetImageAsync(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(2, "失败的显式刷新之后，普通读取不能再发第二次"); clock.Now += TimeSpan.FromMinutes(10); - await fixture.Client.GetImageAsync(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(3); } @@ -297,15 +297,15 @@ public async Task Image_ColdReadAndRefresh_ShareOneFlightAndThenServeCache() return Png(IconBytes, "\"icon-1\""); }; - var cold = fixture.Client.GetImageAsync(root, "icon"); + var cold = fixture.Client.GetImageAsync(root, "metadata/icon"); await entered.Task.WaitAsync(TimeSpan.FromSeconds(5)); - var refresh = fixture.Client.RefreshImageAsync(root, "icon"); + var refresh = fixture.Client.RefreshImageAsync(root, "metadata/icon"); release.TrySetResult(); var results = await Task.WhenAll(cold, refresh); results.Should().OnlyContain(image => image.Ok && image.Bytes!.SequenceEqual(IconBytes)); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(1); - (await fixture.Client.GetImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Stale); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Stale); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(1); } @@ -317,11 +317,11 @@ public async Task Image_ConcurrentColdReads_ShareOneInstantRequest() request.RequestUri!.AbsoluteUri == RootUrl ? Xml(RootXml()) : Png(IconBytes, "\"icon-1\"")); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var reads = await Task.WhenAll(Enumerable.Range(0, 12).Select(_ => fixture.Client.GetImageAsync(root, "icon"))); + var reads = await Task.WhenAll(Enumerable.Range(0, 12).Select(_ => fixture.Client.GetImageAsync(root, "metadata/icon"))); reads.Should().OnlyContain(image => image.Ok && image.Bytes!.SequenceEqual(IconBytes)); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(1); - (await fixture.Client.GetImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Stale); + (await fixture.Client.GetImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Stale); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(1); } @@ -332,22 +332,22 @@ public async Task Image_SourceChange_ChangesRevisionEvenWhenBytesMatch() var source = "images/a.png"; fixture.Handler.Next = (request, _) => Task.FromResult( request.RequestUri!.AbsoluteUri == RootUrl - ? Xml(RootXml($"""""")) + ? Xml(RootXml($"""""")) : Png(IconBytes, "\"icon\"")); var rootA = await fixture.Client.RefreshRootAsync(RootUrl); - await fixture.Client.GetImageAsync(rootA, "icon"); - var revisionA = fixture.Client.ImageRevision(rootA, "icon"); + await fixture.Client.GetImageAsync(rootA, "metadata/icon"); + var revisionA = fixture.Client.ImageRevision(rootA, "metadata/icon"); source = "images/b.png"; var rootB = await fixture.Client.RefreshRootAsync(RootUrl); - var beforeDownload = fixture.Client.ImageRevision(rootB, "icon"); - await fixture.Client.GetImageAsync(rootB, "icon"); - var revisionB = fixture.Client.ImageRevision(rootB, "icon"); + var beforeDownload = fixture.Client.ImageRevision(rootB, "metadata/icon"); + await fixture.Client.GetImageAsync(rootB, "metadata/icon"); + var revisionB = fixture.Client.ImageRevision(rootB, "metadata/icon"); revisionA.Should().NotBe(0); beforeDownload.Should().NotBe(revisionA); revisionB.Should().NotBe(revisionA); - fixture.Client.ImageRevision(rootA, "icon").Should().Be(revisionA); + fixture.Client.ImageRevision(rootA, "metadata/icon").Should().Be(revisionA); } [TestMethod] @@ -358,8 +358,8 @@ public async Task Image_Revision_IsStableAcrossProcessAndTracksCachedBytes() using (var first = Fixture.Create(cache, keepCache: true)) { var root = await RootAsync(first, IconBytes, "\"icon-1\""); - await first.Client.GetImageAsync(root, "icon"); - revision = first.Client.ImageRevision(root, "icon"); + await first.Client.GetImageAsync(root, "metadata/icon"); + revision = first.Client.ImageRevision(root, "metadata/icon"); revision.Should().NotBe(0); } @@ -367,8 +367,8 @@ public async Task Image_Revision_IsStableAcrossProcessAndTracksCachedBytes() { second.Handler.Next = (_, _) => throw new HttpRequestException("restart must not refetch unchanged bytes"); var opened = await second.Client.OpenSnapshotAsync(RootUrl); - second.Client.ImageRevision(opened, "icon").Should().Be(revision); - (await second.Client.GetImageAsync(opened, "icon")).Bytes.Should().Equal(IconBytes); + second.Client.ImageRevision(opened, "metadata/icon").Should().Be(revision); + (await second.Client.GetImageAsync(opened, "metadata/icon")).Bytes.Should().Equal(IconBytes); second.Calls.Should().BeEmpty(); } @@ -377,8 +377,8 @@ public async Task Image_Revision_IsStableAcrossProcessAndTracksCachedBytes() using var third = Fixture.Create(cache); third.Handler.Next = (_, _) => throw new HttpRequestException("changed disk bytes are already cached"); var reopened = await third.Client.OpenSnapshotAsync(RootUrl); - third.Client.ImageRevision(reopened, "icon").Should().NotBe(revision); - (await third.Client.GetImageAsync(reopened, "icon")).Bytes.Should().Equal(changed); + third.Client.ImageRevision(reopened, "metadata/icon").Should().NotBe(revision); + (await third.Client.GetImageAsync(reopened, "metadata/icon")).Bytes.Should().Equal(changed); third.Calls.Should().BeEmpty(); } @@ -399,13 +399,13 @@ public async Task Image_CallerCancel_DoesNotCancelSharedDownload() return Png(IconBytes, "\"icon-1\""); }; using var cts = new CancellationTokenSource(); - var cancelled = fixture.Client.GetImageAsync(root, "icon", cts.Token); + var cancelled = fixture.Client.GetImageAsync(root, "metadata/icon", cts.Token); await entered.Task.WaitAsync(TimeSpan.FromSeconds(5)); cts.Cancel(); var act = async () => await cancelled; await act.Should().ThrowAsync(); - var joined = fixture.Client.GetImageAsync(root, "icon"); + var joined = fixture.Client.GetImageAsync(root, "metadata/icon"); release.TrySetResult(); (await joined).Bytes.Should().Equal(IconBytes); @@ -427,7 +427,7 @@ public async Task Image_Dispose_CancelsOutstandingDownload() await Task.Delay(Timeout.Infinite, token); return Png(IconBytes); }; - var pending = fixture.Client.GetImageAsync(root, "icon"); + var pending = fixture.Client.GetImageAsync(root, "metadata/icon"); await entered.Task.WaitAsync(TimeSpan.FromSeconds(5)); fixture.Client.Dispose(); @@ -444,8 +444,8 @@ public async Task DisabledDiskCache_RefreshStillReadsLiveSource() fixture.Handler.Next = (request, _) => Task.FromResult( request.RequestUri!.AbsoluteUri == IconUrl ? Png(IconBytes) : Xml(RootXml())); - (await fixture.Client.RefreshImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Fresh); - (await fixture.Client.RefreshImageAsync(root, "icon")).Status.Should().Be(MetadataFreshness.Fresh); + (await fixture.Client.RefreshImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Fresh); + (await fixture.Client.RefreshImageAsync(root, "metadata/icon")).Status.Should().Be(MetadataFreshness.Fresh); fixture.Calls.Count(call => call.Uri == IconUrl).Should().Be(2); fixture.Calls.Where(call => call.Uri == IconUrl).Should().OnlyContain(call => !call.Conditional); @@ -458,14 +458,14 @@ public async Task Image_QueryUrl_UsesRequestKeyAndKeepsQueriesApart() fixture.Handler.Next = (request, _) => Task.FromResult( request.RequestUri!.AbsoluteUri == RootUrl ? Xml(RootXml(""" - - + + """)) : Png([1, 2, 3, 4])); var root = await fixture.Client.RefreshRootAsync(RootUrl); - await fixture.Client.GetImageAsync(root, "banner-a"); - await fixture.Client.GetImageAsync(root, "banner-b"); + await fixture.Client.GetImageAsync(root, "metadata/banner-a"); + await fixture.Client.GetImageAsync(root, "metadata/banner-b"); var bodies = Directory.EnumerateFiles(fixture.Cache, "body", SearchOption.AllDirectories).ToList(); bodies.Should().HaveCount(2, "不同查询串不能撞到同一份正文"); @@ -487,7 +487,7 @@ public async Task Image_NonImageResponse_IsRejectedAndNotCached() : Png(Encoding.UTF8.GetBytes(""), contentType: "text/html")); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var image = await fixture.Client.GetImageAsync(root, "icon"); + var image = await fixture.Client.GetImageAsync(root, "metadata/icon"); image.Status.Should().Be(MetadataFreshness.Unavailable); Directory.EnumerateFiles(fixture.Cache, "icon.png", SearchOption.AllDirectories).Should().BeEmpty(); @@ -498,12 +498,12 @@ public async Task Image_CorruptCacheBody_IsRefetchedInsteadOfServed() { using var fixture = Fixture.Create(); var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); - await fixture.Client.GetImageAsync(root, "icon"); + await fixture.Client.GetImageAsync(root, "metadata/icon"); var body = Directory.EnumerateFiles(fixture.Cache, "icon.png", SearchOption.AllDirectories).Single(); File.WriteAllBytes(body, [7, 7, 7]); fixture.Handler.Next = (_, _) => Task.FromResult(Png(IconBytes, "\"icon-1\"")); - var image = await fixture.Client.GetImageAsync(root, "icon"); + var image = await fixture.Client.GetImageAsync(root, "metadata/icon"); image.Status.Should().Be(MetadataFreshness.Fresh); image.Bytes.Should().Equal(IconBytes); @@ -516,12 +516,12 @@ public async Task Image_UnregisteredId_IsUnavailableWithoutRequest() var root = await RootAsync(fixture, IconBytes, "\"icon-1\""); var before = fixture.Calls.Count; - var image = await fixture.Client.GetImageAsync(root, "mods/nope:icon"); + var image = await fixture.Client.GetImageAsync(root, "mods/nope/icon"); image.Status.Should().Be(MetadataFreshness.Unavailable); image.Error.Should().NotBeNullOrWhiteSpace(); fixture.Calls.Should().HaveCount(before); - fixture.Client.ImageRevision(root, "mods/nope:icon").Should().Be(0); + fixture.Client.ImageRevision(root, "mods/nope/icon").Should().Be(0); } [TestMethod] @@ -530,7 +530,7 @@ public async Task Image_MissingDocument_IsUnavailable() using var fixture = Fixture.Create(); var snapshot = new RootSnapshotResult(null, null, null, MetadataFreshness.Unavailable, "x", DateTimeOffset.UtcNow, "none"); - (await fixture.Client.GetImageAsync(snapshot, "icon")).Status.Should().Be(MetadataFreshness.Unavailable); + (await fixture.Client.GetImageAsync(snapshot, "metadata/icon")).Status.Should().Be(MetadataFreshness.Unavailable); } [TestMethod] @@ -589,9 +589,9 @@ private sealed class ManualClock : TimeProvider public override DateTimeOffset GetUtcNow() => Now; } - private static string RootXml(string images = """""") => $$""" + private static string RootXml(string images = """""") => $$""" - + 1.0.0 diff --git a/Ra3.BattleNet.Metadata.Tests/MetadataClientTests.cs b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTests.cs similarity index 83% rename from Ra3.BattleNet.Metadata.Tests/MetadataClientTests.cs rename to Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTests.cs index bd60275..8f65cc9 100644 --- a/Ra3.BattleNet.Metadata.Tests/MetadataClientTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTests.cs @@ -16,6 +16,9 @@ public class MetadataClientTests { private const string RootUrl = "https://metadata.example/metadata.xml"; + private static PackageEntry Package(RootSnapshotResult snapshot, string version) => + snapshot.Document!.Applications().Single().Packages.Single(package => package.Version == version); + [TestMethod] public async Task RawFingerprint_SurvivesDiskCacheReopen() { @@ -24,14 +27,13 @@ public async Task RawFingerprint_SurvivesDiskCacheReopen() fixture.Handler.Next = (request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : leafXml)); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var uri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - var loaded = await fixture.Client.GetLeafAsync(root, "1.0.0", uri); + var loaded = await fixture.Client.GetPackageAsync(root, Package(root, "1.0.0")); loaded.Entry!.Files[0].RawHash.Should().Be("3507CD54"); loaded.Entry.Files[0].RawHashAlgorithm.Should().Be("CRC32C"); using var reopened = new MetadataClient(fixture.Cache, TimeSpan.FromSeconds(5)); var snapshot = await reopened.OpenSnapshotAsync(RootUrl); - var cached = await reopened.GetCachedLeafAsync(snapshot, "1.0.0", uri); + var cached = await reopened.GetCachedPackageAsync(snapshot, Package(snapshot, "1.0.0")); cached.Status.Should().Be(MetadataFreshness.Stale); cached.Entry!.Files[0].RawHash.Should().Be("3507CD54"); cached.Entry.Files[0].RawHashAlgorithm.Should().Be("CRC32C"); @@ -45,7 +47,7 @@ public async Task InvalidRawFingerprint_IsRejectedBeforeCaching() fixture.Handler.Next = (request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : leafXml)); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var leaf = await fixture.Client.GetLeafAsync(root, "1.0.0", MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml")); + var leaf = await fixture.Client.GetPackageAsync(root, Package(root, "1.0.0")); leaf.Status.Should().Be(MetadataFreshness.Unavailable); leaf.Error.Should().Contain("RawHash"); leaf.Entry.Should().BeNull(); @@ -246,18 +248,17 @@ public async Task Preload_ReportsLeavesThatEndedWithoutDocument() } [TestMethod] - public async Task GetLeaf_MemoryHit_DoesNotRefetch_RefreshLeafDoes() + public async Task GetPackage_MemoryHit_DoesNotRefetch_RefreshLeafDoes() { using var fixture = Fixture.Create(); fixture.Handler.Next = (request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : LeafXml("app-leaf"), "\"v1\"")); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - var loaded = await fixture.Client.RefreshLeafAsync(root, "1.0.0", leafUri); + var loaded = await fixture.Client.RefreshPackageAsync(root, Package(root, "1.0.0")); var before = fixture.Calls.Count; - var shared = await fixture.Client.GetLeafAsync(root, "1.0.0", leafUri); + var shared = await fixture.Client.GetPackageAsync(root, Package(root, "1.0.0")); shared.Document.Should().BeSameAs(loaded.Document); shared.ManifestNode.Should().BeSameAs(loaded.ManifestNode); shared.Entry.Should().BeSameAs(loaded.Entry); @@ -269,42 +270,56 @@ public async Task GetLeaf_MemoryHit_DoesNotRefetch_RefreshLeafDoes() return Task.FromResult(NotModified()); return Task.FromResult(Xml(LeafXml("app-leaf"), "\"v1\"")); }; - var refreshed = await fixture.Client.RefreshLeafAsync(root, "1.0.0", leafUri); + var refreshed = await fixture.Client.RefreshPackageAsync(root, Package(root, "1.0.0")); refreshed.Status.Should().Be(MetadataFreshness.Fresh); refreshed.ManifestNode.Should().BeSameAs(loaded.ManifestNode); fixture.Calls.Should().HaveCount(before + 1); } [TestMethod] - public async Task Leaf_RejectsStubIncludeAndMissingFile_AcceptsEmptyManifest() + public async Task Leaf_RejectsStubCycleAndMissingFile_AcceptsEmptyManifest() { - await AssertLeafRejected(StubXml(), "存根"); - await AssertLeafRejected(IncludeXml(), "Include"); - await AssertLeafRejected(MissingHashXml(), "FileName 或 Hash"); + await AssertLeafRejected(StubXml()); + await AssertLeafRejected(CyclicLeafXml()); + await AssertLeafRejected(MissingHashXml()); using var fixture = Fixture.Create(); fixture.Handler.Next = (request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : EmptyLeafXml())); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var leaf = await fixture.Client.RefreshLeafAsync(root, "1.0.0", MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml")); + var leaf = await fixture.Client.RefreshPackageAsync(root, Package(root, "1.0.0")); leaf.Status.Should().Be(MetadataFreshness.Fresh); leaf.Entry!.Files.Should().BeEmpty(); leaf.ManifestNode.Should().NotBeNull(); } [TestMethod] - public async Task Leaf_VersionMismatch_DoesNotFetch() + public async Task Leaf_StubSource_IsRejectedWithoutFollowingItself() { using var fixture = Fixture.Create(); - fixture.Handler.Next = (_, _) => Task.FromResult(Xml(RootXml())); + fixture.Handler.Next = (request, _) => Task.FromResult(Xml( + request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : StubXml())); var root = await fixture.Client.RefreshRootAsync(RootUrl); - fixture.Calls.Clear(); - - var leaf = await fixture.Client.GetLeafAsync(root, "9.9.9", MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml")); + var leaf = await fixture.Client.RefreshPackageAsync(root, Package(root, "1.0.0")); leaf.Status.Should().Be(MetadataFreshness.Unavailable); - leaf.Entry.Should().BeNull(); - fixture.Calls.Should().BeEmpty(); + leaf.Error.Should().Contain("存根"); + // 只请求根清单和目标叶子：存根里的 Source 不能再被当成待内联的正文追下去。 + fixture.Calls.Should().HaveCount(2); + } + + [TestMethod] + public async Task PackageFromDifferentSnapshot_IsRejectedWithoutFetch() + { + using var fixture = Fixture.Create(); + fixture.Handler.Next = (_, _) => Task.FromResult(Xml(RootXml())); + var first = await fixture.Client.RefreshRootAsync(RootUrl); + var package = first.Document!.Applications().Single().Packages.First(); + fixture.Handler.Next = (_, _) => Task.FromResult(Xml(RootXml("apps/b.xml", "leaf-b"))); + var second = await fixture.Client.RefreshRootAsync(RootUrl); + var before = fixture.Calls.Count; + (await fixture.Client.GetPackageAsync(second, package)).Status.Should().Be(MetadataFreshness.Unavailable); + fixture.Calls.Should().HaveCount(before); } [TestMethod] @@ -327,21 +342,20 @@ public async Task SourceChange_UsesNewUri_AndKeepsCapturedUriDistinct() }; var capturedRoot = await fixture.Client.RefreshRootAsync(RootUrl); - var oldUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - var oldLeaf = await fixture.Client.RefreshLeafAsync(capturedRoot, "1.0.0", oldUri); + var oldLeaf = await fixture.Client.RefreshPackageAsync(capturedRoot, Package(capturedRoot, "1.0.0")); generation = 1; var newer = await fixture.Client.RefreshRootAsync(RootUrl); var newUri = MetadataResourceUri.Resolve(RootUrl, "apps/b.xml"); var before = fixture.Calls.Count; - var captured = await fixture.Client.GetLeafAsync(capturedRoot, "1.0.0", oldUri); + var captured = await fixture.Client.GetPackageAsync(capturedRoot, Package(capturedRoot, "1.0.0")); captured.Document.Should().BeSameAs(oldLeaf.Document); fixture.Calls.Should().HaveCount(before); - var moved = await fixture.Client.GetLeafAsync(newer, "1.0.0", newUri); + var moved = await fixture.Client.GetPackageAsync(newer, Package(newer, "1.0.0")); moved.Document.Should().NotBeSameAs(oldLeaf.Document); moved.Entry!.Files[0].FileName.Should().Be("b.bin"); - moved.SourceUri.AbsoluteUri.Should().Be(newUri.AbsoluteUri); + moved.SourceUri.Should().Be(newUri); } [TestMethod] @@ -410,9 +424,9 @@ public async Task FileEntry_RefreshIsFresh_AndResolvesLeavesAgainstOriginNotCach var leafUri = MetadataResourceUri.Resolve(root.OriginUri.AbsoluteUri, "apps/leaf.xml"); leafUri.LocalPath.Should().StartWith(Path.GetFullPath(source)); - var leaf = await client.RefreshLeafAsync(root, "1.0.0", leafUri); + var leaf = await client.RefreshPackageAsync(root, Package(root, "1.0.0")); leaf.Status.Should().Be(MetadataFreshness.Fresh); - leaf.Entry!.Id.Should().Be("app-leaf"); + leaf.Entry!.Id.Should().Be("apps/leaf/app-leaf"); handler.Calls.Should().BeEmpty(); var other = await client.OpenSnapshotAsync("https://other.example/metadata.xml"); @@ -426,16 +440,18 @@ public async Task FileEntry_RefreshIsFresh_AndResolvesLeavesAgainstOriginNotCach } } - private static async Task AssertLeafRejected(string leafXml, string errorFragment) + private static async Task AssertLeafRejected(string leafXml) { using var fixture = Fixture.Create(); fixture.Handler.Next = (request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : leafXml)); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var leaf = await fixture.Client.RefreshLeafAsync(root, "1.0.0", MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml")); + var leaf = await fixture.Client.RefreshPackageAsync(root, Package(root, "1.0.0")); leaf.Status.Should().Be(MetadataFreshness.Unavailable); leaf.Entry.Should().BeNull(); - leaf.Error.Should().Contain(errorFragment); + leaf.Error.Should().NotBeNullOrWhiteSpace(); + root.Status.Should().Be(MetadataFreshness.Fresh); + root.Document.Should().NotBeNull(); if (Directory.Exists(fixture.Cache)) Directory.EnumerateFiles(fixture.Cache, "leaf.xml", SearchOption.AllDirectories).Should().BeEmpty(); } @@ -468,7 +484,7 @@ public async Task OtherOrigin_DoesNotEraseCachedRoot() request.RequestUri!.Host == "other.example" ? """ - + 9.0 """ @@ -509,20 +525,19 @@ public async Task NotModified_WhenVerifiedBodyDoesNotParse_RetriesUnconditionalO } [TestMethod] - public async Task GetLeaf_SameUriDifferentManifestId_DoesNotReturnWrongEntry() + public async Task GetPackage_SameUriDifferentManifestId_DoesNotReturnWrongEntry() { using var fixture = Fixture.Create(); fixture.Handler.Next = (request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? SharedSourceRoot() : BothIdsLeafXml())); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - var first = await fixture.Client.GetLeafAsync(root, "1.0.0", leafUri); + var first = await fixture.Client.GetPackageAsync(root, Package(root, "1.0.0")); var before = fixture.Calls.Count; - var second = await fixture.Client.GetLeafAsync(root, "2.0.0", leafUri); - var missing = await fixture.Client.GetLeafAsync(root, "3.0.0", leafUri); + var second = await fixture.Client.GetPackageAsync(root, Package(root, "2.0.0")); + var missing = await fixture.Client.GetPackageAsync(root, Package(root, "3.0.0")); - first.Entry!.Id.Should().Be("id-a"); - second.Entry!.Id.Should().Be("id-b"); + first.Entry!.Id.Should().Be("apps/leaf/id-a"); + second.Entry!.Id.Should().Be("apps/leaf/id-b"); second.Document.Should().BeSameAs(first.Document); fixture.Calls.Should().HaveCount(before); missing.Status.Should().Be(MetadataFreshness.Unavailable); @@ -530,35 +545,10 @@ public async Task GetLeaf_SameUriDifferentManifestId_DoesNotReturnWrongEntry() missing.ManifestNode.Should().BeNull(); } - [TestMethod] - public async Task SameVersionSameUri_CapturedRoots_KeepTheirOwnManifestId() - { - using var fixture = Fixture.Create(); - var generation = 0; - fixture.Handler.Next = (request, _) => Task.FromResult(Xml( - request.RequestUri!.AbsoluteUri == RootUrl - ? IdRoot(generation == 0 ? "id-a" : "id-b") - : BothIdsLeafXml())); - var rootA = await fixture.Client.RefreshRootAsync(RootUrl); - var uri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - var fromA = await fixture.Client.GetLeafAsync(rootA, "1.0.0", uri); - var before = fixture.Calls.Count; - generation = 1; - var rootB = await fixture.Client.RefreshRootAsync(RootUrl); - var fromB = await fixture.Client.GetLeafAsync(rootB, "1.0.0", uri); - var againA = await fixture.Client.GetLeafAsync(rootA, "1.0.0", uri); - - fromA.Entry!.Id.Should().Be("id-a"); - fromB.Entry!.Id.Should().Be("id-b"); - fromB.Document.Should().BeSameAs(fromA.Document); - againA.Entry.Should().BeSameAs(fromA.Entry); - fromB.Entry.Should().NotBeSameAs(fromA.Entry); - fixture.Calls.Should().HaveCount(before + 1); - } [TestMethod] - public async Task GetLeaf_Canceled_DoesNotFetch() + public async Task GetPackage_Canceled_DoesNotFetch() { using var fixture = Fixture.Create(); fixture.Handler.Next = (_, _) => Task.FromResult(Xml(RootXml())); @@ -567,45 +557,24 @@ public async Task GetLeaf_Canceled_DoesNotFetch() using var cts = new CancellationTokenSource(); cts.Cancel(); - var act = async () => await fixture.Client.GetLeafAsync( - root, "1.0.0", MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"), cts.Token); + var act = async () => await fixture.Client.GetPackageAsync(root, Package(root, "1.0.0"), cts.Token); await act.Should().ThrowAsync(); fixture.Calls.Should().BeEmpty(); } - [TestMethod] - public async Task GetCachedLeaf_Missing_IsUnavailableAndDoesNotFetch() - { - using var fixture = Fixture.Create(); - fixture.Handler.Next = (request, _) => Task.FromResult(Xml( - request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : LeafXml("app-leaf"))); - var root = await fixture.Client.RefreshRootAsync(RootUrl); - var before = fixture.Calls.Count; - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - - var cached = await fixture.Client.GetCachedLeafAsync(root, "1.0.0", leafUri); - - cached.Status.Should().Be(MetadataFreshness.Unavailable); - cached.Document.Should().BeNull(); - cached.ManifestNode.Should().BeNull(); - cached.Entry.Should().BeNull(); - cached.Error.Should().NotBeNullOrWhiteSpace(); - fixture.Calls.Should().HaveCount(before); - } [TestMethod] - public async Task GetCachedLeaf_MemoryAndDisk_DoNotFetchAndStayStale() + public async Task GetCachedPackage_MemoryHit_DoesNotFetchAndStaysStale() { using var fixture = Fixture.Create(); fixture.Handler.Next = (request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : LeafXml("app-leaf"))); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - var fresh = await fixture.Client.RefreshLeafAsync(root, "1.0.0", leafUri); + var fresh = await fixture.Client.RefreshPackageAsync(root, Package(root, "1.0.0")); var before = fixture.Calls.Count; - var memory = await fixture.Client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var memory = await fixture.Client.GetCachedPackageAsync(root, Package(root, "1.0.0")); memory.Status.Should().Be(MetadataFreshness.Stale); memory.Error.Should().BeNull(); @@ -616,33 +585,32 @@ public async Task GetCachedLeaf_MemoryAndDisk_DoNotFetchAndStayStale() } [TestMethod] - public async Task GetCachedLeaf_NewProcess_ReadsDiskWithoutHttp() + public async Task GetCachedPackage_NewProcess_ReadsDiskWithoutHttp() { var cache = NewCache(); var handler = new ScriptedHandler((request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : LeafXml("app-leaf")))); using var http = new HttpClient(handler); - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); using (var first = Client(cache, http)) { var root = await first.RefreshRootAsync(RootUrl); - (await first.RefreshLeafAsync(root, "1.0.0", leafUri)).Status.Should().Be(MetadataFreshness.Fresh); + (await first.RefreshPackageAsync(root, Package(root, "1.0.0"))).Status.Should().Be(MetadataFreshness.Fresh); } handler.Calls.Clear(); using var second = Client(cache, http); var opened = await second.OpenSnapshotAsync(RootUrl); var reads = await Task.WhenAll(Enumerable.Range(0, 8).Select(_ => - second.GetCachedLeafAsync(opened, "1.0.0", leafUri))); + second.GetCachedPackageAsync(opened, Package(opened, "1.0.0")))); - reads.Should().OnlyContain(item => item.Status == MetadataFreshness.Stale && item.Entry!.Id == "app-leaf"); + reads.Should().OnlyContain(item => item.Status == MetadataFreshness.Stale && item.Entry!.Id == "apps/leaf/app-leaf"); reads.Select(item => item.Document).Distinct().Should().ContainSingle(); handler.Calls.Should().BeEmpty(); TryDelete(cache); } [TestMethod] - public async Task GetCachedLeaf_SameUriDifferentSnapshots_KeepOwnManifest() + public async Task GetCachedPackage_SameUriDifferentSnapshots_KeepOwnManifest() { using var fixture = Fixture.Create(); var generation = 0; @@ -651,17 +619,16 @@ public async Task GetCachedLeaf_SameUriDifferentSnapshots_KeepOwnManifest() ? IdRoot(generation == 0 ? "id-a" : "id-b") : BothIdsLeafXml())); var rootA = await fixture.Client.RefreshRootAsync(RootUrl); - var uri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); - (await fixture.Client.GetLeafAsync(rootA, "1.0.0", uri)).Entry!.Id.Should().Be("id-a"); + (await fixture.Client.GetPackageAsync(rootA, Package(rootA, "1.0.0"))).Entry!.Id.Should().Be("apps/leaf/id-a"); generation = 1; var rootB = await fixture.Client.RefreshRootAsync(RootUrl); var before = fixture.Calls.Count; - var fromA = await fixture.Client.GetCachedLeafAsync(rootA, "1.0.0", uri); - var fromB = await fixture.Client.GetCachedLeafAsync(rootB, "1.0.0", uri); + var fromA = await fixture.Client.GetCachedPackageAsync(rootA, Package(rootA, "1.0.0")); + var fromB = await fixture.Client.GetCachedPackageAsync(rootB, Package(rootB, "1.0.0")); - fromA.Entry!.Id.Should().Be("id-a"); - fromB.Entry!.Id.Should().Be("id-b"); + fromA.Entry!.Id.Should().Be("apps/leaf/id-a"); + fromB.Entry!.Id.Should().Be("apps/leaf/id-b"); fromB.Document.Should().BeSameAs(fromA.Document); fromB.Entry.Should().NotBeSameAs(fromA.Entry); fromA.Status.Should().Be(MetadataFreshness.Stale); @@ -669,12 +636,11 @@ public async Task GetCachedLeaf_SameUriDifferentSnapshots_KeepOwnManifest() } [TestMethod] - public async Task GetCachedLeaf_DoesNotWaitForInFlightFetch() + public async Task GetCachedPackage_DoesNotWaitForInFlightFetch() { using var fixture = Fixture.Create(); fixture.Handler.Next = (request, _) => Task.FromResult(Xml(RootXml())); var root = await fixture.Client.RefreshRootAsync(RootUrl); - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); var entered = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); fixture.Handler.Next = async (request, token) => @@ -685,11 +651,11 @@ public async Task GetCachedLeaf_DoesNotWaitForInFlightFetch() await release.Task.WaitAsync(token); return Xml(LeafXml("app-leaf")); }; - var inflight = fixture.Client.GetLeafAsync(root, "1.0.0", leafUri); + var inflight = fixture.Client.GetPackageAsync(root, Package(root, "1.0.0")); await entered.Task.WaitAsync(TimeSpan.FromSeconds(5)); var before = fixture.Calls.Count; - var cached = await fixture.Client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var cached = await fixture.Client.GetCachedPackageAsync(root, Package(root, "1.0.0")); cached.Status.Should().Be(MetadataFreshness.Unavailable); fixture.Calls.Should().HaveCount(before); @@ -698,7 +664,7 @@ public async Task GetCachedLeaf_DoesNotWaitForInFlightFetch() } [TestMethod] - public async Task GetCachedLeaf_NoDiskCache_FileSourceReadsCurrentFile() + public async Task GetCachedPackage_NoDiskCache_FileSourceReadsCurrentFile() { var source = Path.Combine(Path.GetTempPath(), "metadata-cached-leaf-" + Guid.NewGuid().ToString("N")); Directory.CreateDirectory(Path.Combine(source, "apps")); @@ -712,11 +678,10 @@ public async Task GetCachedLeaf_NoDiskCache_FileSourceReadsCurrentFile() try { var root = await client.RefreshRootAsync(metadataPath); - var leafUri = MetadataResourceUri.Resolve(root.OriginUri!.AbsoluteUri, "apps/leaf.xml"); - var first = await client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var first = await client.GetCachedPackageAsync(root, Package(root, "1.0.0")); File.WriteAllText(leafPath, LeafXml("app-leaf", "b.bin"), new UTF8Encoding(false)); - var second = await client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var second = await client.GetCachedPackageAsync(root, Package(root, "1.0.0")); first.Status.Should().Be(MetadataFreshness.Stale); first.Entry!.Files[0].FileName.Should().Be("a.bin"); @@ -731,33 +696,32 @@ public async Task GetCachedLeaf_NoDiskCache_FileSourceReadsCurrentFile() } [TestMethod] - public async Task GetCachedLeaf_NoDiskCache_HttpDoesNotFetch() + public async Task GetCachedPackage_NoDiskCache_HttpDoesNotFetch() { var handler = new ScriptedHandler((request, _) => Task.FromResult(Xml( request.RequestUri!.AbsoluteUri == RootUrl ? RootXml() : LeafXml("app-leaf")))); using var http = new HttpClient(handler); using var client = new MetadataClient("", TimeSpan.FromSeconds(5), httpClient: http, enableDiskCache: false); var root = await client.RefreshRootAsync(RootUrl); - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); var before = handler.Calls.Count; - var missing = await client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var missing = await client.GetCachedPackageAsync(root, Package(root, "1.0.0")); missing.Status.Should().Be(MetadataFreshness.Unavailable); handler.Calls.Should().HaveCount(before); - var loaded = await client.GetLeafAsync(root, "1.0.0", leafUri); + var loaded = await client.GetPackageAsync(root, Package(root, "1.0.0")); var afterLoad = handler.Calls.Count; - var cached = await client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var cached = await client.GetCachedPackageAsync(root, Package(root, "1.0.0")); loaded.Status.Should().Be(MetadataFreshness.Fresh); cached.Status.Should().Be(MetadataFreshness.Stale); - cached.Entry!.Id.Should().Be("app-leaf"); + cached.Entry!.Id.Should().Be("apps/leaf/app-leaf"); cached.Document.Should().BeSameAs(loaded.Document); handler.Calls.Should().HaveCount(afterLoad); } [TestMethod] - public async Task GetCachedLeaf_DiskCache_DoesNotReadUncachedFileSource() + public async Task GetCachedPackage_DiskCache_DoesNotReadUncachedFileSource() { var source = Path.Combine(Path.GetTempPath(), "metadata-cached-miss-" + Guid.NewGuid().ToString("N")); var cache = NewCache(); @@ -771,9 +735,8 @@ public async Task GetCachedLeaf_DiskCache_DoesNotReadUncachedFileSource() try { var root = await client.RefreshRootAsync(metadataPath); - var leafUri = MetadataResourceUri.Resolve(root.OriginUri!.AbsoluteUri, "apps/leaf.xml"); - var cached = await client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var cached = await client.GetCachedPackageAsync(root, Package(root, "1.0.0")); cached.Status.Should().Be(MetadataFreshness.Unavailable); handler.Calls.Should().BeEmpty(); @@ -815,9 +778,6 @@ public async Task RefreshRoot_MapsPublishedTreeDirectlyUnderCache() File.Exists(Path.Combine(cache, "apps", "old.xml")).Should().BeTrue(); File.Exists(Path.Combine(cache, "mods", "current.xml")).Should().BeTrue(); File.Exists(Path.Combine(cache, "mods", "old.xml")).Should().BeTrue(); - Directory.Exists(Path.Combine(cache, "roots")).Should().BeFalse(); - Directory.Exists(Path.Combine(cache, "leaves")).Should().BeFalse(); - Directory.EnumerateDirectories(cache).Should().NotContain(dir => Path.GetFileName(dir).Length == 64); File.ReadAllText(Path.Combine(cache, "origin.json")).Should().Contain("https://metadata.example/"); } @@ -831,7 +791,7 @@ public async Task RefreshRoot_SubPathBase_MapsSourcesRelativeToPublishingBase() request.RequestUri!.AbsoluteUri == rootUrl ? RootXml() : LeafXml("app-leaf"), "\"v1\"")); var root = await fixture.Client.RefreshRootAsync(rootUrl); - var leaf = await fixture.Client.GetLeafAsync(root, "1.0.0", MetadataResourceUri.Resolve(rootUrl, "apps/leaf.xml")); + var leaf = await fixture.Client.GetPackageAsync(root, Package(root, "1.0.0")); leaf.Status.Should().Be(MetadataFreshness.Fresh); File.Exists(Path.Combine(cache, "metadata.xml")).Should().BeTrue(); @@ -853,14 +813,13 @@ public async Task ColdCache_Leaf_ReusesPublishedBodyOn304() : Xml(LeafXml("app-leaf"), "\"v1\"")); }); using var http = new HttpClient(handler); - var leafUri = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml"); var published = Path.Combine(cache, "apps", "leaf.xml"); try { using (var first = Client(cache, http)) { var root = await first.RefreshRootAsync(RootUrl); - (await first.RefreshLeafAsync(root, "1.0.0", leafUri)).Status.Should().Be(MetadataFreshness.Fresh); + (await first.RefreshPackageAsync(root, Package(root, "1.0.0"))).Status.Should().Be(MetadataFreshness.Fresh); } File.Exists(published).Should().BeTrue(); @@ -870,10 +829,10 @@ public async Task ColdCache_Leaf_ReusesPublishedBodyOn304() var opened = await second.OpenSnapshotAsync(RootUrl); opened.Status.Should().Be(MetadataFreshness.Stale); - var reused = await second.RefreshLeafAsync(opened, "1.0.0", leafUri); + var reused = await second.RefreshPackageAsync(opened, Package(opened, "1.0.0")); reused.Status.Should().Be(MetadataFreshness.Fresh); - reused.Entry!.Id.Should().Be("app-leaf"); + reused.Entry!.Id.Should().Be("apps/leaf/app-leaf"); handler.Calls.Should().ContainSingle(); handler.Calls[0].Conditional.Should().BeTrue(); File.Exists(published).Should().BeTrue(); @@ -952,22 +911,17 @@ public async Task UnsafeSourcePaths_AreRejectedBeforeAnyWrite() var root = await fixture.Client.RefreshRootAsync(RootUrl); fixture.Calls.Clear(); - var escapedSlash = MetadataResourceUri.Resolve(RootUrl, "apps/escape%2F..%2F..%2Fevil.xml"); - var escapedBackslash = MetadataResourceUri.Resolve(RootUrl, @"apps\escape%5C..%5Cevil.xml"); - var reserved = MetadataResourceUri.Resolve(RootUrl, "apps/CON.xml"); - - await FluentActions.Awaiting(async () => await fixture.Client.GetLeafAsync(root, "1.0.0", escapedSlash)) - .Should().ThrowAsync(); - await FluentActions.Awaiting(async () => await fixture.Client.GetLeafAsync(root, "1.0.0", escapedBackslash)) - .Should().ThrowAsync(); - await FluentActions.Awaiting(async () => await fixture.Client.GetLeafAsync(root, "1.0.0", reserved)) - .Should().ThrowAsync(); + foreach (var source in new[] { "apps/escape%2F..%2F..%2Fevil.xml", "apps/escape%5C..%5Cevil.xml", "apps/CON.xml" }) + { + var canonicalize = () => MetadataDiskCache.CanonicalizeEntry(new Uri(new Uri(RootUrl), source).AbsoluteUri); + canonicalize.Should().Throw(); + } await FluentActions.Awaiting(async () => await fixture.Client.RefreshRootAsync("https://metadata.example/metadata.xml?rev=1")) .Should().ThrowAsync(); fixture.Calls.Should().BeEmpty(); Directory.EnumerateFiles(cache, "*.xml", SearchOption.AllDirectories) - .Should().ContainSingle().Which.Should().Be(Path.Combine(cache, "metadata.xml")); + .Should().OnlyContain(path => path == Path.Combine(cache, "metadata.xml") || path.Contains(".compositions")); File.Exists(Path.Combine(cache, "evil.xml")).Should().BeFalse(); var parent = Path.GetDirectoryName(cache); if (parent is not null) @@ -1034,20 +988,24 @@ public async Task SidecarSuffixSource_IsRejectedWithoutWrite() var root = await fixture.Client.RefreshRootAsync(RootUrl); fixture.Calls.Clear(); - var sidecar = MetadataResourceUri.Resolve(RootUrl, "apps/leaf.xml.etag"); - await FluentActions.Awaiting(async () => await fixture.Client.GetLeafAsync(root, "1.0.0", sidecar)) + await FluentActions.Awaiting(() => fixture.Client.GetPackageAsync(root, Package(root, "1.0.0"))) .Should().ThrowAsync(); fixture.Calls.Should().BeEmpty(); File.Exists(Path.Combine(cache, "apps", "leaf.xml.etag")).Should().BeFalse(); Directory.EnumerateFiles(cache, "*.xml", SearchOption.AllDirectories) - .Should().ContainSingle().Which.Should().Be(Path.Combine(cache, "metadata.xml")); + .Should().OnlyContain(path => path == Path.Combine(cache, "metadata.xml") || path.Contains(".compositions")); File.Exists(Path.Combine(cache, "metadata.xml.etag")).Should().BeTrue(); + using var reopened = Client(cache, fixture.Http); + var cached = await reopened.OpenSnapshotAsync(RootUrl); + cached.Status.Should().Be(MetadataFreshness.Stale); + cached.Document.Should().NotBeNull(); + fixture.Calls.Should().BeEmpty(); } private static string OtherRootXml() => """ - + 9.0 """; @@ -1062,7 +1020,7 @@ private static void SetStampUri(string bodyPath, string uri) private static string SharedSourceRoot() => """ - + id-a @@ -1078,7 +1036,7 @@ private static string SharedSourceRoot() => """ private static string IdRoot(string manifestId) => $$""" - + {{manifestId}} @@ -1120,7 +1078,7 @@ private static void ReplaceVerifiedBody(string cache, string fileName, byte[] bo private static string RootXml(string currentSource = "apps/leaf.xml", string currentId = "app-leaf") => $$""" - + 1.0.0 @@ -1174,7 +1132,7 @@ private static string StubXml() => """ """; - private static string IncludeXml() => """ + private static string CyclicLeafXml() => """ diff --git a/Ra3.BattleNet.Metadata.Tests/MetadataClientTransferTests.cs b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTransferTests.cs similarity index 95% rename from Ra3.BattleNet.Metadata.Tests/MetadataClientTransferTests.cs rename to Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTransferTests.cs index a4cd27c..c1ce9a3 100644 --- a/Ra3.BattleNet.Metadata.Tests/MetadataClientTransferTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataClientTransferTests.cs @@ -15,7 +15,9 @@ namespace Ra3.BattleNet.Metadata.Tests; public class MetadataClientTransferTests { private const string RootUrl = "https://metadata.example/metadata.xml"; - private const string LeafSource = "apps/leaf.xml"; + + private static PackageEntry Package(RootSnapshotResult snapshot, string version) => + snapshot.Document!.Applications().Single().Packages.Single(package => package.Version == version); [TestMethod] public async Task Success_ParsesStagedFile_AdoptsIntoCache_AndDropsStagingFile() @@ -62,7 +64,7 @@ public async Task InvalidPayload_IsNotPublished_AndStagingFileIsRemoved() var root = await client.RefreshRootAsync(RootUrl); root.Status.Should().Be(MetadataFreshness.Unavailable); - root.Error.Should().Contain("解析失败"); + root.Error.Should().NotBeNullOrWhiteSpace(); File.Exists(transfer.Requests[0].StagingPath).Should().BeFalse(); await AssertNothingPublishedAsync(cache, staging); } @@ -82,18 +84,17 @@ public async Task TruncatedLeaf_IsNotPublished_AndStagingFileIsRemoved() }); using var client = Client(cache, transfer, staging); var root = await client.RefreshRootAsync(RootUrl); - var leafUri = MetadataResourceUri.Resolve(RootUrl, LeafSource); var before = transfer.Requests.Count; - var leaf = await client.RefreshLeafAsync(root, "1.0.0", leafUri); + var leaf = await client.RefreshPackageAsync(root, Package(root, "1.0.0")); leaf.Status.Should().Be(MetadataFreshness.Unavailable); - leaf.Error.Should().Contain("解析失败"); + leaf.Error.Should().NotBeNullOrWhiteSpace(); transfer.Requests.Skip(before).Should().OnlyContain(request => !File.Exists(request.StagingPath)); Directory.EnumerateFiles(staging).Should().BeEmpty("半截叶子正文不能留在临时目录里"); // 叶子没有进缓存：只读缓存也拿不到它。 - var cached = await client.GetCachedLeafAsync(root, "1.0.0", leafUri); + var cached = await client.GetCachedPackageAsync(root, Package(root, "1.0.0")); cached.Status.Should().Be(MetadataFreshness.Unavailable); } @@ -277,9 +278,8 @@ public async Task LeafRefreshAndDistinctColdImages_ShareOneTransferGate_AtMostTh } }; - var leafUri = MetadataResourceUri.Resolve(RootUrl, LeafSource); - var leafTask = client.RefreshLeafAsync(root, "1.0.0", leafUri); - var imageTasks = imageIds.Select(id => client.GetImageAsync(root, id)).ToArray(); + var leafTask = client.RefreshPackageAsync(root, Package(root, "1.0.0")); + var imageTasks = imageIds.Select(id => client.GetImageAsync(root, "metadata/" + id)).ToArray(); var leaf = await leafTask; var images = await Task.WhenAll(imageTasks); @@ -326,9 +326,9 @@ public async Task QueuedTransfer_DoesNotSpendItsRequestTimeoutWhileWaitingForASl return new MetadataTransferResult(200, null, null, "image/png"); }; - var first = client.GetImageAsync(root, "iconA"); + var first = client.GetImageAsync(root, "metadata/iconA"); await slowEntered.Task.WaitAsync(TimeSpan.FromSeconds(5)); - var queued = client.GetImageAsync(root, "iconB"); + var queued = client.GetImageAsync(root, "metadata/iconB"); var results = await Task.WhenAll(first, queued); results.Should().OnlyContain(image => image.Ok && image.Bytes!.SequenceEqual(PngBytes)); @@ -365,9 +365,9 @@ public async Task DisposeWhileSlotIsHeld_ReleasesQueuedWaiterWithoutSlotErrors() return new MetadataTransferResult(200, null, null, "image/png"); }; - var holder = client.GetImageAsync(root, "iconA"); + var holder = client.GetImageAsync(root, "metadata/iconA"); await holderEntered.Task.WaitAsync(TimeSpan.FromSeconds(5)); - var queued = client.GetImageAsync(root, "iconB"); + var queued = client.GetImageAsync(root, "metadata/iconB"); await Task.Delay(100); client.Dispose(); @@ -395,7 +395,7 @@ private static string RootXml(params string[] imageIds) var images = string.Join("\n", imageIds.Select(id => $" ")); return $$""" - + {{images}} 1.0.0 diff --git a/Ra3.BattleNet.Metadata.Tests/Cache/MetadataCompositionCacheTests.cs b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataCompositionCacheTests.cs new file mode 100644 index 0000000..5399f9b --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Cache/MetadataCompositionCacheTests.cs @@ -0,0 +1,417 @@ +using System.Net; +using System.Net.Sockets; +using System.Text; +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; +using Ra3.BattleNet.Metadata.Cache; + +namespace Ra3.BattleNet.Metadata.Tests; + +[TestClass] +public sealed class MetadataCompositionCacheTests +{ + [TestMethod] + public async Task Root304_RevalidatesChild_AndPersistsLastCompleteGraph() + { + using var server = new Server(); + server.Xml("metadata.xml", "", "root"); + server.Xml("child.xml", Child("one"), "one"); + var cache = NewDirectory(); + using var client = Client(cache); + var first = await client.RefreshRootAsync(server.Root); + first.Status.Should().Be(MetadataFreshness.Fresh); + server.Xml("child.xml", Child("two"), "two"); + var second = await client.RefreshRootAsync(server.Root); + second.Status.Should().Be(MetadataFreshness.Fresh); + second.Document!.Mods().Single().Id.Should().Be("two"); + server.Requests.Should().Contain(item => item.Path == "metadata.xml" && item.Conditional == "\"root\""); + server.Requests.Should().Contain(item => item.Path == "child.xml" && item.Conditional == "\"one\""); + File.ReadAllText(Path.Combine(cache, "metadata.xml")).Should().Contain("Include"); + var count = server.Requests.Count; + using (var reopened = Client(cache)) + { + var offline = await reopened.OpenSnapshotAsync(server.Root); + offline.Status.Should().Be(MetadataFreshness.Stale); + offline.Document!.Mods().Single().Id.Should().Be("two"); + } + server.Requests.Count.Should().Be(count); + server.Fail("child.xml"); + var failed = await client.RefreshRootAsync(server.Root); + failed.Status.Should().Be(MetadataFreshness.Stale); + failed.Document.Should().BeSameAs(second.Document); + using var cold = Client(NewDirectory()); + (await cold.RefreshRootAsync(server.Root)).Status.Should().Be(MetadataFreshness.Unavailable); + using var last = Client(cache); + (await last.OpenSnapshotAsync(server.Root)).Document!.Mods().Single().Id.Should().Be("two"); + } + + [TestMethod] + public async Task SharedLeafAcrossSourceRoots_PersistsEachNormalizedIdentity() + { + using var server = new Server(); + server.Xml("metadata.xml", """ + + m + + + """, "root"); + server.Xml("nested/metadata.xml", """ + + m + + + """, "nested-root"); + server.Xml("nested/leaf.xml", "", "leaf"); + var nestedRoot = new Uri(new Uri(server.Root), "nested/metadata.xml").AbsoluteUri; + var cache = NewDirectory(); + using (var client = Client(cache)) + { + foreach (var entry in new[] { server.Root, nestedRoot }) + { + var root = await client.RefreshRootAsync(entry); + var package = root.Document!.Mods().Single().Packages.Single(); + (await client.GetPackageAsync(root, package)).Status.Should().Be(MetadataFreshness.Fresh); + } + } + var requests = server.Requests.Count; + using var reopened = Client(cache); + foreach (var entry in new[] { server.Root, nestedRoot }) + { + var root = await reopened.OpenSnapshotAsync(entry); + var package = root.Document!.Mods().Single().Packages.Single(); + var leaf = await reopened.GetCachedPackageAsync(root, package); + leaf.Status.Should().Be(MetadataFreshness.Stale); + leaf.Entry!.Id.Should().Be(entry == server.Root ? "nested/leaf/m" : "leaf/m"); + } + server.Requests.Should().HaveCount(requests); + } + + + [TestMethod] + public async Task EncodedHttpPaths_MatchDirectReaderIdentityAndResourceProjection() + { + using var server = new Server(); + server.Xml("metadata.xml", "", "root"); + server.Xml("nested/literal%2520.xml", """ + + + + """, "child"); + var direct = MetadataReader.Read(server.Root).Root; + using var client = Client(NewDirectory()); + var cached = await client.RefreshRootAsync(server.Root); + cached.Status.Should().Be(MetadataFreshness.Fresh); + var image = cached.Document!.GetAllElements("Image").Single(); + var directImage = direct.GetAllElements("Image").Single(); + image.Get("ID").Should().Be("nested/literal%2520/icon"); + image.Get("Source").Should().Be("nested/image%2520.png"); + image.Get("ID").Should().Be(directImage.Get("ID")); + image.Get("Source").Should().Be(directImage.Get("Source")); + } + + + [TestMethod] + public async Task ColdOffline_NeverRequestsNetwork_AndCorruptCompositionIsRejected() + { + using var server = new Server(); + var cache = NewDirectory(); + using var client = Client(cache); + (await client.OpenSnapshotAsync(server.Root)).Status.Should().Be(MetadataFreshness.Unavailable); + server.Requests.Should().BeEmpty(); + server.Xml("metadata.xml", "", "root"); + (await client.RefreshRootAsync(server.Root)).Status.Should().Be(MetadataFreshness.Fresh); + var record = Directory.EnumerateFiles(Path.Combine(cache, ".compositions"), "*.xml").Single(); + File.WriteAllText(record, " + 1 + inline + external + + inline.bin + + + + """, "root"); + server.Xml("nested/leaf.xml", "external.bin", "leaf"); + server.Bytes("nested/icon.png", [1, 2, 3], "image/png"); + using var client = Client(NewDirectory()); + var snapshot = await client.RefreshRootAsync(server.Root); + var packages = snapshot.Document!.Mods().Single().Packages.ToList(); + var inline = await client.GetPackageAsync(snapshot, packages[0]); + inline.Status.Should().Be(MetadataFreshness.Fresh); + inline.SourceUri.Should().BeNull(); + inline.Entry!.Files.Single().FileName.Should().Be("inline.bin"); + server.Requests.Should().HaveCount(1); + (await client.GetCachedPackageAsync(snapshot, packages[1])).Status.Should().Be(MetadataFreshness.Unavailable); + var external = await client.GetPackageAsync(snapshot, packages[1]); + external.Entry!.Files.Single().FileName.Should().Be("external.bin"); + external.SourceUri!.AbsoluteUri.Should().Be(new Uri(new Uri(server.Root), "nested/leaf.xml").AbsoluteUri); + var stale = await client.OpenSnapshotAsync(server.Root); + (await client.RefreshPackageAsync(stale, packages[0])).Status.Should().Be(MetadataFreshness.Stale); + var forged = packages[0] with { Raw = new Metadata() }; + (await client.GetPackageAsync(snapshot, forged)).Status.Should().Be(MetadataFreshness.Unavailable); + var summary = await client.PreloadLeavesAsync(snapshot); + summary.Total.Should().Be(2); + summary.Failed.Should().Be(0); + (await client.GetImageAsync(snapshot, "metadata/icon")).Bytes.Should().Equal(1, 2, 3); + server.Requests.Should().Contain(item => item.Path == "nested/icon.png"); + } + + [TestMethod] + public async Task IncludedManifestBody_RemainsInline_WhileExternalStubUsesSharedScope() + { + using var server = new Server(); + server.Xml("metadata.xml", """ + + + + """, "root"); + server.Xml("nested/packages.xml", """ + + + inline + external + + inline.bin + + + """, "packages"); + server.Xml("nested/leaf.xml", "external.bin", "leaf"); + using var client = Client(NewDirectory()); + var snapshot = await client.RefreshRootAsync(server.Root); + var packages = snapshot.Document!.Mods().Single().Packages.ToList(); + var inline = await client.GetPackageAsync(snapshot, packages[0]); + inline.Entry!.Id.Should().Be("nested/packages/inline"); + inline.SourceUri.Should().BeNull(); + server.Requests.Should().HaveCount(2); + var external = await client.GetPackageAsync(snapshot, packages[1]); + external.Entry!.Id.Should().Be("nested/leaf/external"); + external.Entry.Files.Single().FileName.Should().Be("external.bin"); + server.Requests.Should().HaveCount(3); + } + + [TestMethod] + public async Task RawBodyWithoutNormalizedSnapshot_IsNotAnOfflineRoot() + { + using var server = new Server(); + server.Xml("metadata.xml", "", "root"); + var cache = NewDirectory(); + using (var client = Client(cache)) + (await client.RefreshRootAsync(server.Root)).Status.Should().Be(MetadataFreshness.Fresh); + foreach (var record in Directory.EnumerateFiles(Path.Combine(cache, ".compositions"), "*.xml")) + File.Delete(record); + var requests = server.Requests.Count; + using var reopened = Client(cache); + (await reopened.OpenSnapshotAsync(server.Root)).Status.Should().Be(MetadataFreshness.Unavailable); + server.Requests.Should().HaveCount(requests); + } + + + [TestMethod] + public async Task LocalRoot_RemoteInclude_IsRejected() + { + using var server = new Server(); + server.Xml("child.xml", Child("remote"), "child"); + var source = NewDirectory(); + var path = Path.Combine(source, "metadata.xml"); + File.WriteAllText(path, $""); + using var client = Client(NewDirectory()); + (await client.RefreshRootAsync(path)).Status.Should().Be(MetadataFreshness.Unavailable); + server.Requests.Should().BeEmpty(); + } + + [TestMethod] + public void SourceManifest_RequiresExactIdentity() + { + var directory = NewDirectory(); + var path = Path.Combine(directory, "metadata.xml"); + File.WriteAllText(path, """ + + wanted/item + + + """); + File.WriteAllText(Path.Combine(directory, "leaf.xml"), """ + + exact.bin + other.bin + + """); + var declaration = MetadataReader.Read(path); + var package = declaration.Root.Mods().Single().Packages.Single(); + MetadataSourceManifest.Resolve(declaration, package.Raw)!.ToManifestEntry().Files.Single().FileName.Should().Be("exact.bin"); + File.WriteAllText(Path.Combine(directory, "leaf.xml"), """ + + + + + """); + var resolve = () => MetadataSourceManifest.Resolve(declaration, package.Raw); + resolve.Should().Throw(); + } + + [TestMethod] + public void SourceManifest_EmptyInlineDeclaration_ReturnsRegisteredNode() + { + var path = Path.Combine(NewDirectory(), "metadata.xml"); + File.WriteAllText(path, """ + + empty/empty + + + """); + File.WriteAllText(Path.Combine(Path.GetDirectoryName(path)!, "empty.xml"), ""); + var declaration = MetadataReader.Read(path); + var package = declaration.Root.Mods().Single().Packages.Single(); + var registration = declaration.Root.ManifestRegistration(package.ManifestId!); + MetadataSourceManifest.Resolve(declaration, package.Raw).Should().BeSameAs(registration); + } + + + [TestMethod] + public void SourceManifest_ComposedExternalLeaf_ResolvesIncludedBody() + { + var directory = NewDirectory(); + var path = Path.Combine(directory, "metadata.xml"); + File.WriteAllText(path, """ + + body/item + + + """); + File.WriteAllText(Path.Combine(directory, "leaf.xml"), ""); + File.WriteAllText(Path.Combine(directory, "body.xml"), "composed.bin"); + var declaration = MetadataReader.Read(path); + var package = declaration.Root.Mods().Single().Packages.Single(); + var manifest = MetadataSourceManifest.Resolve(declaration, package.Raw)!; + manifest.ToManifestEntry().Files.Single().FileName.Should().Be("composed.bin"); + } + + [TestMethod] + public async Task IncludeRedirect_IsRejectedWithoutRequestingRedirectTarget() + { + using var server = new Server(); + server.Xml("metadata.xml", "", "root"); + server.Redirect("child.xml", new Uri(new Uri(server.Root), "outside.xml").AbsoluteUri); + server.Xml("outside.xml", Child("outside"), "outside"); + using var client = Client(NewDirectory()); + (await client.RefreshRootAsync(server.Root)).Status.Should().Be(MetadataFreshness.Unavailable); + server.Requests.Should().NotContain(request => request.Path == "outside.xml"); + } + + + [TestMethod] + public async Task CachedLocalPackage_PreservesPathIds_AndExpandsLocalIncludesWithoutNetwork() + { + using var server = new Server(); + var directory = NewDirectory(); + var path = Path.Combine(directory, "metadata.xml"); + var leaf = Path.Combine(directory, "leaf.xml"); + File.WriteAllText(path, """ + + scope/item + + + """); + var body = "local.bin"; + File.WriteAllText(leaf, body); + using var client = new MetadataClient("", TimeSpan.FromSeconds(5), enableDiskCache: false); + var snapshot = await client.RefreshRootAsync(path); + var package = snapshot.Document!.Mods().Single().Packages.Single(); + var flat = await client.GetCachedPackageAsync(snapshot, package); + flat.Status.Should().Be(MetadataFreshness.Stale); + flat.Entry!.Id.Should().Be("scope/item"); + File.WriteAllText(leaf, ""); + File.WriteAllText(Path.Combine(directory, "body.xml"), body); + (await client.GetCachedPackageAsync(snapshot, package)).Entry!.Files.Single().FileName.Should().Be("local.bin"); + File.WriteAllText(leaf, $""); + (await client.GetCachedPackageAsync(snapshot, package)).Status.Should().Be(MetadataFreshness.Unavailable); + server.Requests.Should().BeEmpty(); + } + + private static string Child(string id) => $""; + private static string NewDirectory() + { + var path = Path.Combine(Path.GetTempPath(), "metadata-composition-" + Guid.NewGuid().ToString("N")); + Directory.CreateDirectory(path); + return path; + } + private static MetadataClient Client(string cache) => new(cache, TimeSpan.FromSeconds(5)); + + private sealed class Server : IDisposable + { + private readonly HttpListener _listener = new(); + private readonly Dictionary _responses = new(StringComparer.Ordinal); + private readonly Dictionary _redirects = new(StringComparer.Ordinal); + private readonly Task _loop; + public string Root { get; } + public List<(string Path, string? Conditional)> Requests { get; } = []; + + public Server() + { + var socket = new TcpListener(IPAddress.Loopback, 0); + socket.Start(); + var port = ((IPEndPoint)socket.LocalEndpoint).Port; + socket.Stop(); + var prefix = $"http://127.0.0.1:{port}/"; + Root = prefix + "metadata.xml"; + _listener.Prefixes.Add(prefix); + _listener.Start(); + _loop = RunAsync(); + } + public void Xml(string path, string body, string etag) => + _responses[path] = (Encoding.UTF8.GetBytes(body), "application/xml", "\"" + etag + "\"", 200); + public void Bytes(string path, byte[] body, string type) => _responses[path] = (body, type, null, 200); + public void Fail(string path) => _responses[path] = ([], "text/plain", null, 503); + public void Redirect(string path, string destination) => _redirects[path] = destination; + private async Task RunAsync() + { + try + { + while (_listener.IsListening) + { + var context = await _listener.GetContextAsync(); + var path = context.Request.Url!.AbsolutePath.TrimStart('/'); + var conditional = context.Request.Headers["If-None-Match"]; + Requests.Add((path, conditional)); + if (_redirects.TryGetValue(path, out var destination)) + { + context.Response.StatusCode = 302; + context.Response.RedirectLocation = destination; + context.Response.Close(); + continue; + } + var response = _responses.TryGetValue(path, out var registered) + ? registered : (Body: Array.Empty(), Type: "text/plain", ETag: (string?)null, Status: 404); + context.Response.StatusCode = response.Status == 200 && response.ETag is not null && conditional == response.ETag ? 304 : response.Status; + if (response.ETag is not null) + context.Response.Headers["ETag"] = response.ETag; + context.Response.ContentType = response.Type; + if (context.Response.StatusCode == 200) + { + context.Response.ContentLength64 = response.Body.Length; + await context.Response.OutputStream.WriteAsync(response.Body); + } + context.Response.Close(); + } + } + catch (Exception ex) when (ex is HttpListenerException or ObjectDisposedException) { } + } + public void Dispose() + { + _listener.Close(); + _loop.GetAwaiter().GetResult(); + } + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/Exporting/MetadataExportTests.cs b/Ra3.BattleNet.Metadata.Tests/Exporting/MetadataExportTests.cs new file mode 100644 index 0000000..84064b4 --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Exporting/MetadataExportTests.cs @@ -0,0 +1,291 @@ +using System.Xml.Linq; +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +[TestClass] +public class MetadataExportTests +{ + [TestMethod] + public void EveryLayout_RoundTripsAssetsAndCanonicalIdentity() + { + var sandbox = CreateFixture(); + try + { + var input = MetadataReader.Read(Path.Combine(sandbox, "source", "metadata.xml")); + var original = input.Document.ToString(); + var captured = input.Documents.Select(document => document.NormalizedDocument.ToString()).ToArray(); + foreach (var layout in Enum.GetValues()) + foreach (var inlineManifests in new[] { false, true }) + { + var output = Path.Combine(sandbox, $"{layout}-{inlineManifests}"); + var paths = MetadataExporter.Export(input, output, new MetadataExportOptions + { + Layout = layout, + Inline = path => path == "mods/demo.xml", + InlineManifests = inlineManifests + }); + paths.Should().Contain(Path.Combine(output, "metadata.xml")); + foreach (var path in paths) + { + var written = XDocument.Load(path); + written.Root!.Attribute("IdFormat").Should().BeNull(); + written.Root.Attribute("SourceBase").Should().BeNull(); + written.Descendants().Where(node => node.Name.LocalName is "Image" or "Markdown" or "Manifest") + .Attributes("ID").Should().OnlyContain(attribute => attribute.Value.Contains('/')); + } + var result = MetadataReader.Read(Path.Combine(output, "metadata.xml"), new MetadataSourceOptions { InlineManifests = true }); + result.Root.Images().Select(image => image.Id).Should().BeEquivalentTo(input.Root.Images().Select(image => image.Id)); + result.Root.Mods().Single().Raw.Find("Icon")!.Value.Should().Be("mods/demo/icon"); + result.Root.Mods().Single().Id.Should().Be("Demo"); + result.Root.ManifestRegistration("leaf/m")!.Find("File:FileName")!.Value.Should().Be("demo.big"); + var image = result.Root.Images().Single(entry => entry.Source is not null); + File.ReadAllText(MetadataResourceUri.Resolve(Path.Combine(output, "metadata.xml"), image.Source!).LocalPath) + .Should().Be("image-bytes"); + var markdown = result.Root.Markdowns().Single(); + File.ReadAllText(MetadataResourceUri.Resolve(Path.Combine(output, "metadata.xml"), markdown.Source!).LocalPath) + .Should().Be("# news"); + var remote = result.Root.Images().Single(entry => entry.Url is not null); + remote.Url.Should().Be("https://example.test/icon.png"); + remote.Source.Should().BeNull(); + if (layout == MetadataExportLayout.Flat) + XDocument.Load(Path.Combine(output, "metadata.xml")).Descendants("Include").Should().BeEmpty(); + if (layout == MetadataExportLayout.Tree) + File.Exists(Path.Combine(output, "mods", "demo.xml")).Should().BeTrue(); + input.Document.ToString().Should().Be(original); + input.Documents.Select(document => document.NormalizedDocument.ToString()).Should().Equal(captured); + } + File.ReadAllText(Path.Combine(sandbox, "source", "mods", "icon.png")).Should().Be("image-bytes"); + } + finally { Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void Mixed_SelectsTargetEdgesAndWritesDocumentRelativeSources() + { + var sandbox = CreateFixture(); + try + { + var input = MetadataReader.Read(Path.Combine(sandbox, "source", "metadata.xml")); + var output = Path.Combine(sandbox, "mixed"); + MetadataExporter.Export(input, output, new MetadataExportOptions + { + Layout = MetadataExportLayout.Mixed, + Inline = path => path == "mods/demo.xml", + InlineManifests = true + }); + var root = XDocument.Load(Path.Combine(output, "metadata.xml")); + root.Root!.Elements("Mod").Should().ContainSingle(); + root.Descendants("Include").Select(node => (string?)node.Attribute("Source")).Should().Equal("leaf.xml"); + File.Exists(Path.Combine(output, "mods", "demo.xml")).Should().BeFalse(); + root.Descendants("Image").Single(node => node.Attribute("Source") is not null) + .Attribute("Source")!.Value.Should().Be("mods/icon.png"); + } + finally { Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void ExportAgain_KeepsCanonicalIdsWithoutPhysicalPathPrefixes() + { + var sandbox = CreateFixture(); + try + { + var input = MetadataReader.Read(Path.Combine(sandbox, "source", "metadata.xml")); + var first = Path.Combine(sandbox, "first"); + var second = Path.Combine(sandbox, "second"); + MetadataExporter.Export(input, first, new MetadataExportOptions { Layout = MetadataExportLayout.Tree }); + MetadataExporter.Export(MetadataReader.Read(Path.Combine(first, "metadata.xml")), second, + new MetadataExportOptions { Layout = MetadataExportLayout.Flat, InlineManifests = true }); + var result = MetadataReader.Read(Path.Combine(second, "metadata.xml"), new MetadataSourceOptions { InlineManifests = true }); + result.Root.Images().Select(image => image.Id).Should().BeEquivalentTo(input.Root.Images().Select(image => image.Id)); + result.Root.ManifestRegistration("leaf/m")!.Find("File:FileName")!.Value.Should().Be("demo.big"); + } + finally { Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void Failure_ClearsOnlyOutputAndLeavesInputUntouched() + { + var sandbox = CreateFixture(); + try + { + var source = Path.Combine(sandbox, "source"); + var input = MetadataReader.Read(Path.Combine(source, "metadata.xml")); + var output = Path.Combine(sandbox, "output"); + Directory.CreateDirectory(output); + File.WriteAllText(Path.Combine(output, "old.txt"), "old"); + File.Delete(Path.Combine(source, "mods", "icon.png")); + var export = () => MetadataExporter.Export(input, output); + export.Should().Throw(); + Directory.Exists(output).Should().BeFalse(); + File.Exists(Path.Combine(source, "metadata.xml")).Should().BeTrue(); + var overlap = () => MetadataExporter.Export(input, source); + overlap.Should().Throw(); + File.Exists(Path.Combine(source, "metadata.xml")).Should().BeTrue(); + } + finally { Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void Planning_RejectsTraversalAndOutputCollisionsBeforeWriting() + { + var sandbox = CreateFixture(); + try + { + var source = Path.Combine(sandbox, "source"); + var input = MetadataReader.Read(Path.Combine(source, "metadata.xml")); + var output = Path.Combine(sandbox, "output"); + var image = input.Documents.Single(document => document.RelativePath == "mods/demo.xml") + .NormalizedDocument.Descendants("Image").Single(node => node.Attribute("Source") is not null); + image.SetAttributeValue("Source", Path.Combine(sandbox, "outside.png")); + var traversal = () => MetadataExporter.Export(input, output); + traversal.Should().Throw(); + Directory.Exists(output).Should().BeFalse(); + image.SetAttributeValue("Source", Path.Combine(source, "metadata.xml")); + var collision = () => MetadataExporter.Export(input, output); + collision.Should().Throw(); + Directory.Exists(output).Should().BeFalse(); + } + finally { Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void SharedExternalManifestSource_PreservesEveryRegistration() + { + var sandbox = Path.Combine(Path.GetTempPath(), "metadata-export-" + Guid.NewGuid().ToString("N")); + try + { + var source = Path.Combine(sandbox, "source"); + Write(source, "metadata.xml", ""); + Write(source, "both.xml", "one.bigtwo.big"); + var output = Path.Combine(sandbox, "output"); + MetadataExporter.Export(MetadataReader.Read(Path.Combine(source, "metadata.xml")), output); + var leaf = XDocument.Load(Path.Combine(output, "both.xml")); + leaf.Root!.Elements("Manifest").Select(element => (string?)element.Attribute("ID")).Should().Equal("both/one", "both/two"); + var result = MetadataReader.Read(Path.Combine(output, "metadata.xml"), new MetadataSourceOptions { InlineManifests = true }); + result.Root.ManifestRegistration("both/one")!.Find("File:FileName")!.Value.Should().Be("one.big"); + result.Root.ManifestRegistration("both/two")!.Find("File:FileName")!.Value.Should().Be("two.big"); + } + finally { if (Directory.Exists(sandbox)) Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void Tree_ExternalManifestInsideBusinessDocument_DoesNotReferenceItself() + { + var sandbox = Path.Combine(Path.GetTempPath(), "metadata-export-" + Guid.NewGuid().ToString("N")); + try + { + var source = Path.Combine(sandbox, "source"); + Write(source, "metadata.xml", ""); + Write(source, "parts/mod.xml", """ + + demo.big + payload + + """); + var input = MetadataReader.Read(Path.Combine(source, "metadata.xml")); + foreach (var layout in Enum.GetValues()) + { + var output = Path.Combine(sandbox, layout.ToString()); + MetadataExporter.Export(input, output, new MetadataExportOptions + { + Layout = layout, + Inline = _ => false + }); + var result = MetadataReader.Read(Path.Combine(output, "metadata.xml"), + new MetadataSourceOptions { InlineManifests = true }); + result.Root.ManifestRegistration("parts/mod/payload")!.Find("File:FileName")!.Value.Should().Be("demo.big"); + if (layout != MetadataExportLayout.Flat) + { + var business = XDocument.Load(Path.Combine(output, "parts", "mod.xml")); + business.Root!.Elements("Mod").Should().ContainSingle(); + business.Root.Elements("Manifest").Single().Attribute("Source")!.Value.Should().NotBe("mod.xml"); + } + } + } + finally { if (Directory.Exists(sandbox)) Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void EmptyFullManifest_AllLayoutsPreserveHashAlgorithmAndRoundTrip() + { + var sandbox = Path.Combine(Path.GetTempPath(), "metadata-export-" + Guid.NewGuid().ToString("N")); + try + { + var source = Path.Combine(sandbox, "source"); + Write(source, "metadata.xml", ""); + Write(source, "leaf.xml", ""); + var input = MetadataReader.Read(Path.Combine(source, "metadata.xml")); + foreach (var layout in Enum.GetValues()) + foreach (var inline in new[] { false, true }) + { + var output = Path.Combine(sandbox, $"{layout}-{inline}"); + MetadataExporter.Export(input, output, new MetadataExportOptions + { + Layout = layout, + Inline = _ => true, + InlineManifests = inline + }); + var result = MetadataReader.Read(Path.Combine(output, "metadata.xml"), + new MetadataSourceOptions { InlineManifests = true }); + var manifest = result.Root.ManifestRegistration("leaf/empty")!; + manifest.Get("HashAlgorithm").Should().Be("None"); + manifest.Get("Source").Should().BeNull(); + manifest.Children.Should().BeEmpty(); + if (!inline) + { + var leaf = XDocument.Load(Path.Combine(output, "leaf.xml")); + leaf.Root!.Element("Manifest")!.Attribute("HashAlgorithm")!.Value.Should().Be("None"); + } + } + } + finally { if (Directory.Exists(sandbox)) Directory.Delete(sandbox, recursive: true); } + } + + [TestMethod] + public void ExternalManifest_WithSameLocalIdInDifferentScope_FailsWithoutRenaming() + { + var sandbox = Path.Combine(Path.GetTempPath(), "metadata-export-" + Guid.NewGuid().ToString("N")); + try + { + var source = Path.Combine(sandbox, "source"); + Write(source, "metadata.xml", ""); + Write(source, "leaf.xml", ""); + var input = MetadataReader.Read(Path.Combine(source, "metadata.xml")); + var output = Path.Combine(sandbox, "output"); + var export = () => MetadataExporter.Export(input, output); + export.Should().Throw().WithMessage("*leaf/wanted*"); + Directory.Exists(output).Should().BeFalse(); + } + finally { if (Directory.Exists(sandbox)) Directory.Delete(sandbox, recursive: true); } + } + + private static string CreateFixture() + { + var sandbox = Path.Combine(Path.GetTempPath(), "metadata-export-" + Guid.NewGuid().ToString("N")); + var source = Path.Combine(sandbox, "source"); + Write(source, "metadata.xml", ""); + Write(source, "mods/demo.xml", """ + + + + iconDemoDemo + news + leaf/m + + + """); + Write(source, "leaf.xml", "demo.big/"); + Write(source, "mods/icon.png", "image-bytes"); + Write(source, "mods/news.md", "# news"); + return sandbox; + } + + private static void Write(string root, string relative, string text) + { + var path = Path.Combine(root, relative); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + File.WriteAllText(path, text); + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/SourceFormatApiTests.cs b/Ra3.BattleNet.Metadata.Tests/Exporting/SourceFormatApiTests.cs similarity index 81% rename from Ra3.BattleNet.Metadata.Tests/SourceFormatApiTests.cs rename to Ra3.BattleNet.Metadata.Tests/Exporting/SourceFormatApiTests.cs index dc84873..d4fa33a 100644 --- a/Ra3.BattleNet.Metadata.Tests/SourceFormatApiTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Exporting/SourceFormatApiTests.cs @@ -13,31 +13,31 @@ public class SourceFormatApiTests private const string FlattenedRoot = """ - - - - + + + + 3.259 - corona:icon + corona/icon 日冕 Corona - corona:icon - corona:newscorona:news + corona/icon + corona/newscorona/news - corona:link + corona/link - corona:manifest-3258 - corona:manifest-3259 + corona/manifest-3258 + corona/manifest-3259 @@ -153,14 +153,24 @@ public void Export_DoesNotMutateTheLoadedTree() MetadataSourceExporter.ExportWhole(root, "Corona", "https://metadata.example.com/metadata.xml"); - root.Mods().Single(m => m.Id == "Corona").Raw.Find("Icon")!.Value.Should().Be("corona:icon"); + root.Mods().Single(m => m.Id == "Corona").Raw.Find("Icon")!.Value.Should().Be("corona/icon"); root.Mods().Single(m => m.Id == "Corona").Raw .Find("Packages:Package:Manifest")!.Value - .Should().Be("corona:manifest-3258"); + .Should().Be("corona/manifest-3258"); } [TestMethod] - public void Resolve_StubPointingAtLeaf_LoadsLeafAndAlignsLocalId() + public void ExportManifestElement_WritesUnescapedLocalAuthorIdWithoutMutatingInput() + { + var manifest = XElement.Parse(""); + var exported = MetadataSourceExporter.ExportManifestElement(manifest); + exported.Attribute("ID")!.Value.Should().Be("manifest one"); + exported.Attribute("HashAlgorithm")!.Value.Should().Be("None"); + manifest.Attribute("ID")!.Value.Should().Be("mods/demo/manifest%20one"); + } + + [TestMethod] + public void Resolve_StubPointingAtLeaf_PreservesExactIdentity() { var root = TempDir(); try @@ -186,12 +196,13 @@ public void Resolve_StubPointingAtLeaf_LoadsLeafAndAlignsLocalId() """); var declarationPath = Path.Combine(modDir, "mod.xml"); - var loaded = MetadataBuilder.LoadSource(declarationPath); - var package = loaded.Mods().Single().Packages.Single(); + var loaded = MetadataReader.Read(declarationPath); + var package = loaded.Root.Mods().Single().Packages.Single(); - var manifest = MetadataSourceManifest.Resolve(loaded, declarationPath, package.Raw); + var manifest = MetadataSourceManifest.Resolve(loaded, package.Raw); manifest.Should().NotBeNull(); + manifest!.Get("ID").Should().Be(package.ManifestId).And.Be("manifests/1.0/manifest-1"); manifest!.Find("File:FileName")!.Value.Should().Be("demo.lyi"); manifest.ToManifestEntry().Files.Single().Hash.Should().Be("3507CD54"); } @@ -223,10 +234,10 @@ public void Resolve_InlineManifest_ReturnsRegistrationWithoutReadingLeaf() """); var declarationPath = Path.Combine(modDir, "mod.xml"); - var loaded = MetadataBuilder.LoadSource(declarationPath); - var package = loaded.Mods().Single().Packages.Single(); + var loaded = MetadataReader.Read(declarationPath); + var package = loaded.Root.Mods().Single().Packages.Single(); - var manifest = MetadataSourceManifest.Resolve(loaded, declarationPath, package.Raw); + var manifest = MetadataSourceManifest.Resolve(loaded, package.Raw); manifest!.Find("File:FileName")!.Value.Should().Be("inline.lyi"); } @@ -237,28 +248,11 @@ public void Resolve_InlineManifest_ReturnsRegistrationWithoutReadingLeaf() } [TestMethod] - public void Resolve_MissingSourceOrMismatchedLeafId_Throws() + public void Resolve_MismatchedCanonicalIdentity_Throws() { var root = TempDir(); try { - var modDir = Path.Combine(root, "missing"); - WriteFile(modDir, "mod.xml", """ - - - 1.0 - 演示 - Demo - manifest-1 - - - -"""); - var missingPath = Path.Combine(modDir, "mod.xml"); - var missing = MetadataBuilder.LoadSource(missingPath); - var missingPackage = missing.Mods().Single().Packages.Single(); - var missingAct = () => MetadataSourceManifest.Resolve(missing, missingPath, missingPackage.Raw); - missingAct.Should().Throw().WithMessage("*Source*"); var mismatchDir = Path.Combine(root, "mismatch"); WriteFile(mismatchDir, "mod.xml", """ @@ -273,12 +267,12 @@ public void Resolve_MissingSourceOrMismatchedLeafId_Throws() """); WriteFile(mismatchDir, "manifests/1.0.xml", - ""); + ""); var mismatchPath = Path.Combine(mismatchDir, "mod.xml"); - var mismatch = MetadataBuilder.LoadSource(mismatchPath); - var mismatchPackage = mismatch.Mods().Single().Packages.Single(); - var mismatchAct = () => MetadataSourceManifest.Resolve(mismatch, mismatchPath, mismatchPackage.Raw); - mismatchAct.Should().Throw().WithMessage("*不一致*"); + var mismatch = MetadataReader.Read(mismatchPath); + var mismatchPackage = mismatch.Root.Mods().Single().Packages.Single(); + var mismatchAct = () => MetadataSourceManifest.Resolve(mismatch, mismatchPackage.Raw); + mismatchAct.Should().Throw(); } finally { diff --git a/Ra3.BattleNet.Metadata.Tests/IdScopeTests.cs b/Ra3.BattleNet.Metadata.Tests/IdScopeTests.cs deleted file mode 100644 index 1258c13..0000000 --- a/Ra3.BattleNet.Metadata.Tests/IdScopeTests.cs +++ /dev/null @@ -1,217 +0,0 @@ -using FluentAssertions; -using Microsoft.VisualStudio.TestTools.UnitTesting; - -namespace Ra3.BattleNet.Metadata.Tests; - -[TestClass] -public class IdScopeTests -{ - private static string RepoMetadataDir => - Path.GetFullPath(Path.Combine(AppContext.BaseDirectory, "..", "..", "..", "..", "Metadata")); - - private static void SeedSchemas(string dir) - { - Directory.CreateDirectory(dir); - foreach (var name in new[] { SchemaValidator.SourceSchemaFileName, SchemaValidator.PublishSchemaFileName }) - File.Copy(Path.Combine(RepoMetadataDir, name), Path.Combine(dir, name), overwrite: true); - } - - [TestMethod] - public void QualifyId_UsesPathPrefixColonLocal() - { - MetadataFlattener.QualifyId("mods/corona/corona", "icon").Should().Be("mods/corona/corona:icon"); - MetadataFlattener.LocalId("mods/corona/corona:icon").Should().Be("icon"); - MetadataFlattener.PathPrefix( - Path.Combine(RepoMetadataDir, "mods", "corona", "corona.xml"), - RepoMetadataDir) - .Replace('\\', '/').Should().Be("mods/corona/corona"); - } - - [TestMethod] - public void QualifyId_WithManualPrefix_Throws() - { - var act = () => MetadataFlattener.QualifyId("mods/corona/corona", "mods/corona/corona:icon"); - act.Should().Throw().WithMessage("*含 ':'*"); - } - - [TestMethod] - public void Build_DuplicateEntityId_HardFailsWithBothSources() - { - var temp = Path.Combine(Path.GetTempPath(), $"id-dup-{Guid.NewGuid():N}"); - var src = Path.Combine(temp, "src"); - SeedSchemas(src); - Directory.CreateDirectory(Path.Combine(src, "a")); - Directory.CreateDirectory(Path.Combine(src, "b")); - try - { - File.WriteAllText(Path.Combine(src, "a", "mod.xml"), """ - - - - -"""); - File.WriteAllText(Path.Combine(src, "b", "mod.xml"), """ - - - - -"""); - File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - - - - - - - -"""); - - var act = () => MetadataBuilder.Build(src, Path.Combine(temp, "out"), contentRevision: "x"); - act.Should().Throw() - .WithMessage("*实体 ID 重复*") - .WithMessage("*a/mod.xml*") - .WithMessage("*b/mod.xml*"); - } - finally - { - if (Directory.Exists(temp)) - Directory.Delete(temp, recursive: true); - } - } - - [TestMethod] - public void Build_DuplicateEntityIdAcrossKinds_HardFails() - { - var temp = Path.Combine(Path.GetTempPath(), $"id-dup-kind-{Guid.NewGuid():N}"); - var src = Path.Combine(temp, "src"); - SeedSchemas(src); - Directory.CreateDirectory(Path.Combine(src, "a")); - Directory.CreateDirectory(Path.Combine(src, "b")); - try - { - File.WriteAllText(Path.Combine(src, "a", "mod.xml"), """ - - - - -"""); - File.WriteAllText(Path.Combine(src, "b", "app.xml"), """ - - - - -"""); - File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - - - - - - - -"""); - - var act = () => MetadataBuilder.Build(src, Path.Combine(temp, "out"), contentRevision: "x"); - act.Should().Throw() - .WithMessage("*实体 ID 重复*") - .WithMessage("*Corona*"); - } - finally - { - if (Directory.Exists(temp)) - Directory.Delete(temp, recursive: true); - } - } - - [TestMethod] - public void Build_ManualQualifiedRegistrationId_HardFails() - { - var temp = Path.Combine(Path.GetTempPath(), $"id-manual-{Guid.NewGuid():N}"); - var src = Path.Combine(temp, "src"); - SeedSchemas(src); - try - { - File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - - - - -"""); - - var act = () => MetadataBuilder.Build(src, Path.Combine(temp, "out"), contentRevision: "x"); - act.Should().Throw().WithMessage("*含 ':'*"); - } - finally - { - if (Directory.Exists(temp)) - Directory.Delete(temp, recursive: true); - } - } - - [TestMethod] - public void Build_SameLocalIdInDifferentMods_BothSurviveAsQualified() - { - var temp = Path.Combine(Path.GetTempPath(), $"id-scope-{Guid.NewGuid():N}"); - var src = Path.Combine(temp, "src"); - SeedSchemas(src); - Directory.CreateDirectory(Path.Combine(src, "a")); - Directory.CreateDirectory(Path.Combine(src, "b")); - try - { - File.WriteAllText(Path.Combine(src, "a", "mod.xml"), """ - - - - - shared-icon - 模组 A - Mod A - - -"""); - File.WriteAllText(Path.Combine(src, "b", "mod.xml"), """ - - - - - shared-icon - 模组 B - Mod B - - -"""); - // minimal png - var png = Convert.FromBase64String("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="); - File.WriteAllBytes(Path.Combine(src, "a", "icon.png"), png); - File.WriteAllBytes(Path.Combine(src, "b", "icon.png"), png); - - File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - - - - - - - -"""); - - var dst = Path.Combine(temp, "out"); - MetadataBuilder.Build(src, dst, contentRevision: "scope"); - var doc = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")); - - var images = doc.GetAllElements("Image").Where(i => MetadataFlattener.LocalId(i.Get("ID")!) == "shared-icon").ToList(); - images.Should().HaveCount(2); - images.Select(i => i.Get("ID")).Should().OnlyHaveUniqueItems(); - - var modA = doc.Mods().Single(m => m.Id == "ModA"); - var modB = doc.Mods().Single(m => m.Id == "ModB"); - modA.Icon.Should().Be("a/mod:shared-icon"); - modB.Icon.Should().Be("b/mod:shared-icon"); - } - finally - { - if (Directory.Exists(temp)) - Directory.Delete(temp, recursive: true); - } - } -} diff --git a/Ra3.BattleNet.Metadata.Tests/IncludeTests.cs b/Ra3.BattleNet.Metadata.Tests/IncludeTests.cs deleted file mode 100644 index 12c4520..0000000 --- a/Ra3.BattleNet.Metadata.Tests/IncludeTests.cs +++ /dev/null @@ -1,37 +0,0 @@ -using FluentAssertions; -using Microsoft.VisualStudio.TestTools.UnitTesting; - -namespace Ra3.BattleNet.Metadata.Tests; - -/// -/// Load 侧拒绝源树文件（含 Include/Module）。Include 展平语义由构建侧覆盖（StageATests）。 -/// -[TestClass] -public class IncludeTests -{ - private string _testDataPath = null!; - - [TestInitialize] - public void Init() - { - _testDataPath = Path.Combine(AppContext.BaseDirectory, "TestData"); - } - - [TestMethod] - public void LoadFromFile_WithInclude_Throws() - { - var filePath = Path.Combine(_testDataPath, "source-with-include.xml"); - var act = () => Metadata.LoadFromFile(filePath); - act.Should().Throw() - .WithMessage("*Include*"); - } - - [TestMethod] - public void LoadFromFile_WithIncludesContainer_Throws() - { - var filePath = Path.Combine(_testDataPath, "source-with-includes-container.xml"); - var act = () => Metadata.LoadFromFile(filePath); - act.Should().Throw() - .WithMessage("*Includes*"); - } -} diff --git a/Ra3.BattleNet.Metadata.Tests/LanguageTagTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/LanguageTagTests.cs similarity index 100% rename from Ra3.BattleNet.Metadata.Tests/LanguageTagTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/LanguageTagTests.cs diff --git a/Ra3.BattleNet.Metadata.Tests/ManifestMountPlannerTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/ManifestMountPlannerTests.cs similarity index 100% rename from Ra3.BattleNet.Metadata.Tests/ManifestMountPlannerTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/ManifestMountPlannerTests.cs diff --git a/Ra3.BattleNet.Metadata.Tests/ModCompatibilityTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/ModCompatibilityTests.cs similarity index 96% rename from Ra3.BattleNet.Metadata.Tests/ModCompatibilityTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/ModCompatibilityTests.cs index 991ebdf..dd6f6a2 100644 --- a/Ra3.BattleNet.Metadata.Tests/ModCompatibilityTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Queries/ModCompatibilityTests.cs @@ -20,13 +20,13 @@ public Tree(string content) var metadata = Path.GetFullPath(Path.Combine(AppContext.BaseDirectory, "..", "..", "..", "..", "Metadata")); foreach (var schema in new[] { SchemaValidator.SourceSchemaFileName, SchemaValidator.PublishSchemaFileName }) File.Copy(Path.Combine(metadata, schema), Path.Combine(Source, schema)); - File.WriteAllText(Path.Combine(Source, "metadata.xml"), "" + content + ""); + File.WriteAllText(Path.Combine(Source, "metadata.xml"), "" + content + ""); } public MetadataCatalog Build() { - MetadataBuilder.Build(Source, Output, contentRevision: "compatibility-test"); + MetadataBuilder.Build(Source, Output); SchemaValidator.ValidateFile(Path.Combine(Output, "metadata.xml"), Path.Combine(Source, SchemaValidator.PublishSchemaFileName)).Should().BeEmpty(); - return MetadataBuilder.Load(Path.Combine(Output, "metadata.xml")).Catalog(); + return MetadataReader.Read(Path.Combine(Output, "metadata.xml")).Root.Catalog(); } public void Dispose() => Directory.Delete(_root, recursive: true); } diff --git a/Ra3.BattleNet.Metadata.Tests/PostHeadImageTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/PostHeadImageTests.cs similarity index 92% rename from Ra3.BattleNet.Metadata.Tests/PostHeadImageTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/PostHeadImageTests.cs index 421b44c..6b9c825 100644 --- a/Ra3.BattleNet.Metadata.Tests/PostHeadImageTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Queries/PostHeadImageTests.cs @@ -30,12 +30,12 @@ public void Build_PostHeadImage_QualifiesReferenceAndPublishes() File.WriteAllText(Path.Combine(src, "metadata.xml"), MetadataXml("cover", PostOrder.HeadImageBeforeContents)); - MetadataBuilder.Build(src, dst, contentRevision: "head-image"); - var post = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")) + MetadataBuilder.Build(src, dst); + var post = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root .Mods().Single(m => m.Id == "Demo").Raw .Find("Posts")!.Children.First(c => c.Name == "Post"); - post.Find("HeadImage")!.Value.Should().Be("metadata:cover"); + post.Find("HeadImage")!.Value.Should().Be("metadata/cover"); } finally { @@ -75,7 +75,7 @@ private static void AssertRejected(PostOrder order, string headImage, params str File.WriteAllText(Path.Combine(src, "news.md"), "# 公告\n\n正文内容足够长，用于通过校验。"); File.WriteAllText(Path.Combine(src, "metadata.xml"), MetadataXml(headImage, order)); - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "head-image"); + var act = () => MetadataBuilder.Build(src, dst); var assertion = act.Should().Throw(); foreach (var part in messageParts) assertion.WithMessage(part); @@ -99,7 +99,7 @@ private static string MetadataXml(string headImage, PostOrder order) return $""" - + diff --git a/Ra3.BattleNet.Metadata.Tests/PostTypeTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/PostTypeTests.cs similarity index 95% rename from Ra3.BattleNet.Metadata.Tests/PostTypeTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/PostTypeTests.cs index a719032..ff9f412 100644 --- a/Ra3.BattleNet.Metadata.Tests/PostTypeTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Queries/PostTypeTests.cs @@ -29,7 +29,7 @@ public void SourceAndPublishSchema_PostType_OptionalAndAnyString(string typeAttr var publishPath = Path.Combine(temp, "publish.xml"); File.WriteAllText(sourcePath, Document(typeAttribute, rootAttributes: string.Empty)); File.WriteAllText(publishPath, - Document(typeAttribute, rootAttributes: " SchemaVersion=\"1.0\" ContentRevision=\"post-type\"")); + Document(typeAttribute, rootAttributes: " SchemaVersion=\"1.0\"")); SchemaValidator.ValidateFile(sourcePath, sourceSchema).Should().BeEmpty(because); SchemaValidator.ValidateFile(publishPath, publishSchema).Should().BeEmpty(because); @@ -41,7 +41,7 @@ public void SourceAndPublishSchema_PostType_OptionalAndAnyString(string typeAttr } } - /// 同一份 Post 树分别用作源文件（根无版本属性）与发布物（根带 SchemaVersion/ContentRevision）。 + /// 同一份 Post 树分别用作源文件（根无版本属性）与发布物（根带 SchemaVersion）。 private static string Document(string typeAttribute, string rootAttributes) => $""" diff --git a/Ra3.BattleNet.Metadata.Tests/QueryApiTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/QueryApiTests.cs similarity index 93% rename from Ra3.BattleNet.Metadata.Tests/QueryApiTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/QueryApiTests.cs index 5e65f97..db94dfa 100644 --- a/Ra3.BattleNet.Metadata.Tests/QueryApiTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Queries/QueryApiTests.cs @@ -15,8 +15,8 @@ public class QueryApiTests private static Metadata LoadFlattened(out string outputDir) { outputDir = Path.Combine(Path.GetTempPath(), $"query-api-{Guid.NewGuid():N}"); - MetadataBuilder.Build(RepoMetadataDir, outputDir, contentRevision: "query-api"); - return MetadataBuilder.Load(Path.Combine(outputDir, "metadata.xml")); + MetadataBuilder.Build(RepoMetadataDir, outputDir); + return MetadataReader.Read(Path.Combine(outputDir, "metadata.xml")).Root; } [TestMethod] @@ -34,7 +34,7 @@ public void ModsAndApplications_ListEveryEntity_AndComposeWithLinq() 9.9 """); - var root = Metadata.LoadFromFile(path); + var root = MetadataReader.Read(path).Root; root.Mods().Select(m => m.Id).Should().Equal("Alpha", "Beta"); root.Applications().Select(a => a.Id).Should().Equal("Launcher"); @@ -64,10 +64,11 @@ public void Mod_ManifestSource_ResolvesRegisteredLeafManifest() var corona = root.Catalog().Mod("Corona")!; corona.Package().Should().NotBeNull("版本缺省取实体当前版本"); - corona.ManifestSource().Replace('\\', '/').Should().EndWith("manifests/3.259.xml"); + var leaf = MetadataReader.Read(MetadataResourceUri.Resolve(Path.Combine(dst, "metadata.xml"), corona.ManifestSource()).LocalPath).Root; + leaf.ManifestRegistration(corona.Package()!.ManifestId!)!.ToManifestEntry().Files.Should().NotBeEmpty(); var act = () => corona.ManifestSource("9.9.9"); - act.Should().Throw().WithMessage("*没有版本*"); + act.Should().Throw(); } finally { @@ -102,7 +103,7 @@ public void Build_PackagesAndPosts_AreIndependent() File.WriteAllText(Path.Combine(src, "bare-en.md"), bareEn); File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + @@ -174,10 +175,9 @@ public void Build_PackagesAndPosts_AreIndependent() """); - MetadataBuilder.Build(src, dst, schemaVersion: "1.0", contentRevision: "posts-independent"); - var loaded = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")); + MetadataBuilder.Build(src, dst); + var loaded = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; loaded.Get("SchemaVersion").Should().Be("1.0"); - loaded.Get("ContentRevision").Should().Be("posts-independent"); var packageOnly = loaded.Catalog().Mod("PackageOnly")!; packageOnly.Version.Should().Be("9.0"); @@ -219,7 +219,6 @@ static string ContentId(Metadata post, string language) => static void AssertPublishedBody(Metadata loaded, string outputDir, string contentId, string expected) { - contentId.Should().Contain(":"); var markdown = loaded.Markdowns().Single(m => m.Id == contentId); var relative = markdown.Source!.Replace('/', Path.DirectorySeparatorChar); File.ReadAllText(Path.Combine(outputDir, relative)).Should().Be(expected); diff --git a/Ra3.BattleNet.Metadata.Tests/StyleResourceTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/StyleResourceTests.cs similarity index 94% rename from Ra3.BattleNet.Metadata.Tests/StyleResourceTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/StyleResourceTests.cs index 66a4cf3..e5b4ac6 100644 --- a/Ra3.BattleNet.Metadata.Tests/StyleResourceTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Queries/StyleResourceTests.cs @@ -64,7 +64,7 @@ public void Build_InheritedStyleImages_KeepQualifiedIdsAndValues() { File.WriteAllText(Path.Combine(src, "metadata.xml"), """ - + @@ -95,12 +95,12 @@ public void Build_InheritedStyleImages_KeepQualifiedIdsAndValues() """); - MetadataBuilder.Build(src, dst, contentRevision: "style-inherit"); - var loaded = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")); + MetadataBuilder.Build(src, dst); + var loaded = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; var style = loaded.Mods().Single(m => m.Id == "Demo").Raw.Find("Style")!; var logo = style.Find("Logo")!; - logo.Value.Should().Be("styles/theme:banner"); + logo.Value.Should().Be("styles/theme/banner"); logo.Get("Width").Should().Be("400"); logo.Get("Height").Should().Be("160"); logo.Get("Position").Should().Be("TopLeft"); @@ -110,7 +110,7 @@ public void Build_InheritedStyleImages_KeepQualifiedIdsAndValues() var background = style.Find("Background")!; background.Get("Random").Should().Be("true"); background.Children.Where(c => c.Name == "Image").Select(c => c.Value) - .Should().Equal("styles/theme:bg-a", "styles/theme:bg-b"); + .Should().Equal("styles/theme/bg-a", "styles/theme/bg-b"); var color = background.Find("Color")!; color.Value.Should().Be("#112233"); color.Get("Format").Should().Be("CSS"); @@ -147,7 +147,7 @@ private static void AssertStyleRefsRejected(bool packages, bool posts, string lo """); - var act = () => MetadataBuilder.Build(src, dst, contentRevision: "style-ref"); + var act = () => MetadataBuilder.Build(src, dst); act.Should().Throw() .WithMessage("*Logo*") .WithMessage($"*{logoRef}*") @@ -190,7 +190,7 @@ private static string MetadataXml(bool packages, bool posts, string logoRef, str return $""" - + diff --git a/Ra3.BattleNet.Metadata.Tests/StyleTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/StyleTests.cs similarity index 97% rename from Ra3.BattleNet.Metadata.Tests/StyleTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/StyleTests.cs index adce3f0..e78eb2f 100644 --- a/Ra3.BattleNet.Metadata.Tests/StyleTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Queries/StyleTests.cs @@ -46,7 +46,7 @@ public void Schemas_AllowOmittedFieldsWithoutMaterializingValues() { var xml = Path.Combine(temp, "metadata.xml"); File.WriteAllText(xml, """ - + "); + File.WriteAllText(xml, $""); foreach (var name in new[] { SchemaValidator.SourceSchemaFileName, SchemaValidator.PublishSchemaFileName }) SchemaValidator.ValidateFile(xml, Path.Combine(RepoMetadataDir, name)).Should().NotBeEmpty(); } diff --git a/Ra3.BattleNet.Metadata.Tests/UpdaterEndpointTests.cs b/Ra3.BattleNet.Metadata.Tests/Queries/UpdaterEndpointTests.cs similarity index 91% rename from Ra3.BattleNet.Metadata.Tests/UpdaterEndpointTests.cs rename to Ra3.BattleNet.Metadata.Tests/Queries/UpdaterEndpointTests.cs index 44bbda1..b9354ad 100644 --- a/Ra3.BattleNet.Metadata.Tests/UpdaterEndpointTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Queries/UpdaterEndpointTests.cs @@ -32,7 +32,7 @@ public void ResolveUpdaterEndpoint_UsesFirstDirectKind_AndResolvesRelativeSource """); try { - var app = Metadata.LoadFromFile(path).Catalog().Application("RA3BattleNet"); + var app = MetadataReader.Read(path).Root.Catalog().Application("RA3BattleNet"); var endpoint = app!.ResolveUpdaterEndpoint(new Uri("https://metadata.example/metadata.xml")); endpoint.Should().NotBeNull(); @@ -75,12 +75,12 @@ public void ResolveUpdaterEndpoint_MissingOrUnmatched_ReturnsNull() try { var origin = new Uri("https://metadata.example/metadata.xml"); - Metadata.LoadFromFile(unmatched).Catalog().Application("RA3BattleNet")! + MetadataReader.Read(unmatched).Root.Catalog().Application("RA3BattleNet")! .ResolveUpdaterEndpoint(origin).Should().BeNull("第一个 UpdateKind 没有等值版本时不落到后面的更新线"); - Metadata.LoadFromFile(absent).Catalog().Application("RA3BattleNet")! + MetadataReader.Read(absent).Root.Catalog().Application("RA3BattleNet")! .ResolveUpdaterEndpoint(origin).Should().BeNull(); - var app = Metadata.LoadFromFile(absent).Catalog().Application("RA3BattleNet")!; + var app = MetadataReader.Read(absent).Root.Catalog().Application("RA3BattleNet")!; var act = () => app.ResolveUpdaterEndpoint(null!); act.Should().Throw(); } diff --git a/Ra3.BattleNet.Metadata.Tests/Reading/HttpSourceTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/HttpSourceTests.cs new file mode 100644 index 0000000..574d549 --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Reading/HttpSourceTests.cs @@ -0,0 +1,371 @@ +using System.Net; +using System.Net.Http.Headers; +using System.Text; +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +/// +/// HttpXmlSource：URL 路径语义、GET 状态分支、重定向拒绝与 XML 正文缓存， +/// 以及统一读取器的远端组合、路径边界和不预取资源行为。 +/// 全部请求由内存 handler 应答，测试不发真实网络请求。 +/// +[TestClass] +public class HttpSourceTests +{ + private const string Root = "https://example.test/mods/"; + + private static HttpXmlSource Source(FakeHandler handler) => new(new Uri(Root), handler); + + [TestMethod] + public void GetFullPath_ResolvesRelativeAndNormalizesUrl() + { + using var source = new HttpXmlSource(new Uri(Root)); + + source.GetFullPath("metadata.xml").Should().Be(Root + "metadata.xml"); + source.GetFullPath("parts/./more/../base.xml").Should().Be(Root + "parts/base.xml"); + source.GetFullPath("images/").Should().Be(Root + "images/"); + source.GetFullPath(Root).Should().Be(Root); + source.GetFullPath("https://Example.TEST/Mods/../Mods/base.xml") + .Should().Be("https://example.test/Mods/base.xml"); + } + + [TestMethod] + public void Combine_MergesIntoDirectoryAndNormalizesBackSlash() + { + using var source = Source(new FakeHandler()); + + source.Combine(Root + "parts/", "more/leaf.xml").Should().Be(Root + "parts/more/leaf.xml"); + source.Combine(Root + "parts/", @"more\leaf.xml").Should().Be(Root + "parts/more/leaf.xml"); + source.Combine("https://example.test/mods/parts", "more/leaf.xml") + .Should().Be(Root + "parts/more/leaf.xml"); + source.Combine(Root + "parts/", "https://other.test/x.xml").Should().Be("https://other.test/x.xml"); + } + + [TestMethod] + public void GetDirectoryName_ReturnsParentDirectoryWithTrailingSlash() + { + using var source = Source(new FakeHandler()); + + source.GetDirectoryName(Root + "parts/base.xml").Should().Be(Root + "parts/"); + source.GetDirectoryName(Root + "metadata.xml").Should().Be(Root); + source.GetDirectoryName(Root + "parts/").Should().Be(Root); + source.GetDirectoryName("https://example.test/metadata.xml").Should().Be("https://example.test/"); + source.GetDirectoryName("https://example.test/").Should().BeNull(); + } + + [TestMethod] + public void GetRelativePath_SameOriginUnderFrom_ReturnsRelative() + { + using var source = Source(new FakeHandler()); + + source.GetRelativePath(Root, Root + "parts/base.xml").Should().Be("parts/base.xml"); + source.GetRelativePath("https://example.test/mods", Root + "images/logo.png") + .Should().Be("images/logo.png"); + } + + [TestMethod] + public void GetRelativePath_CrossOriginStaysAbsoluteAndSameOriginCanTraverseParent() + { + using var source = Source(new FakeHandler()); + + source.GetRelativePath(Root, "https://other.test/x.xml").Should().Be("https://other.test/x.xml"); + source.GetRelativePath(Root, "https://example.test/outside.xml") + .Should().Be("../outside.xml"); + source.GetRelativePath(Root, "http://example.test/mods/x.xml") + .Should().Be("http://example.test/mods/x.xml"); + } + + [TestMethod] + [DataRow("part%2520name.xml", "part%2520name/release")] + [DataRow("part%2Fname.xml", "part%2Fname/release")] + public void Read_EscapedDocumentPathsPreserveIdentityAndResourceAddresses(string leaf, string id) + { + var handler = new FakeHandler(); + handler.AddXml(Root + "metadata.xml", $""); + handler.AddXml(Root + leaf, ""); + using var source = Source(handler); + source.GetRelativePath(Root, Root + leaf).Should().Be(leaf); + var read = MetadataReader.Read(Root + "metadata.xml", new MetadataSourceOptions + { CustomSource = source, SourceRoot = Root }); + read.Root.GetAllElements("Manifest").Single().Get("ID").Should().Be(id); + read.Root.Images().Single().Source.Should().Be("images/icon%2520%2Fname.png"); + new Uri(new Uri(Root), read.Root.Images().Single().Source!).AbsoluteUri + .Should().Be(Root + "images/icon%2520%2Fname.png"); + handler.Requests.Should().BeEquivalentTo(Root + "metadata.xml", Root + leaf); + } + + [TestMethod] + public void IsPathRooted_OnlyAbsoluteHttpUrls() + { + using var source = Source(new FakeHandler()); + + source.IsPathRooted(Root + "metadata.xml").Should().BeTrue(); + source.IsPathRooted("metadata.xml").Should().BeFalse(); + source.IsPathRooted("parts/base.xml").Should().BeFalse(); + } + + [TestMethod] + public void Exists_OkIsTrue_NotFoundIsFalse() + { + var handler = new FakeHandler(); + handler.AddContent(Root + "images/logo.png", [0x89, 0x50, 0x4E, 0x47], "image/png"); + using var source = Source(handler); + + source.Exists(Root + "images/logo.png").Should().BeTrue(); + source.Exists(Root + "images/missing.png").Should().BeFalse(); + handler.Requests.Should().Equal(Root + "images/logo.png", Root + "images/missing.png"); + } + + [TestMethod] + public void Exists_OtherStatus_Throws() + { + var handler = new FakeHandler(); + handler.AddStatus(Root + "images/boom.png", HttpStatusCode.InternalServerError); + using var source = Source(handler); + + var act = () => source.Exists(Root + "images/boom.png"); + + act.Should().Throw(); + } + + [TestMethod] + public void Get_RedirectResponse_ThrowsAndDoesNotFollow() + { + var handler = new FakeHandler(); + handler.AddRedirect(Root + "redirected.xml", HttpStatusCode.Redirect, "https://evil.test/stolen.xml"); + using var source = Source(handler); + + var act = () => source.LoadDocument(Root + "redirected.xml"); + + act.Should().Throw() + .WithMessage("*302*") + .WithMessage("*" + Root + "redirected.xml*"); + handler.Requests.Should().Equal(Root + "redirected.xml"); + } + + [TestMethod] + public void ExistsThenLoadDocument_XmlBody_ReusesCachedResponse() + { + var handler = new FakeHandler(); + handler.AddXml(Root + "metadata.xml", """"""); + using var source = Source(handler); + + source.Exists(Root + "metadata.xml").Should().BeTrue(); + source.LoadDocument(Root + "metadata.xml").Root!.Name.LocalName.Should().Be("Metadata"); + + handler.Requests.Should().Equal(Root + "metadata.xml"); + } + + [TestMethod] + public void Exists_XmlWithoutContentType_IdentifiedByExtension() + { + var handler = new FakeHandler(); + handler.AddContent(Root + "metadata.xml", Encoding.UTF8.GetBytes("")); + using var source = Source(handler); + + source.Exists(Root + "metadata.xml").Should().BeTrue(); + source.LoadDocument(Root + "metadata.xml").Root!.Name.LocalName.Should().Be("Metadata"); + + handler.Requests.Should().Equal(Root + "metadata.xml"); + } + + [TestMethod] + public void Exists_ImageResponse_DoesNotCacheBody() + { + var handler = new FakeHandler(); + handler.AddContent(Root + "images/logo.png", [0x89, 0x50, 0x4E, 0x47], "image/png"); + handler.AddXml(Root + "images/logo.png", ""); + using var source = Source(handler); + + source.Exists(Root + "images/logo.png").Should().BeTrue(); + // 图片正文没进缓存，LoadDocument 必须重新请求（这一次拿到的是 XML） + source.LoadDocument(Root + "images/logo.png").Root!.Name.LocalName.Should().Be("Metadata"); + + handler.Requests.Should().HaveCount(2); + } + + [TestMethod] + public void LoadDocument_ParsesXml_MissingThrowsFileNotFound() + { + var handler = new FakeHandler(); + handler.AddXml(Root + "metadata.xml", """"""); + using var source = Source(handler); + + source.LoadDocument(Root + "metadata.xml").Root!.Name.LocalName.Should().Be("Metadata"); + + var act = () => source.LoadDocument(Root + "missing.xml"); + act.Should().Throw().WithMessage("*missing.xml*"); + } + + [TestMethod] + public void LoadDocument_ProhibitsDtd() + { + var handler = new FakeHandler(); + handler.AddXml(Root + "evil.xml", """ + +]> +&xxe; +"""); + using var source = Source(handler); + + var act = () => source.LoadDocument(Root + "evil.xml"); + + act.Should().Throw(); + } + + [TestMethod] + public void Read_RemoteTree_ExpandsIncludesWithoutFetchingResources() + { + var handler = new FakeHandler(); + handler.AddXml(Root + "metadata.xml", """ + + + + + +"""); + handler.AddXml(Root + "parts/base.xml", """ + + + + + + +"""); + handler.AddXml(Root + "parts/more/leaf.xml", """ + + + +"""); + handler.AddContent(Root + "images/logo.png", [0x89, 0x50, 0x4E, 0x47], "image/png"); + handler.AddContent(Root + "parts/more/images/leaf.png", [0x89, 0x50, 0x4E, 0x47], "image/png"); + + using var source = Source(handler); + var doc = MetadataReader.Read(Root + "metadata.xml", new MetadataSourceOptions + { SourceRoot = Root, CustomSource = source }).Document; + + doc.Root!.Attribute("SchemaVersion").Should().BeNull(); + doc.Descendants().Should().NotContain(e => + e.Name.LocalName == "Include" || e.Name.LocalName == "Includes"); + + var images = doc.Descendants("Image").ToList(); + images.Should().HaveCount(2); + + var logo = images.Single(i => i.Attribute("ID")!.Value.EndsWith("/logo", StringComparison.Ordinal)); + logo.Attribute("ID")!.Value.Should().Be("parts/base/logo"); + logo.Attribute("Source")!.Value.Should().Be("images/logo.png"); + + var leaf = images.Single(i => i.Attribute("ID")!.Value.EndsWith("/leaf", StringComparison.Ordinal)); + leaf.Attribute("ID")!.Value.Should().Be("parts/more/leaf/leaf"); + leaf.Attribute("Source")!.Value.Should().Be("parts/more/images/leaf.png"); + + // 递归 Include 真的发了请求，且全部落在源根下（没有任何真实外部请求） + handler.Requests.Should().Contain(Root + "metadata.xml"); + handler.Requests.Should().Contain(Root + "parts/base.xml"); + handler.Requests.Should().Contain(Root + "parts/more/leaf.xml"); + handler.Requests.Should().BeEquivalentTo(Root + "metadata.xml", Root + "parts/base.xml", Root + "parts/more/leaf.xml"); + } + + [TestMethod] + public void Read_IncludeEscapingSourceRoot_RejectedBeforeFetching() + { + var handler = new FakeHandler(); + handler.AddXml(Root + "metadata.xml", """ + + + + + +"""); + using var source = Source(handler); + + var act = () => MetadataReader.Read(Root + "metadata.xml", new MetadataSourceOptions + { SourceRoot = Root, CustomSource = source }); + + act.Should().Throw().WithMessage("*Include 越出源目录*"); + handler.Requests.Should().OnlyContain(url => url == Root + "metadata.xml"); + } + + [TestMethod] + public void Read_IncludeCrossHostAbsoluteUrl_RejectedBeforeFetching() + { + var handler = new FakeHandler(); + handler.AddXml(Root + "metadata.xml", """ + + + + + +"""); + using var source = Source(handler); + + var act = () => MetadataReader.Read(Root + "metadata.xml", new MetadataSourceOptions + { SourceRoot = Root, CustomSource = source }); + + act.Should().Throw().WithMessage("*Include 越出源目录*"); + handler.Requests.Should().OnlyContain(url => url == Root + "metadata.xml"); + } + + /// 按绝对 URL 路由的内存报文 handler：未登记的 URL 一律 404，绝不发真实请求。 + private sealed class FakeHandler : HttpMessageHandler + { + private readonly Dictionary>> _routes = + new(StringComparer.Ordinal); + + public List Requests { get; } = new(); + + public void AddXml(string url, string xml) => + Add(url, () => new HttpResponseMessage(HttpStatusCode.OK) + { + Content = new StringContent(xml, Encoding.UTF8, "application/xml") + }); + + public void AddContent(string url, byte[] content, string? mediaType = null) => + Add(url, () => + { + var body = new ByteArrayContent(content); + if (mediaType is not null) + body.Headers.ContentType = new MediaTypeHeaderValue(mediaType); + return new HttpResponseMessage(HttpStatusCode.OK) { Content = body }; + }); + + public void AddStatus(string url, HttpStatusCode statusCode) => + Add(url, () => new HttpResponseMessage(statusCode)); + + public void AddRedirect(string url, HttpStatusCode statusCode, string location) => + Add(url, () => + { + var response = new HttpResponseMessage(statusCode); + response.Headers.Location = new Uri(location); + return response; + }); + + private void Add(string url, Func response) + { + if (!_routes.TryGetValue(url, out var queue)) + _routes[url] = queue = new Queue>(); + queue.Enqueue(response); + } + + protected override Task SendAsync( + HttpRequestMessage request, CancellationToken cancellationToken) => + Task.FromResult(Respond(request)); + + // HttpXmlSource 走同步 Send（IMetadataXmlSource 是同步接口），自定义 handler 必须覆盖这个重载 + protected override HttpResponseMessage Send( + HttpRequestMessage request, CancellationToken cancellationToken) => Respond(request); + + private HttpResponseMessage Respond(HttpRequestMessage request) + { + var url = request.RequestUri!.AbsoluteUri; + Requests.Add(url); + if (!_routes.TryGetValue(url, out var queue)) + return new HttpResponseMessage(HttpStatusCode.NotFound); + + // 同一 URL 多次请求：队列只有一条时复用最后一条（每次都造新响应实例） + return queue.Count > 1 ? queue.Dequeue()() : queue.Peek()(); + } + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/Reading/IdScopeTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/IdScopeTests.cs new file mode 100644 index 0000000..01496ce --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Reading/IdScopeTests.cs @@ -0,0 +1,38 @@ +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +[TestClass] +public sealed class IdScopeTests +{ + [TestMethod] + public void PathPrefix_EscapesDefinitionSegmentsAndPreservesVersionDots() + { + var root = Path.Combine(Path.GetTempPath(), "source"); + MetadataId.PathPrefix(Path.Combine(root, "space name", "3.259.xml"), root) + .Should().Be("space%20name/3.259"); + MetadataId.Qualify("space%20name/3.259", "icon%name").Should().Be("space%20name/3.259/icon%25name"); + MetadataId.LocalId("space%20name/3.259/icon%25name").Should().Be("icon%name"); + } + + + [TestMethod] + [DataRow("Mod")] + [DataRow("Application")] + public void Read_DuplicateEntityIdsAcrossDocuments_ReportsBothDefinitions(string kind) + { + var root = Path.Combine(Path.GetTempPath(), $"identity-{Guid.NewGuid():N}"); + Directory.CreateDirectory(root); + try + { + File.WriteAllText(Path.Combine(root, "entry.xml"), ""); + File.WriteAllText(Path.Combine(root, "a.xml"), ""); + File.WriteAllText(Path.Combine(root, "b.xml"), $"<{kind} ID=\"Same\"/>"); + var act = () => MetadataReader.Read(Path.Combine(root, "entry.xml")); + var exception = act.Should().Throw().Which; + exception.Message.Should().Contain(Path.Combine(root, "a.xml")).And.Contain(Path.Combine(root, "b.xml")); + } + finally { Directory.Delete(root, recursive: true); } + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/Reading/MetadataReaderTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/MetadataReaderTests.cs new file mode 100644 index 0000000..365fc14 --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Reading/MetadataReaderTests.cs @@ -0,0 +1,401 @@ +using System.Xml.Linq; +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +[TestClass] +public sealed class MetadataReaderTests +{ + [TestMethod] + public void Read_BaseWithoutInclude_ResolvesInheritanceAndResourceReferences() + { + var source = new MemorySource(); + source.Add("entry.xml", """ +icon +"""); + var result = Read(source); + result.Root.Mods().Single().Icon.Should().Be("entry/icon"); + result.Document.Descendants("Base").Should().BeEmpty(); + source.LoadDocument(MemorySource.Root + "entry.xml").Descendants("Image").Single().Attribute("ID")!.Value.Should().Be("icon"); + } + + [TestMethod] + [DataRow("")] + [DataRow("%raw")] + [DataRow("%20")] + public void Read_FullIdsMatchRegistrationsAndExternalManifestWithoutConversion(string suffix) + { + var source = new MemorySource(); + source.Add("entry.xml", $""" +icon:main{suffix}releases/v1:manifest{suffix}posts/intro:zh{suffix} +"""); + source.Add("releases/v1.xml", $""" +mod.big/MOD; +"""); + var result = MetadataReader.Read(MemorySource.Root + "entry.xml", new MetadataSourceOptions + { + CustomSource = source, + ValidateReferences = true, + InlineManifests = true + }); + var mod = result.Root.Mods().Single(); + mod.Id.Should().Be("Demo:one"); + mod.Icon.Should().Be("icon:main" + suffix); + result.Root.Images().Single(i => i.Id == mod.Icon).Url.Should().Be("https://assets.test/icon.png"); + result.Root.Markdowns().Single().Id.Should().Be("posts/intro:zh" + suffix); + mod.Package("1")!.Raw.Find("Manifest")!.Value.Should().Be("releases/v1:manifest" + suffix); + result.Root.ManifestRegistration("releases/v1:manifest" + suffix)!.Find("File")!.Find("FileName")!.Value.Should().Be("mod.big"); + } + + [TestMethod] + public void Read_ThreeMatchingAliases_RemainAmbiguous() + { + var source = new MemorySource(); + source.Add("entry.xml", """ +shared +"""); + foreach (var name in new[] { "a", "b", "c" }) + source.Add(name + ".xml", ""); + var act = () => Read(source); + act.Should().Throw(); + } + + [TestMethod] + public void Read_ChildAndBaseReferencesResolveParentAndSiblingRegistrations() + { + var source = new MemorySource(); + source.Add("entry.xml", """ + +"""); + source.Add("child.xml", """ +parentIcon +"""); + source.Add("base.xml", """ +parentIcon +"""); + source.Add("resources.xml", """ + +"""); + var result = Read(source); + foreach (var mod in result.Root.Mods()) + { + mod.Icon.Should().Be("entry/parentIcon"); + mod.Raw.Find("Style:Logo")!.Value.Should().Be("resources/siblingLogo"); + } + MetadataValidator.FindMissingReferences(result.Root).Should().BeEmpty(); + } + + [TestMethod] + public void Read_MixedShortAndCanonicalIds_PreservesScopeAndAddress() + { + var source = new MemorySource(); + source.Add("entry.xml", """ +icon +"""); + source.Add("split/flat.xml", """ + +"""); + var result = Read(source); + result.Root.Images().Select(i => i.Id).Should().BeEquivalentTo("original/icons/icon", "entry/local"); + result.Root.Images().Single(i => i.Id == "original/icons/icon").Source.Should().Be("split/picture.png"); + result.Documents[0].Includes.Single().TargetPath.Should().Be(MemorySource.Root + "split/flat.xml"); + result.Documents[1].NormalizedDocument.Descendants("Image").Single().Attribute("Source")!.Value + .Should().Be(MemorySource.Root + "split/picture.png"); + } + + [TestMethod] + public void Read_OwnShortReferenceWinsOverSiblingRegistration() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("published.xml", """ +icon +"""); + source.Add("child.xml", """ +icon +"""); + var root = Read(source).Root; + root.Mods().Single(m => m.Id == "Child").Icon.Should().Be("child/icon"); + root.Mods().Single(m => m.Id == "Published").Icon.Should().Be("published/icon"); + MetadataValidator.FindMissingReferences(root).Should().BeEmpty(); + } + + [TestMethod] + public void Read_EntryBelowAllowedRoot_ResourceBaseRemainsEntry() + { + var source = new MemorySource(); + source.Add("nested/entry.xml", """ + +"""); + var result = MetadataReader.Read(MemorySource.Root + "nested/entry.xml", new MetadataSourceOptions + { + CustomSource = source, + SourceRoot = MemorySource.Root + }); + result.Root.Images().Single().Source.Should().Be("icon.png"); + result.Root.Images().Single().Id.Should().Be("nested/entry/icon"); + } + + [TestMethod] + public void Read_SingleDocument_QualifiesShortIdsWithoutProbingResources() + { + var source = new MemorySource(); + source.Add("entry.xml", """ +icon +"""); + var result = Read(source); + result.Root.Images().Single().Id.Should().Be("entry/icon"); + result.Root.Images().Single().Source.Should().Be("assets/icon.png"); + result.Root.Get("SchemaVersion").Should().Be("declared"); + source.Probes.Should().BeEmpty(); + } + + [TestMethod] + public void Read_CanonicalDuplicatesAcrossDocuments_Fail() + { + var source = new MemorySource(); + source.Add("entry.xml", """ + +"""); + source.Add("a.xml", ""); + source.Add("b.xml", ""); + var act = () => Read(source); + act.Should().Throw(); + } + + [TestMethod] + public void Read_HttpPathsDifferingOnlyByCase_AreDifferentDocuments() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("A.xml", ""); + source.Add("a.xml", ""); + Read(source).Root.Mods().Select(m => m.Id).Should().BeEquivalentTo("Upper", "Lower"); + } + + [TestMethod] + public void Read_DiamondGraph_CapturesAndExpandsOnce() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("a.xml", ""); + source.Add("shared.xml", ""); + var result = Read(source); + result.Documents.Should().HaveCount(3); + result.Root.Mods().Should().ContainSingle(); + } + + [TestMethod] + public void Read_IncludedFullManifest_RemainsInline() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("wrapper.xml", ""); + source.Add("normalized.xml", """ +release/v1 +"""); + var manifest = Read(source).Root.ManifestRegistration("release/v1")!; + manifest.Get("Source").Should().BeNullOrEmpty(); + manifest.GetAllElements("File").Should().ContainSingle(); + } + + [TestMethod] + public void Read_ExternalManifestComposition_UsesSameReader() + { + var source = new MemorySource(); + source.Add("entry.xml", """ +release +"""); + source.Add("leaves/wrapper.xml", ""); + source.Add("leaves/actual.xml", ""); + var result = MetadataReader.Read(MemorySource.Root + "entry.xml", new MetadataSourceOptions + { + CustomSource = source, + InlineManifests = true + }); + result.Root.ManifestRegistration("leaves/actual/release")!.GetAllElements("File").Should().ContainSingle(); + result.Documents.Should().HaveCount(3); + } + + [TestMethod] + public void Read_UrlImageDoesNotResolveInformationalSource() + { + var source = new MemorySource(); + source.Add("entry.xml", """ + +"""); + var settings = new MetadataSourceOptions { CustomSource = source }; + var read = MetadataReader.Read(MemorySource.Root + "entry.xml", settings).Root; + read.Images().Single().Source.Should().Be("../../outside.png"); + source.Loads.Should().OnlyContain(path => path == MemorySource.Root + "entry.xml"); + source.Probes.Should().BeEmpty(); + } + + [TestMethod] + public void Read_IncludeTraversalFailsBeforeLoadingOutsideDocument() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + var act = () => Read(source); + act.Should().Throw(); + source.Loads.Should().Equal(MemorySource.Root + "entry.xml"); + } + + [TestMethod] + public void PathIdentity_PreservesCanonicalAndEscapesLocalSegments() + { + MetadataId.Qualify("mods/space%20name/versions", "3.259%icon") + .Should().Be("mods/space%20name/versions/3.259%25icon"); + MetadataId.LocalId("mods/space%20name/versions/3.259%25icon").Should().Be("3.259%icon"); + MetadataId.Qualify("other", "mods/icon").Should().Be("mods/icon"); + } + + [TestMethod] + [DataRow("file:///local/icon.png")] + [DataRow("C:\\local\\icon.png")] + public void Read_HttpDocumentRejectsLocalResources(string path) + { + var source = new MemorySource(); + source.Add("entry.xml", new XElement("Metadata", new XElement("Image", + new XAttribute("ID", "icon"), new XAttribute("Source", path))).ToString()); + var act = () => Read(source); + act.Should().Throw(); + source.Probes.Should().BeEmpty(); + } + + [TestMethod] + [DataRow("Markdown")] + [DataRow("Manifest")] + public void Read_HttpNonImageUrlCannotBypassLocalSourceRejection(string kind) + { + var source = new MemorySource(); + source.Add("entry.xml", $"<{kind} ID=\"item\" Url=\"https://assets.test/item\" Source=\"file:///C:/outside.xml\"/>"); + var act = () => Read(source); + act.Should().Throw(); + source.Loads.Should().Equal(MemorySource.Root + "entry.xml"); + source.Probes.Should().BeEmpty(); + } + + [TestMethod] + public void Read_CycleFailsBeforeReloadingDocument() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("child.xml", ""); + var act = () => Read(source); + act.Should().Throw(); + source.Loads.Should().Equal(MemorySource.Root + "entry.xml", MemorySource.Root + "child.xml"); + } + + [TestMethod] + [DataRow("Image", "icon")] + [DataRow("Image", "ICON")] + [DataRow("Markdown", "ICON")] + public void Read_DuplicateRegistrationsInOneDocumentFail(string kind, string id) + { + var source = new MemorySource(); + source.Add("entry.xml", $"<{kind} ID=\"{id}\"/>"); + var act = () => Read(source); + act.Should().Throw(); + } + + [TestMethod] + public void Read_ReferenceValidationIsExplicit() + { + var source = new MemorySource(); + source.Add("entry.xml", "missing"); + Read(source).Root.Mods().Single().Icon.Should().Be("missing"); + var act = () => MetadataReader.Read(MemorySource.Root + "entry.xml", + new MetadataSourceOptions { CustomSource = source, ValidateReferences = true }); + act.Should().Throw(); + } + + [TestMethod] + public void Read_WrapperStubMustNameActualCanonicalDefinition() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("wrapper.xml", ""); + source.Add("actual.xml", ""); + var act = () => MetadataReader.Read(MemorySource.Root + "entry.xml", + new MetadataSourceOptions { CustomSource = source, InlineManifests = true }); + act.Should().Throw().WithMessage("*wrapper/release*"); + } + + [TestMethod] + public void Read_ResourceSourceUsesDefiningDocumentWithoutExistenceFallback() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("parts/resources.xml", ""); + var result = Read(source); + result.Root.Images().Single().Source.Should().Be("parts/images/icon.png"); + source.Probes.Should().BeEmpty(); + source.Loads.Should().Equal(MemorySource.Root + "entry.xml", MemorySource.Root + "parts/resources.xml"); + } + + [TestMethod] + public void Read_DescendantAliasWinsOverUnrelatedGlobalAlias() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + source.Add("branch.xml", "icon"); + source.Add("child.xml", ""); + source.Add("other.xml", ""); + Read(source).Root.Mods().Single().Icon.Should().Be("child/icon"); + } + + [TestMethod] + public void Read_ExternalManifestOutsideSourceRootFailsBeforeLoading() + { + var source = new MemorySource(); + source.Add("entry.xml", ""); + var act = () => Read(source); + act.Should().Throw(); + source.Loads.Should().Equal(MemorySource.Root + "entry.xml"); + } + + [TestMethod] + [DataRow("")] + [DataRow("")] + public void Read_ExternalManifestCannotAlsoContainBody(string manifest) + { + var source = new MemorySource(); + source.Add("entry.xml", "" + manifest + ""); + var act = () => Read(source); + act.Should().Throw(); + source.Loads.Should().Equal(MemorySource.Root + "entry.xml"); + } + + private static MetadataReadResult Read(MemorySource source) => MetadataReader.Read(MemorySource.Root + "entry.xml", + new MetadataSourceOptions { CustomSource = source }); + + private sealed class MemorySource : IMetadataXmlSource + { + internal const string Root = "https://reader.test/root/"; + private readonly Dictionary _documents = new(StringComparer.Ordinal); + internal List Loads { get; } = []; + internal List Probes { get; } = []; + internal void Add(string relative, string xml) => _documents[Root + relative] = XDocument.Parse(xml); + public bool Exists(string path) + { + Probes.Add(path); + return _documents.ContainsKey(GetFullPath(path)); + } + public XDocument LoadDocument(string path) + { + Loads.Add(path); + return new XDocument(_documents[path]); + } + public string Combine(string basePath, string relativePath) => new Uri(new Uri(basePath.EndsWith('/') ? basePath : basePath + "/"), relativePath).AbsoluteUri; + public string? GetDirectoryName(string path) => new Uri(new Uri(path), ".").AbsoluteUri; + public string GetRelativePath(string fromPath, string toPath) + { + var prefix = fromPath.EndsWith('/') ? fromPath : fromPath + "/"; + return toPath.StartsWith(prefix, StringComparison.Ordinal) ? toPath[prefix.Length..] : toPath; + } + public string GetFullPath(string path) => new Uri(new Uri(Root), path).AbsoluteUri; + public bool IsPathRooted(string path) => Uri.TryCreate(path, UriKind.Absolute, out _); + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/MetadataTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/MetadataTests.cs similarity index 73% rename from Ra3.BattleNet.Metadata.Tests/MetadataTests.cs rename to Ra3.BattleNet.Metadata.Tests/Reading/MetadataTests.cs index 0fb0c46..ff5dc9a 100644 --- a/Ra3.BattleNet.Metadata.Tests/MetadataTests.cs +++ b/Ra3.BattleNet.Metadata.Tests/Reading/MetadataTests.cs @@ -18,7 +18,7 @@ public void Init() public void LoadFromFile_ValidXml_Success() { var filePath = Path.Combine(_testDataPath, "valid-metadata.xml"); - var metadata = Metadata.LoadFromFile(filePath); + var metadata = MetadataReader.Read(filePath).Root; metadata.Should().NotBeNull(); metadata.Name.Should().Be("Metadata"); metadata.Children.Should().HaveCountGreaterThan(0); @@ -28,15 +28,15 @@ public void LoadFromFile_ValidXml_Success() public void LoadFromFile_InvalidRoot_ThrowsException() { var filePath = Path.Combine(_testDataPath, "invalid-root.xml"); - var act = () => Metadata.LoadFromFile(filePath); - act.Should().Throw(); + var act = () => MetadataReader.Read(filePath); + act.Should().Throw(); } [TestMethod] public void LoadFromFile_MissingFile_ThrowsFileNotFoundException() { var filePath = Path.Combine(_testDataPath, "nonexistent.xml"); - var act = () => Metadata.LoadFromFile(filePath); + var act = () => MetadataReader.Read(filePath); act.Should().Throw(); } @@ -44,7 +44,7 @@ public void LoadFromFile_MissingFile_ThrowsFileNotFoundException() public void Get_MissingVariable_ReturnsDefault() { var filePath = Path.Combine(_testDataPath, "valid-metadata.xml"); - var metadata = Metadata.LoadFromFile(filePath); + var metadata = MetadataReader.Read(filePath).Root; metadata.Get("NonExistent", "default").Should().Be("default"); } @@ -52,7 +52,7 @@ public void Get_MissingVariable_ReturnsDefault() public void Find_ValidPath_ReturnsMetadata() { var filePath = Path.Combine(_testDataPath, "valid-metadata.xml"); - var metadata = Metadata.LoadFromFile(filePath); + var metadata = MetadataReader.Read(filePath).Root; var tags = metadata.Find("Tags"); tags.Should().NotBeNull(); tags!.Name.Should().Be("Tags"); @@ -62,7 +62,7 @@ public void Find_ValidPath_ReturnsMetadata() public void GetAllElements_ByName_ReturnsMatchingElements() { var filePath = Path.Combine(_testDataPath, "valid-metadata.xml"); - var metadata = Metadata.LoadFromFile(filePath); + var metadata = MetadataReader.Read(filePath).Root; var applications = metadata.GetAllElements("Application"); applications.Should().ContainSingle().Which.Get("ID").Should().Be("TestApp"); } @@ -71,22 +71,13 @@ public void GetAllElements_ByName_ReturnsMatchingElements() public void GetElementPath_ReturnsFullPath() { var filePath = Path.Combine(_testDataPath, "valid-metadata.xml"); - var metadata = Metadata.LoadFromFile(filePath); + var metadata = MetadataReader.Read(filePath).Root; var app = metadata.Find("Application"); var path = app?.GetElementPath(); path.Should().Contain("Metadata"); path.Should().Contain("Application"); } - [TestMethod] - public void LoadFromFile_SourceTreeWithInclude_Throws() - { - // Load 只接受展平产物；源树（带 Include）必须走 Build 后再解析 - var filePath = Path.Combine(_testDataPath, "source-with-include.xml"); - var act = () => Metadata.LoadFromFile(filePath); - act.Should().Throw() - .WithMessage("*只接受展平*"); - } [TestMethod] public void BuildThenLoad_RepoSample_AppAndModResolve() @@ -97,13 +88,13 @@ public void BuildThenLoad_RepoSample_AppAndModResolve() try { // 本地测试链路：Build(源仓) → Load(展平缓存)，与生产 URL 同一套解析 - MetadataBuilder.Build(src, dst, schemaVersion: "1.0", contentRevision: "test-rev"); - var metadata = MetadataBuilder.Load(Path.Combine(dst, "metadata.xml")); + MetadataBuilder.Build(src, dst); + var metadata = MetadataReader.Read(Path.Combine(dst, "metadata.xml")).Root; var corona = metadata.Mods().Single(m => m.Id == "Corona"); corona.Version.Should().Be("3.259"); corona.Packages.Should().NotBeEmpty(); - corona.Icon.Should().Contain(":"); + corona.Icon.Should().Contain("/"); var app = metadata.Catalog().Application("RA3BattleNet"); app.Should().NotBeNull(); diff --git a/Ra3.BattleNet.Metadata.Tests/Reading/PathSecurityTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/PathSecurityTests.cs new file mode 100644 index 0000000..ac95487 --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Reading/PathSecurityTests.cs @@ -0,0 +1,334 @@ +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +/// +/// 路径收敛：Include 与资源 Source 里的 .. / 绝对路径 / UNC 越出源根必须在读取目标前被拒绝， +/// 留在源根内的规范化路径不能被误伤。 +/// +[TestClass] +public class PathSecurityTests +{ + private static string CreateTempDir() + { + var dir = Path.Combine(Path.GetTempPath(), $"path-security-{Guid.NewGuid():N}"); + Directory.CreateDirectory(dir); + return dir; + } + + private static void WriteFile(string root, string relative, string content) + { + var path = Path.Combine(root, relative.Replace('/', Path.DirectorySeparatorChar)); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + File.WriteAllText(path, content); + } + + private static void DeleteTemp(string dir) + { + if (Directory.Exists(dir)) + Directory.Delete(dir, recursive: true); + } + + [TestMethod] + public void Include_ParentTraversal_EscapingRoot_Rejected() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "outside.xml", ""); + WriteFile(root, "source/entry.xml", """ + + + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*Include 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Include_NestedParentTraversal_EscapingRoot_Rejected() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "outside.xml", ""); + WriteFile(root, "source/entry.xml", """ + + + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*Include 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Include_AbsolutePathOutsideRoot_Rejected() + { + var root = CreateTempDir(); + try + { + var outside = Path.Combine(root, "outside.xml"); + WriteFile(root, "outside.xml", ""); + WriteFile(root, "source/entry.xml", $""" + + + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*Include 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Include_UncPath_Rejected() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "source/entry.xml", """ + + + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*Include 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Include_DotDotStayingInsideRoot_Allowed() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "source/entry.xml", """ + + + + + +"""); + WriteFile(root, "source/sub/mod.xml", """ + + + 演示 + Demo + + +"""); + + var loaded = MetadataReader.Read(Path.Combine(root, "source", "entry.xml")).Root; + + loaded.Mods().Should().ContainSingle(m => m.Id == "Demo"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Resource_ParentTraversal_EscapingRoot_Rejected() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "outside.png", "png"); + WriteFile(root, "source/entry.xml", """ + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*资源 Source 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Resource_NestedParentTraversal_EscapingRoot_Rejected() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "outside.png", "png"); + WriteFile(root, "source/entry.xml", """ + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*资源 Source 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Resource_AbsolutePathOutsideRoot_Rejected() + { + var root = CreateTempDir(); + try + { + var outside = Path.Combine(root, "outside.png"); + WriteFile(root, "outside.png", "png"); + WriteFile(root, "source/entry.xml", $""" + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*资源 Source 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Resource_UncPath_Rejected() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "source/entry.xml", """ + + + +"""); + + var act = () => MetadataReader.Read(Path.Combine(root, "source", "entry.xml")); + + act.Should().Throw().WithMessage("*资源 Source 越出源目录*"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Resource_DotDotStayingInsideRoot_Allowed() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "source/entry.xml", """ + + + + + + +"""); + WriteFile(root, "source/sub/posts.xml", """ + + + +"""); + WriteFile(root, "source/images/pic.png", "png"); + + var loaded = MetadataReader.Read(Path.Combine(root, "source", "entry.xml")).Root; + + loaded.Images().Single(i => i.Id == "entry/local").Source.Should().Be("images/pic.png"); + loaded.Images().Single(i => i.Id == "sub/posts/pic").Source.Should().Be("images/pic.png"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Resource_UrlNodeWithEscapingSource_NotPathChecked() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "outside.png", "png"); + WriteFile(root, "source/entry.xml", """ + + + +"""); + + // 外链图片不落盘，Source 不参与路径解析，越界不应误伤。 + var loaded = MetadataReader.Read(Path.Combine(root, "source", "entry.xml")).Root; + + var image = loaded.Images().Single(i => i.Id == "entry/remote"); + image.Source.Should().Be("../outside.png"); + image.Url.Should().Be("https://example.com/pic.png"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void Entry_OutsideExplicitSourceRoot_Rejected() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "other/entry.xml", " MetadataReader.Read( + Path.Combine(root, "other", "entry.xml"), + new MetadataSourceOptions { SourceRoot = Path.Combine(root, "source") }); + + act.Should().Throw(); + } + finally + { + DeleteTemp(root); + } + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/Reading/RegistrationValidationTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/RegistrationValidationTests.cs new file mode 100644 index 0000000..c600895 --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Reading/RegistrationValidationTests.cs @@ -0,0 +1,46 @@ +using System.Xml.Linq; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +[TestClass] +public class RegistrationValidationTests +{ + [TestMethod] + public void DuplicateRegistrationIds_AreRejectedAcrossKindsAndCase() + { + var root = MetadataParser.Load(XDocument.Parse(""" + + + + shared + + """)); + + Assert.ThrowsExactly(() => MetadataValidator.EnsureReferences(root)); + } + + [TestMethod] + public void LinkIcon_RequiresRegisteredImage() + { + var root = MetadataParser.Load(XDocument.Parse(""" + + + + + + image + article + missing + + + + + """)); + + var missing = MetadataValidator.FindMissingReferences(root); + Assert.AreEqual(2, missing.Count); + Assert.IsTrue(missing.Any(item => item.Contains("article", StringComparison.Ordinal))); + Assert.IsTrue(missing.Any(item => item.Contains("missing", StringComparison.Ordinal))); + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/Reading/SourceLoadTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/SourceLoadTests.cs new file mode 100644 index 0000000..59216e5 --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Reading/SourceLoadTests.cs @@ -0,0 +1,62 @@ +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +[TestClass] +public sealed class SourceLoadTests +{ + [TestMethod] + public void Read_PhysicalTree_PreservesFilesAndVariablesAndIncludedManifestBody() + { + var root = Path.Combine(Path.GetTempPath(), $"reader-{Guid.NewGuid():N}"); + Directory.CreateDirectory(Path.Combine(root, "parts")); + try + { + var entry = Path.Combine(root, "entry.xml"); + const string xml = """ +${TIMESTAMP}${ENV:MISSING}release +"""; + File.WriteAllText(entry, xml); + File.WriteAllText(Path.Combine(root, "parts", "release.xml"), """ + +"""); + var before = Directory.GetFiles(root, "*", SearchOption.AllDirectories) + .ToDictionary(path => path, File.ReadAllBytes); + var result = MetadataReader.Read(entry); + result.Root.Find("Tags:Commit")!.Value.Should().Be("${TIMESTAMP}"); + result.Root.Find("Tags:Flavor")!.Value.Should().Be("${ENV:MISSING}"); + result.Root.ManifestRegistration("parts/release/release")!.GetAllElements("File").Should().ContainSingle(); + result.Root.ManifestRegistration("parts/release/release")!.Get("Source").Should().BeNull(); + result.Root.Markdowns().Single().Source.Should().Be("parts/news.md"); + Directory.GetFiles(root, "*", SearchOption.AllDirectories).Should().BeEquivalentTo(before.Keys); + foreach (var (path, bytes) in before) File.ReadAllBytes(path).Should().Equal(bytes); + } + finally { Directory.Delete(root, recursive: true); } + } + + [TestMethod] + public void Read_ShortExternalStubAndLeaf_HaveExactIdentityWithSharedRoot() + { + var root = Path.Combine(Path.GetTempPath(), $"reader-{Guid.NewGuid():N}"); + Directory.CreateDirectory(Path.Combine(root, "leaves")); + try + { + var entry = Path.Combine(root, "entry.xml"); + var leafPath = Path.Combine(root, "leaves", "1.0.xml"); + File.WriteAllText(entry, """ +release +"""); + File.WriteAllText(leafPath, ""); + var read = MetadataReader.Read(entry); + var stub = read.Root.ManifestRegistration("leaves/1.0/release")!; + stub.Get("Source").Should().Be("leaves/1.0.xml"); + read.Root.Mods().Single().Packages.Single().ManifestId.Should().Be("leaves/1.0/release"); + var leaf = MetadataReader.Read(leafPath, new MetadataSourceOptions { SourceRoot = read.SourceRoot }); + leaf.Root.GetAllElements("Manifest").Single().Get("ID").Should().Be(stub.Get("ID")); + MetadataReader.Read(entry, new MetadataSourceOptions { InlineManifests = true }).Root + .ManifestRegistration("leaves/1.0/release")!.GetAllElements("File").Should().ContainSingle(); + } + finally { Directory.Delete(root, recursive: true); } + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/Reading/ValidationTests.cs b/Ra3.BattleNet.Metadata.Tests/Reading/ValidationTests.cs new file mode 100644 index 0000000..242cb14 --- /dev/null +++ b/Ra3.BattleNet.Metadata.Tests/Reading/ValidationTests.cs @@ -0,0 +1,344 @@ +using FluentAssertions; +using Microsoft.VisualStudio.TestTools.UnitTesting; + +namespace Ra3.BattleNet.Metadata.Tests; + +[TestClass] +public class ValidationTests +{ + private static string CreateTempDir() + { + var dir = Path.Combine(Path.GetTempPath(), $"metadata-validation-{Guid.NewGuid():N}"); + Directory.CreateDirectory(dir); + return dir; + } + + private static void WriteFile(string root, string relative, string content) + { + var path = Path.Combine(root, relative.Replace('/', Path.DirectorySeparatorChar)); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + File.WriteAllText(path, content); + } + + private static void DeleteTemp(string dir) + { + if (Directory.Exists(dir)) + Directory.Delete(dir, recursive: true); + } + + private static Metadata LoadTree(string root) => + MetadataReader.Read(Path.Combine(root, "metadata.xml")).Root; + + [TestMethod] + public void EnsureReferences_AllReferenceFieldsResolve_DoesNotThrow() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "metadata.xml", """ + + + + + + + + + 1.0 + icon + 演示 + Demo + + + manifest-1 + + + + + head + + news + news + + + + + + +"""); + WriteFile(root, "images/res.xml", """ + + + + + + + +"""); + WriteFile(root, "news/news.xml", """ + + + + +"""); + WriteFile(root, "manifests/1.0.xml", """ + + + + + demo.lyi + / + MOD; + + + +"""); + + var loaded = LoadTree(root); + + // 短名引用在展平时已限定为跨文件前缀 ID，校验器比对的是限定后的 ID。 + loaded.Find("Mod:Icon")!.Value.Should().Be("images/res/icon"); + loaded.Find("Mod:Packages:Package:Manifest")!.Value.Should().Be("manifests/1.0/manifest-1"); + + MetadataValidator.FindMissingReferences(loaded).Should().BeEmpty(); + + var act = () => MetadataValidator.EnsureReferences(loaded); + act.Should().NotThrow(); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void FindMissingReferences_CrossFilePrefixedId_Resolves() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "metadata.xml", """ + + + + + + icon + + +"""); + WriteFile(root, "shared/resources.xml", """ + + + +"""); + + var loaded = LoadTree(root); + + // 登记在 Include 文件里：ID 带该文件路径前缀，入口里的短名引用被改写过去。 + loaded.Find("Mod:Icon")!.Value.Should().Be("shared/resources/icon"); + MetadataValidator.FindMissingReferences(loaded).Should().BeEmpty(); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void FindMissingReferences_CaseOnlyDifference_Resolves() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "metadata.xml", """ + + + + + + RES/ICON + + +"""); + WriteFile(root, "res.xml", """ + + + +"""); + + var loaded = LoadTree(root); + + MetadataValidator.FindMissingReferences(loaded).Should().BeEmpty(); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void EnsureReferences_MissingIcons_ReportsBothInvalidReferencesAndThrows() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "metadata.xml", """ + + + ghost-icon + + + ghost-app-icon + + +"""); + + var loaded = LoadTree(root); + + var missing = MetadataValidator.FindMissingReferences(loaded); + missing.Should().HaveCount(2); + missing.Should().Contain(item => item.Contains("ghost-app-icon", StringComparison.Ordinal)); + missing.Should().Contain(item => item.Contains("ghost-icon", StringComparison.Ordinal)); + + var act = () => MetadataValidator.EnsureReferences(loaded); + act.Should().Throw(); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void FindMissingReferences_MultipleBrokenFields_ReportsEachWithEntityId() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "metadata.xml", """ + + + ghost-icon + + + ghost-manifest + + + + + ghost-head + + ghost-content + + + + + + +"""); + + var loaded = LoadTree(root); + + MetadataValidator.FindMissingReferences(loaded).Should().Equal( + "Mod (Demo): Icon -> ghost-icon", + "Mod (Demo): Manifest -> ghost-manifest", + "Mod (Demo): Post HeadImage -> ghost-head", + "Mod (Demo): Post Content -> ghost-content", + "Mod (Demo): Logo -> ghost-logo", + "Mod (Demo): Background Image -> ghost-background"); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void EnsureReferences_HeadImageReferencingMarkdown_IsReported() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "metadata.xml", """ + + + + + + news + + + news + + + + +"""); + WriteFile(root, "news/news.xml", """ + + + +"""); + + var loaded = LoadTree(root); + + // Icon 查登记全集，可以引 Markdown；HeadImage 只认 Image 登记，同一个 ID 必须报错。 + loaded.Find("Mod:Icon")!.Value.Should().Be("news/news/news"); + loaded.Find("Mod:Posts:Post:HeadImage")!.Value.Should().Be("news/news/news"); + + MetadataValidator.FindMissingReferences(loaded) + .Should().ContainSingle(item => item.Contains("news/news/news", StringComparison.Ordinal)); + + var act = () => MetadataValidator.EnsureReferences(loaded); + act.Should().Throw(); + } + finally + { + DeleteTemp(root); + } + } + + [TestMethod] + public void EnsureReferences_PackageManifestWithoutRegistration_IsReported() + { + var root = CreateTempDir(); + try + { + WriteFile(root, "metadata.xml", """ + + + + + ghost-manifest + + + + +"""); + + var loaded = LoadTree(root); + + MetadataValidator.FindMissingReferences(loaded) + .Should().Equal("Mod (Demo): Manifest -> ghost-manifest"); + + var act = () => MetadataValidator.EnsureReferences(loaded); + var ex = act.Should().Throw().Which; + ex.Message.Should().Contain("Manifest").And.Contain("ghost-manifest"); + } + finally + { + DeleteTemp(root); + } + } +} diff --git a/Ra3.BattleNet.Metadata.Tests/SourceLoadTests.cs b/Ra3.BattleNet.Metadata.Tests/SourceLoadTests.cs deleted file mode 100644 index 207e479..0000000 --- a/Ra3.BattleNet.Metadata.Tests/SourceLoadTests.cs +++ /dev/null @@ -1,553 +0,0 @@ -using FluentAssertions; -using Microsoft.VisualStudio.TestTools.UnitTesting; - -namespace Ra3.BattleNet.Metadata.Tests; - -/// -/// LoadSource：本地作者源文件在内存里展平 Include、合并继承、限定短 ID，不写盘、不替换 ENV。 -/// -[TestClass] -public class SourceLoadTests -{ - private static string CreateTempDir() - { - var dir = Path.Combine(Path.GetTempPath(), $"source-load-{Guid.NewGuid():N}"); - Directory.CreateDirectory(dir); - return dir; - } - - private static void WriteFile(string root, string relative, string content) - { - var path = Path.Combine(root, relative.Replace('/', Path.DirectorySeparatorChar)); - Directory.CreateDirectory(Path.GetDirectoryName(path)!); - File.WriteAllText(path, content); - } - - private static void WriteBytes(string root, string relative, byte[] content) - { - var path = Path.Combine(root, relative.Replace('/', Path.DirectorySeparatorChar)); - Directory.CreateDirectory(Path.GetDirectoryName(path)!); - File.WriteAllBytes(path, content); - } - - private static void DeleteTemp(string dir) - { - if (Directory.Exists(dir)) - Directory.Delete(dir, recursive: true); - } - - private static byte[] TinyPng() => - Convert.FromBase64String("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=="); - - [TestMethod] - public void LoadSource_ExpandsIncludesInheritanceAndShortIds() - { - var root = CreateTempDir(); - try - { - WriteFile(root, "mod.xml", """ - - - - - - - - 2.0 - icon - 演示 - Demo - - -"""); - WriteFile(root, "base.xml", """ - - - - - - -"""); - WriteBytes(root, "images/icon.png", TinyPng()); - - var loaded = MetadataBuilder.LoadSource(Path.Combine(root, "mod.xml")); - - loaded.GetAllElements("Include").Should().BeEmpty(); - loaded.GetAllElements("Includes").Should().BeEmpty(); - loaded.GetAllElements("Base").Should().BeEmpty(); - - var mod = loaded.Mods().Single(); - mod.Id.Should().Be("Demo"); - mod.Raw.Get("InheritFrom").Should().BeNull(); - mod.Icon.Should().Be("mod:icon"); - mod.Raw.Find("Style:Controls:PrimaryLabel:FontSize")!.Value.Should().Be("14"); - - var image = loaded.Images().Single(i => i.Id == "mod:icon"); - image.Source.Should().Be("images/icon.png"); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void LoadSource_CoronaStyleDirectory_ResolvesNestedResourcesAndManifestStub() - { - var root = CreateTempDir(); - try - { - var corona = Path.Combine(root, "corona"); - WriteFile(corona, "corona.xml", """ - - - - - - - - - 3.258 - icon - 日冕 - Corona - - - manifest-3258 - - - - - - 更新 - Update - - head - - news - news - - - - - -"""); - WriteFile(corona, "manifests/manifests.xml", """ - - - - - - -"""); - WriteFile(corona, "manifests/3.258.xml", """ - - - - - corona.lyi - / - MOD; - - - -"""); - WriteFile(corona, "posts/posts.xml", """ - - - - - -"""); - WriteBytes(corona, "images/icon.png", TinyPng()); - WriteBytes(corona, "images/head.jpg", new byte[] { 0xFF, 0xD8, 0xFF, 0xD9 }); - WriteFile(corona, "posts/news.md", "# news"); - - var loaded = MetadataBuilder.LoadSource(Path.Combine(corona, "corona.xml")); - - var mod = loaded.Mods().Single(); - mod.Icon.Should().Be("corona:icon"); - mod.Raw.Find("Posts:Post:HeadImage")!.Value.Should().Be("posts/posts:head"); - mod.Raw.Find("Posts:Post:Contents:Content")!.Value.Should().Be("posts/posts:news"); - - loaded.Images().Single(i => i.Id == "corona:icon").Source.Should().Be("images/icon.png"); - loaded.Images().Single(i => i.Id == "posts/posts:head").Source.Should().Be("images/head.jpg"); - loaded.Markdowns().Single(m => m.Id == "posts/posts:news").Source.Should().Be("posts/news.md"); - - // Include 的叶子清单只以登记 stub 形式留在大树里，Source 相对入口目录。 - loaded.ManifestRegistration("manifests/3.258:manifest-3258").Should().NotBeNull(); - mod.ManifestSource().Should().Be("manifests/3.258.xml"); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void LoadSource_ManualLegacyStub_DiffersByPrefixButSharesLocalId() - { - var root = CreateTempDir(); - try - { - var modDir = Path.Combine(root, "mod"); - WriteFile(modDir, "mod.xml", """ - - - - 1.0 - 旧写法 - Legacy - - - manifest-1 - - - - - -"""); - WriteFile(modDir, "manifests/1.0.xml", """ - - - - - legacy.lyi - / - MOD; - - - -"""); - - var loaded = MetadataBuilder.LoadSource(Path.Combine(modDir, "mod.xml")); - - var stub = loaded.GetAllElements("Manifest") - .Single(m => m.Get("Source") == "manifests/1.0.xml"); - stub.Get("ID").Should().Be("mod:manifest-1"); - loaded.Mods().Single().ManifestSource().Should().Be("manifests/1.0.xml"); - - // 叶子按同一源根加载时前缀来自文件路径，与手写 stub 的入口前缀不同；LocalId 相同。 - var leaf = MetadataBuilder.LoadSource( - Path.Combine(modDir, "manifests", "1.0.xml"), modDir); - var leafId = leaf.GetAllElements("Manifest").Single().Get("ID")!; - leafId.Should().Be("manifests/1.0:manifest-1"); - leafId.Should().NotBe(stub.Get("ID")); - MetadataFlattener.LocalId(leafId).Should().Be(MetadataFlattener.LocalId(stub.Get("ID"))); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void LoadSource_GeneratedSourceInclude_StubAndLeafIdsMatchExactly() - { - var root = CreateTempDir(); - try - { - var modDir = Path.Combine(root, "mod"); - WriteFile(modDir, "mod.xml", """ - - - - - - - 1.0 - 演示 - Demo - - - manifest-1 - - - - -"""); - WriteFile(modDir, "manifests/1.0.xml", """ - - - - - generated.lyi - / - MOD; - - - -"""); - - var loaded = MetadataBuilder.LoadSource(Path.Combine(modDir, "mod.xml")); - - var stub = loaded.GetAllElements("Manifest") - .Single(m => m.Get("Source") == "manifests/1.0.xml"); - stub.Get("ID").Should().Be("manifests/1.0:manifest-1"); - loaded.Mods().Single().ManifestSource().Should().Be("manifests/1.0.xml"); - - var leaf = MetadataBuilder.LoadSource( - Path.Combine(modDir, "manifests", "1.0.xml"), modDir); - leaf.GetAllElements("Manifest").Single().Get("ID").Should().Be(stub.Get("ID")); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void Parser_LoadXDocument_ParsesWrappedSavedManifestInMemory() - { - var savedManifest = new System.Xml.Linq.XElement("Manifest", - new System.Xml.Linq.XAttribute("ID", "manifest-3258"), - new System.Xml.Linq.XElement("File", - new System.Xml.Linq.XAttribute("Hash", "3507CD54"), - new System.Xml.Linq.XElement("FileName", "corona.lyi"))); - var document = new System.Xml.Linq.XDocument( - new System.Xml.Linq.XElement("Metadata", new System.Xml.Linq.XElement(savedManifest))); - - var loaded = MetadataParser.Load(document); - - loaded.Name.Should().Be("Metadata"); - var manifest = loaded.Find("Manifest")!; - manifest.Get("ID").Should().Be("manifest-3258"); - manifest.Find("File:FileName")!.Value.Should().Be("corona.lyi"); - } - - [TestMethod] - public void Parser_LoadXDocument_RejectsSourceTreeAndNonMetadataRoot() - { - var withInclude = System.Xml.Linq.XDocument.Parse( - ""); - var includeAct = () => MetadataParser.Load(withInclude); - includeAct.Should().Throw().WithMessage("*Include*"); - - var badRoot = new System.Xml.Linq.XDocument(new System.Xml.Linq.XElement("Manifest")); - var rootAct = () => MetadataParser.Load(badRoot); - rootAct.Should().Throw().WithMessage("*Metadata*"); - } - - [TestMethod] - public void LoadSource_CyclicIncludes_Throws() - { - var root = CreateTempDir(); - try - { - WriteFile(root, "entry.xml", """ - - - - - -"""); - WriteFile(root, "a.xml", """ - - - - - -"""); - WriteFile(root, "b.xml", """ - - - - - -"""); - - var act = () => MetadataBuilder.LoadSource(Path.Combine(root, "entry.xml")); - - act.Should().Throw().WithMessage("*循环引用*"); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void LoadSource_IncludeOutsideSourceRoot_RejectedBeforeReadingTarget() - { - var root = CreateTempDir(); - try - { - var source = Path.Combine(root, "source"); - WriteFile(source, "entry.xml", """ - - - - - -"""); - // 越界目标即使是无效 XML，也必须先被路径收敛拒绝，而不是读进来再报根节点错误。 - WriteFile(root, "outside.xml", ""); - - var act = () => MetadataBuilder.LoadSource(Path.Combine(source, "entry.xml")); - - act.Should().Throw().WithMessage("*Include 越出源目录*"); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void LoadSource_ResourceOutsideSourceRoot_RejectedBeforeProbingTarget() - { - var root = CreateTempDir(); - try - { - var source = Path.Combine(root, "source"); - WriteFile(source, "entry.xml", """ - - - -"""); - WriteBytes(root, "outside.png", TinyPng()); - - var act = () => MetadataBuilder.LoadSource(Path.Combine(source, "entry.xml")); - - act.Should().Throw().WithMessage("*资源 Source 越出源目录*"); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void LoadSource_DoesNotWriteOrMutateAuthorFiles() - { - var root = CreateTempDir(); - try - { - var mod = Path.Combine(root, "mod"); - WriteFile(mod, "mod.xml", """ - - - - ${TIMESTAMP} - ${ENV:SOURCE_LOAD_TEST_MISSING} - - - - - - - 1.0 - icon - 演示 - Demo - - -"""); - WriteFile(mod, "posts/posts.xml", """ - - - - -"""); - WriteFile(mod, "posts/news.md", "# news"); - WriteBytes(mod, "images/icon.png", TinyPng()); - - var before = Snapshot(mod); - var loaded = MetadataBuilder.LoadSource(Path.Combine(mod, "mod.xml")); - var after = Snapshot(mod); - - // 作者文件原样保留，也没有生成任何新文件。 - after.Keys.Should().BeEquivalentTo(before.Keys); - foreach (var (relative, bytes) in before) - after[relative].Should().Equal(bytes); - - // 构建期变量不在本地读取时解析。 - loaded.Find("Tags:Commit")!.Value.Should().Be("${TIMESTAMP}"); - loaded.Find("Tags:Flavor")!.Value.Should().Be("${ENV:SOURCE_LOAD_TEST_MISSING}"); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void LoadSource_DoesNotRequireOtherModsReferencedByCompatibility() - { - var root = CreateTempDir(); - try - { - WriteFile(root, "mod.xml", """ - - - - 1.0 - 语音包 - Voice Pack - -
- - - -"""); - - var loaded = MetadataBuilder.LoadSource(Path.Combine(root, "mod.xml")); - - var mod = loaded.Mods().Single(); - mod.Id.Should().Be("AronaEva"); - mod.Raw.Find("Compatibility:Main")!.Get("Mod").Should().Be("OtherMod"); - } - finally - { - DeleteTemp(root); - } - } - - [TestMethod] - public void Load_StillRejectsSourceTree_WhileLoadSourceAcceptsIt() - { - var root = CreateTempDir(); - try - { - WriteFile(root, "entry.xml", """ - - - - - -"""); - WriteFile(root, "mod.xml", """ - - - 演示 - Demo - - -"""); - - var entry = Path.Combine(root, "entry.xml"); - MetadataBuilder.LoadSource(entry).Mods().Should().ContainSingle(m => m.Id == "Demo"); - - var act = () => Metadata.LoadFromFile(entry); - act.Should().Throw().WithMessage("*Include*"); - } - finally - { - DeleteTemp(root); - } - } - - private static SortedDictionary Snapshot(string root) - { - var snapshot = new SortedDictionary(StringComparer.Ordinal); - foreach (var file in Directory.EnumerateFiles(root, "*", SearchOption.AllDirectories)) - snapshot[Path.GetRelativePath(root, file).Replace('\\', '/')] = File.ReadAllBytes(file); - return snapshot; - } -} diff --git a/Ra3.BattleNet.Metadata/Building/ImagePostProcessor.cs b/Ra3.BattleNet.Metadata/Building/ImagePostProcessor.cs new file mode 100644 index 0000000..74a4e5b --- /dev/null +++ b/Ra3.BattleNet.Metadata/Building/ImagePostProcessor.cs @@ -0,0 +1,79 @@ +using System.Xml.Linq; + +namespace Ra3.BattleNet.Metadata; + +public static class ImagePostProcessor +{ + private static readonly HashSet Convertible = new(StringComparer.OrdinalIgnoreCase) + { + ".png", ".jpg", ".jpeg", ".bmp", ".gif", ".tiff", ".tif" + }; + + public static int ApplyWebP(string outputDir, TextWriter? log = null) => + ApplyWebP(outputDir, ["metadata.xml"], log); + + internal static int ApplyWebP(string outputDir, IEnumerable documentPaths, TextWriter? log = null) + { + ArgumentNullException.ThrowIfNull(documentPaths); + log ??= Console.Out; + outputDir = Path.GetFullPath(outputDir); + var documents = documentPaths.Select(path => Path.GetFullPath(Path.Combine(outputDir, path))) + .Distinct(StringComparer.OrdinalIgnoreCase).ToDictionary(path => path, path => + { + EnsureInside(path, outputDir); + return XDocument.Load(path); + }, StringComparer.OrdinalIgnoreCase); + var converted = new Dictionary(StringComparer.OrdinalIgnoreCase); + var destinations = new Dictionary(StringComparer.OrdinalIgnoreCase); + var registrations = new List<(XAttribute Attribute, string Document, string Source)>(); + foreach (var (path, document) in documents) + { + foreach (var image in document.Descendants("Image")) + { + if (image.Attribute("ID") is null || !string.IsNullOrWhiteSpace((string?)image.Attribute("Url")) + || image.Attribute("Source") is not { } source || string.IsNullOrWhiteSpace(source.Value)) + continue; + var uri = MetadataResourceUri.Resolve(path, source.Value); + if (!uri.IsFile) + continue; + var absolute = Path.GetFullPath(uri.LocalPath); + EnsureInside(absolute, outputDir); + if (!File.Exists(absolute) || !Convertible.Contains(Path.GetExtension(absolute))) + continue; + var target = Path.ChangeExtension(absolute, ".webp"); + if (destinations.TryGetValue(target, out var previous) && !StringComparer.OrdinalIgnoreCase.Equals(previous, absolute) + || File.Exists(target)) + throw new InvalidOperationException($"WebP output path collision: {target}"); + destinations[target] = absolute; + converted.TryAdd(absolute, target); + registrations.Add((source, path, absolute)); + } + } + foreach (var (source, target) in converted) + { + log.WriteLine($" convert {Path.GetRelativePath(outputDir, source)} -> {Path.GetRelativePath(outputDir, target)}"); + ImagingInvoker.ConvertToWebP(source, target); + } + foreach (var (attribute, document, source) in registrations) + attribute.Value = Path.GetRelativePath(Path.GetDirectoryName(document)!, converted[source]).Replace('\\', '/'); + foreach (var (path, document) in documents) + document.Save(path); + foreach (var source in converted.Keys) + File.Delete(source); + return converted.Count; + } + + private static void EnsureInside(string path, string root) + { + var relative = Path.GetRelativePath(root, path); + if (Path.IsPathRooted(relative) || relative == ".." + || relative.StartsWith(".." + Path.DirectorySeparatorChar, StringComparison.Ordinal)) + throw new InvalidOperationException($"Image output escaped root: {path}"); + for (var current = path; current is not null; current = Path.GetDirectoryName(current)) + { + if ((File.Exists(current) || Directory.Exists(current)) + && (File.GetAttributes(current) & FileAttributes.ReparsePoint) != 0) + throw new InvalidOperationException($"Image paths must not traverse links: {current}"); + } + } +} diff --git a/Ra3.BattleNet.Metadata/ImagingInvoker.cs b/Ra3.BattleNet.Metadata/Building/ImagingInvoker.cs similarity index 100% rename from Ra3.BattleNet.Metadata/ImagingInvoker.cs rename to Ra3.BattleNet.Metadata/Building/ImagingInvoker.cs diff --git a/Ra3.BattleNet.Metadata/MarkdownValidator.cs b/Ra3.BattleNet.Metadata/Building/MarkdownValidator.cs similarity index 100% rename from Ra3.BattleNet.Metadata/MarkdownValidator.cs rename to Ra3.BattleNet.Metadata/Building/MarkdownValidator.cs diff --git a/Ra3.BattleNet.Metadata/MetadataBuilder.cs b/Ra3.BattleNet.Metadata/Building/MetadataBuilder.cs similarity index 59% rename from Ra3.BattleNet.Metadata/MetadataBuilder.cs rename to Ra3.BattleNet.Metadata/Building/MetadataBuilder.cs index c8b3309..5ed38ee 100644 --- a/Ra3.BattleNet.Metadata/MetadataBuilder.cs +++ b/Ra3.BattleNet.Metadata/Building/MetadataBuilder.cs @@ -1,5 +1,3 @@ -using System.Net; -using System.Net.Sockets; using System.Text.RegularExpressions; using System.Xml.Linq; @@ -12,8 +10,6 @@ public static class MetadataBuilder { public const string DefaultSchemaVersion = "1.0"; - /// LoadSource 的内部修订号：源文件没有发布修订，标记为本地读取，不代表任何构建产物。 - private const string LocalSourceContentRevision = "local-source"; private static readonly Regex LeftoverVariablePattern = new(@"\$\{[^}]+\}", RegexOptions.Compiled); private static readonly Regex MountTokenPattern = new(@"^[A-Za-z0-9_-]+$", RegexOptions.Compiled); @@ -39,14 +35,12 @@ public static class MetadataBuilder }; /// - /// 从本地源目录执行核心构建。 + /// 从本地源目录执行核心构建。SchemaVersion 由作者写在源根 metadata.xml 上，构建不写任何版本或修订标记。 /// /// true 时在展平后调用 Imaging CLI 转 WebP，并由本管线改写 Source/Hash。 public static void Build( string sourceDir, string outputDir, - string? schemaVersion = null, - string? contentRevision = null, bool convertImages = false) { if (string.IsNullOrWhiteSpace(sourceDir)) @@ -59,6 +53,7 @@ public static void Build( var entry = Path.Combine(src, "metadata.xml"); if (!File.Exists(entry)) throw new FileNotFoundException($"找不到入口文件: {entry}"); + MetadataExporter.ValidateOutput(src, dst); if (Directory.Exists(dst)) Directory.Delete(dst, recursive: true); @@ -71,33 +66,35 @@ public static void Build( SchemaValidator.EnsureDirectoryValid(src, sourceSchema, "源树"); ValidateManifests(src); - var revision = string.IsNullOrWhiteSpace(contentRevision) - ? DateTime.UtcNow.ToString("yyyy-MM-ddTHH:mm:ssZ") - : contentRevision; - var version = string.IsNullOrWhiteSpace(schemaVersion) ? DefaultSchemaVersion : schemaVersion; - - var flattened = MetadataFlattener.Flatten(entry, src, version, revision); + var input = MetadataReader.Read(entry, new MetadataSourceOptions + { + SourceRoot = src + }); + if (string.IsNullOrWhiteSpace(input.Document.Root!.Attribute("SchemaVersion")?.Value)) + throw new InvalidOperationException("源根 metadata.xml 的根节点缺少 SchemaVersion 属性"); + ModCompatibility.Validate(input.Document.Root!); var flatPath = Path.Combine(dst, "metadata.xml"); + var xmlPaths = MetadataExporter.Export(input, dst); var resolver = new VariableResolver(); - - // 发布面 = 展平 metadata.xml + 被引用资源 + _redirects + edgeone.json + editor.html； - // 源 XML / XSD / Templates 不进 Output - var leafPaths = CopyReferencedResources(flattened, src, dst, version, revision, resolver); CopyIfExists(Path.Combine(src, "_redirects"), Path.Combine(dst, "_redirects")); CopyIfExists(Path.Combine(src, "edgeone.json"), Path.Combine(dst, "edgeone.json")); CopyIfExists(Path.Combine(src, "editor.html"), Path.Combine(dst, "editor.html")); - flattened.Save(flatPath); - resolver.ReplaceInFile(flatPath); + foreach (var path in xmlPaths) + { + var document = XDocument.Load(path); + resolver.ReplaceInElement(document.Root!); + document.Save(path); + } - var leftover = FindLeftoverVariables([flatPath, .. leafPaths]); + var leftover = FindLeftoverVariables(xmlPaths); if (leftover.Count > 0) throw new InvalidOperationException("变量替换后仍有残留: " + string.Join("; ", leftover)); // 展平后：按图调用 Imaging（只产 webp），本管线写回 XML if (convertImages) { - var n = ImagePostProcessor.ApplyWebP(dst); + var n = ImagePostProcessor.ApplyWebP(dst, xmlPaths.Select(path => Path.GetRelativePath(dst, path))); Console.WriteLine($" Imaging: 转换 {n} 张图片并写回 Source"); } @@ -118,84 +115,6 @@ public static void Build( } } - /// - /// 从本地作者源文件加载：入口可以带 Includes、Base/InheritFrom 与短 ID， - /// 在内存里完成展平与继承合并后返回普通 树。 - /// - /// 入口 XML，例如某个 Mod 自己的 corona.xml 或独立叶子清单。 - /// - /// Include 与相对 Source 的允许根目录；缺省为入口文件所在目录。 - /// 与构建一致：ID 前缀是文件相对此根、去 .xml 的路径。 - /// - /// - /// 只读：不写任何文件，不解析 ${ENV:...} 等构建期变量，不做 XSD 校验。 - /// 单模组读取没有整棵目录，因此不执行跨实体 Compatibility 引用校验；Build 仍然校验。 - /// 资源 Source 保持相对 的路径，可直接交给 - /// （基准应为该源根；缺省时就是入口文件所在目录）。 - /// 入口、Include 与资源路径越出 时在读取目标前抛出 - /// 。需要发布物的严格入口请用 。 - /// - public static Metadata LoadSource(string entryPath, string? sourceRoot = null) - { - var root = string.IsNullOrWhiteSpace(sourceRoot) - ? Path.GetDirectoryName(Path.GetFullPath(entryPath))! - : sourceRoot; - - return MetadataParser.Load(MetadataFlattener.Flatten( - entryPath, root, MetadataSchema.Current, LocalSourceContentRevision, validateCompatibility: false)); - } - - /// - /// 从展平树收集被引用资源（Image/Markdown/Manifest 登记节点的 Source 文件；Url-only 图跳过），复制到 Output。 - /// 叶子清单不照抄源文件：它同样走一遍展平并解析变量（去掉源树专用的 Includes、限定 ID、替换 ${TIMESTAMP}）， - /// 否则发布物里会留下客户端解析器拒绝的 Includes 元素与未替换的变量。 - /// 返回写出的叶子清单路径。缺失文件不在此报错——由 ValidateHard 统一硬失败。 - /// - private static List CopyReferencedResources( - XDocument flattened, string src, string dst, string schemaVersion, string contentRevision, VariableResolver resolver) - { - var copied = new HashSet(StringComparer.OrdinalIgnoreCase); - var leafPaths = new List(); - - foreach (var el in flattened.Root!.DescendantsAndSelf()) - { - if (el.Name.LocalName is not ("Image" or "Markdown" or "Manifest")) - continue; - if (string.IsNullOrWhiteSpace(el.Attribute("ID")?.Value)) - continue; // 登记节点才有资源 - if (!string.IsNullOrWhiteSpace(el.Attribute("Url")?.Value)) - continue; // 外链不落盘 - var source = el.Attribute("Source")?.Value; - if (string.IsNullOrWhiteSpace(source)) - continue; - - var rel = source.Replace('\\', '/'); - if (!copied.Add(rel)) - continue; - - var from = Path.GetFullPath(Path.Combine(src, rel)); - if (!File.Exists(from)) - continue; // 缺文件由 ValidateHard 报 - - var target = Path.Combine(dst, rel); - var dir = Path.GetDirectoryName(target); - if (!string.IsNullOrEmpty(dir)) - Directory.CreateDirectory(dir); - - if (el.Name.LocalName == "Manifest") - { - MetadataFlattener.Flatten(from, src, schemaVersion, contentRevision).Save(target); - resolver.ReplaceInFile(target); - leafPaths.Add(target); - } - else - { - File.Copy(from, target, overwrite: true); - } - } - - return leafPaths; - } private static void CopyIfExists(string from, string to) { @@ -206,163 +125,6 @@ private static void CopyIfExists(string from, string to) File.Copy(from, to, overwrite: true); } - /// - /// 从本地路径（含 file:// 形式）或 HTTP(S) URL 加载已展平的 metadata.xml。 - /// - public static Metadata Load(string pathOrUrl) - { - if (string.IsNullOrWhiteSpace(pathOrUrl)) - throw new ArgumentException("路径或 URL 不能为空", nameof(pathOrUrl)); - - if (Uri.TryCreate(pathOrUrl, UriKind.Absolute, out var uri)) - { - if (uri.Scheme == Uri.UriSchemeHttp || uri.Scheme == Uri.UriSchemeHttps) - { - return LoadFromUrl(uri); - } - - // MetadataResourceUri.Resolve 对本地入口返回 file:// 地址，这里还原成本地路径 - if (uri.IsFile) - { - return Metadata.LoadFromFile(uri.LocalPath); - } - } - - return Metadata.LoadFromFile(pathOrUrl); - } - - /// 单个 IP 的连接预算：这么久还没连上就换下一个，不在一棵树上吊死。 - private static readonly TimeSpan AddressConnectTimeout = TimeSpan.FromSeconds(8); - - /// 一次尝试的总预算（含握手与正文）。 - private static readonly TimeSpan LoadUrlTimeout = TimeSpan.FromSeconds(60); - - /// URL 入口的总尝试次数（含第一次）。 - private const int LoadUrlAttempts = 8; - - /// 两次尝试之间歇多久。握手失败几十毫秒就报出来，用不着指数退避。 - private static readonly TimeSpan LoadUrlRetryWait = TimeSpan.FromMilliseconds(600); - - /// 连接地址的轮换计数，跨尝试累加，用来错开每次尝试的起点。 - private static int _addressCursor; - - /// - /// 从 URL 加载已展平的 metadata.xml。CDN 边缘节点经常握手失败或压根不可达， - /// 所以最多试 次，每次换一个边缘 IP 的起点； - /// 全部失败才把最后一次的异常抛出去。 - /// - private static Metadata LoadFromUrl(Uri uri) - { - for (var attempt = 1; ; attempt++) - { - try - { - return FetchFromUrl(uri); - } - catch (Exception ex) when (IsRetryableLoadFailure(ex) && attempt < LoadUrlAttempts) - { - Thread.Sleep(LoadUrlRetryWait); - } - } - } - - private static Metadata FetchFromUrl(Uri uri) - { - using var handler = CreateLoadHandler(); - using var client = new HttpClient(handler) { Timeout = LoadUrlTimeout }; - using var stream = client.GetStreamAsync(uri).GetAwaiter().GetResult(); - var temp = Path.Combine(Path.GetTempPath(), $"metadata-url-{Guid.NewGuid():N}.xml"); - try - { - using (var fs = File.Create(temp)) - stream.CopyTo(fs); - return Metadata.LoadFromFile(temp); - } - finally - { - try { File.Delete(temp); } catch { /* ignore */ } - } - } - - /// - /// 什么才算"值得再试一次"：网络层的失败和超时都算。除了 和 - /// ，还有两种得拆开看的 —— HttpClient 的超时是 - /// （里头裹着一个 ）； - /// 响应体读到一半被重置则是 （"响应提前结束"），它也是 - /// ，一块算上。本地落盘失败同样是 IOException，重试它没有意义， - /// 但最多白烧几次几毫秒的失败，比漏掉一次真网络抖动划算。取消和 XML 解析错误一律不重试 —— - /// 那不是网络在抖。 - /// - private static bool IsRetryableLoadFailure(Exception ex) => - ex is HttpRequestException or IOException or TimeoutException - or TaskCanceledException { InnerException: TimeoutException }; - - /// - /// 自己接管连接过程的 handler。 - /// - /// 为什么不直接用 默认那套：它只在 TCP 连不上时才换地址，握手阶段被 - /// RST/EOF 会直接把异常抛出来，不会改连同域名的其它 IP。而 CDN 一个域名背后是一组边缘 IP， - /// 可达性并不一致（实测同一个域名的两个 IP，一个通、一个 12 秒超时），不自己接管的话 - /// 每次重试都撞上同一个坏 IP，重试再多也白试。顺手把 IPv4 排到 IPv6 前面：国内不少网络 - /// 有 IPv6 地址但没有 IPv6 路由，先试 IPv6 等于白等一轮超时。 - /// - private static SocketsHttpHandler CreateLoadHandler() => new() - { - ConnectCallback = async (context, cancellationToken) => - { - var addresses = await Dns.GetHostAddressesAsync(context.DnsEndPoint.Host, cancellationToken); - return await ConnectAsync(OrderAddresses(addresses), context.DnsEndPoint.Port, cancellationToken); - }, - }; - - /// - /// IPv4 一律排在 IPv6 前面；轮换只在 IPv4 这一组内做。 - /// 国内不少网络有 IPv6 地址却没有 IPv6 路由，先试 IPv6 等于白等一轮超时，所以 IPv6 只能垫底。 - /// 而 IPv4 这一组内部要换着起点来，否则每次尝试都先撞同一个边缘 IP，重试再多也白试。 - /// - private static IReadOnlyList OrderAddresses(IPAddress[] addresses) - { - var ipv4 = addresses.Where(a => a.AddressFamily == AddressFamily.InterNetwork).ToArray(); - var others = addresses.Where(a => a.AddressFamily != AddressFamily.InterNetwork).ToArray(); - - if (ipv4.Length == 0) - { - return others; - } - - var offset = Interlocked.Increment(ref _addressCursor) % ipv4.Length; - return ipv4.Skip(offset).Concat(ipv4.Take(offset)).Concat(others).ToArray(); - } - - /// 按顺序连各个地址，单个地址连这么久还没动静就换下一个；全都不通抛最后一个异常。 - private static async Task ConnectAsync(IReadOnlyList addresses, int port, CancellationToken cancellationToken) - { - if (addresses.Count == 0) - { - throw new HttpRequestException("DNS 没有返回任何地址"); - } - - Exception? failure = null; - - foreach (var address in addresses) - { - var socket = new Socket(SocketType.Stream, ProtocolType.Tcp); - try - { - using var budget = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); - budget.CancelAfter(AddressConnectTimeout); - await socket.ConnectAsync(address, port, budget.Token); - return new NetworkStream(socket, ownsSocket: true); - } - catch (Exception ex) - { - socket.Dispose(); - failure = ex; - } - } - - throw failure!; - } private static List FindLeftoverVariables(IEnumerable xmlPaths) { @@ -374,21 +136,12 @@ private static List FindLeftoverVariables(IEnumerable xmlPaths) private static void ValidateHard(string flatPath, string outputDir) { - var metadata = Metadata.LoadFromFile(flatPath); + var metadata = MetadataReader.Read(flatPath).Root; var errors = new List(); - // 仅「登记节点」（带 ID 属性）进入索引；Style 内 id 引用不算登记。 - // Logo / Background Image 只认图片登记，不认 Markdown / Manifest 等同名 ID。 - var idIndex = new HashSet(StringComparer.OrdinalIgnoreCase); var registryImages = metadata.GetAllElements("Image").Where(n => !string.IsNullOrWhiteSpace(n.Get("ID"))).ToList(); var registryMarkdowns = metadata.GetAllElements("Markdown").Where(n => !string.IsNullOrWhiteSpace(n.Get("ID"))).ToList(); var registryManifests = metadata.GetAllElements("Manifest").Where(n => !string.IsNullOrWhiteSpace(n.Get("ID"))).ToList(); - var imageIds = new HashSet(StringComparer.OrdinalIgnoreCase); - - foreach (var node in registryImages.Concat(registryMarkdowns).Concat(registryManifests)) - idIndex.Add(node.Get("ID")!); - foreach (var image in registryImages) - imageIds.Add(image.Get("ID")!); foreach (var md in registryMarkdowns) { @@ -433,55 +186,9 @@ private static void ValidateHard(string flatPath, string outputDir) } } - foreach (var app in metadata.GetAllElements("Application").Concat(metadata.GetAllElements("Mod"))) - { - ValidateEntityDisplayNames(app, errors); - ValidateIdRef(app.Find("Icon")?.Value, "Icon", idIndex, errors); - var packages = app.Find("Packages"); - if (packages != null) - { - foreach (var package in packages.Children.Where(c => c.Name == "Package")) - { - ValidateIdRef(package.Find("Manifest")?.Value, "Manifest", idIndex, errors); - } - } - - var posts = app.Find("Posts"); - if (posts != null) - { - foreach (var post in posts.Children.Where(c => c.Name == "Post")) - { - // Post 封面只认图片登记：引用 Markdown / Manifest 或不存在都报错。 - ValidateIdRef(post.Find("HeadImage")?.Value?.Trim(), "Post HeadImage", imageIds, errors); - - var contents = post.Find("Contents"); - if (contents == null) continue; - foreach (var content in contents.Children.Where(c => c.Name == "Content")) - ValidateIdRef(content.Value, "Post Content", idIndex, errors); - } - } - - var style = app.Find("Style"); - if (style == null) continue; - var logo = style.Find("Logo"); - if (logo?.Value != null) - ValidateIdRef(logo.Value.Trim(), "Logo", imageIds, errors); - var background = style.Find("Background"); - if (background == null) continue; - foreach (var img in background.Children.Where(c => c.Name == "Image")) - ValidateIdRef(img.Value?.Trim(), "Background Image", imageIds, errors); - } - - if (string.IsNullOrWhiteSpace(metadata.Get("SchemaVersion")) - && metadata.Find("SchemaVersion") == null) - { - // 属性写在根上 - if (!metadata.Variables.ContainsKey("SchemaVersion")) - errors.Add("缺少 SchemaVersion"); - } - - if (!metadata.Variables.ContainsKey("ContentRevision") && metadata.Find("ContentRevision") == null) - errors.Add("缺少 ContentRevision"); + errors.AddRange(MetadataValidator.FindMissingReferences(metadata)); + foreach (var entity in metadata.GetAllElements("Application").Concat(metadata.GetAllElements("Mod"))) + ValidateEntityDisplayNames(entity, errors); if (errors.Count > 0) throw new InvalidOperationException("核心构建校验失败:\n- " + string.Join("\n- ", errors)); @@ -518,12 +225,6 @@ private static void ValidateEntityDisplayNames(Metadata entity, List err errors.Add($"{name}: 缺少 en DisplayName"); } - private static void ValidateIdRef(string? id, string kind, HashSet idIndex, List errors) - { - if (string.IsNullOrWhiteSpace(id)) return; - if (!idIndex.Contains(id)) - errors.Add($"{kind} 引用未找到 ID: {id}"); - } /// /// 源树 Manifest 语义硬校验：仅当清单声明新格式（带 HashAlgorithm 属性，或任一 File 含 Sources）时执行； diff --git a/Ra3.BattleNet.Metadata/MetadataInheritance.cs b/Ra3.BattleNet.Metadata/Building/MetadataInheritance.cs similarity index 100% rename from Ra3.BattleNet.Metadata/MetadataInheritance.cs rename to Ra3.BattleNet.Metadata/Building/MetadataInheritance.cs diff --git a/Ra3.BattleNet.Metadata/SchemaValidator.cs b/Ra3.BattleNet.Metadata/Building/SchemaValidator.cs similarity index 100% rename from Ra3.BattleNet.Metadata/SchemaValidator.cs rename to Ra3.BattleNet.Metadata/Building/SchemaValidator.cs diff --git a/Ra3.BattleNet.Metadata/VariableResolver.cs b/Ra3.BattleNet.Metadata/Building/VariableResolver.cs similarity index 100% rename from Ra3.BattleNet.Metadata/VariableResolver.cs rename to Ra3.BattleNet.Metadata/Building/VariableResolver.cs diff --git a/Ra3.BattleNet.Metadata/Cache/HttpMetadataTransfer.cs b/Ra3.BattleNet.Metadata/Cache/HttpMetadataTransfer.cs index fff37dc..5dad614 100644 --- a/Ra3.BattleNet.Metadata/Cache/HttpMetadataTransfer.cs +++ b/Ra3.BattleNet.Metadata/Cache/HttpMetadataTransfer.cs @@ -18,7 +18,7 @@ internal sealed class HttpMetadataTransfer : IMetadataTransfer public HttpMetadataTransfer(HttpClient? httpClient, TimeSpan requestTimeout) { _ownsHttp = httpClient is null; - _http = httpClient ?? new HttpClient { Timeout = requestTimeout }; + _http = httpClient ?? new HttpClient(new SocketsHttpHandler { AllowAutoRedirect = false }) { Timeout = requestTimeout }; } public async Task FetchAsync(MetadataTransferRequest request, CancellationToken cancellationToken) @@ -39,6 +39,9 @@ public async Task FetchAsync(MetadataTransferRequest req .SendAsync(message, HttpCompletionOption.ResponseHeadersRead, cancellationToken) .ConfigureAwait(false); + if (response.RequestMessage?.RequestUri is { } finalUri && finalUri != request.Source) + throw new HttpRequestException("Metadata transfer cannot accept redirected responses"); + if (response.StatusCode != HttpStatusCode.OK) { return new MetadataTransferResult( diff --git a/Ra3.BattleNet.Metadata/Cache/LeafResult.cs b/Ra3.BattleNet.Metadata/Cache/LeafResult.cs index f012db4..1aba9aa 100644 --- a/Ra3.BattleNet.Metadata/Cache/LeafResult.cs +++ b/Ra3.BattleNet.Metadata/Cache/LeafResult.cs @@ -15,7 +15,7 @@ public sealed record LeafResult( Metadata? Document, Metadata? ManifestNode, ManifestEntry? Entry, - Uri SourceUri, + Uri? SourceUri, MetadataFreshness Status, string? Error); diff --git a/Ra3.BattleNet.Metadata/Cache/MetadataClient.cs b/Ra3.BattleNet.Metadata/Cache/MetadataClient.cs index 4992c5c..7f05543 100644 --- a/Ra3.BattleNet.Metadata/Cache/MetadataClient.cs +++ b/Ra3.BattleNet.Metadata/Cache/MetadataClient.cs @@ -3,6 +3,7 @@ using System.Security.Cryptography; using System.Text; using System.Xml; +using System.Xml.Linq; using Ra3.BattleNet.Metadata; namespace Ra3.BattleNet.Metadata.Cache; @@ -10,7 +11,6 @@ namespace Ra3.BattleNet.Metadata.Cache; /// /// 根清单、包叶子与登记图片的共享加载器。叶子与图片身份只来自调用方传入的快照。 /// 相同 Source 的正文只拉一次，每个调用再按自己的 Manifest ID 投影。 -/// 只要本地有缓存，读取展示数据就不走网络；没有缓存（冷启动）或明确调用 GetLeafAsync 时才会请求网络。 /// public sealed class MetadataClient : IDisposable { @@ -124,36 +124,43 @@ public Task RefreshRootAsync(string metadataUrl, Cancellatio return Share(_refreshFlights, origin.AbsoluteUri, token => RefreshRootCore(origin, token), ct); } - /// 取共享叶子文档，再按这份快照里的版本登记投影 Manifest。 - public async Task GetLeafAsync(RootSnapshotResult snapshot, string version, Uri leafSourceUri, CancellationToken ct = default) - { - ct.ThrowIfCancellationRequested(); - if (!TrySelect(snapshot, version, leafSourceUri, out var leaf, out var manifestId, out var error)) - return LeafUnavailable(leaf, error); - return Project(await EnsureLeafAsync(leaf, force: false, ct).ConfigureAwait(false), manifestId); - } + public Task GetPackageAsync(RootSnapshotResult snapshot, PackageEntry package, CancellationToken ct = default) => + ReadPackageAsync(snapshot, package, cachedOnly: false, force: false, ct); - /// 强制重新读取该 Source。身份只看传入的快照，不改去追以后的根。 - public async Task RefreshLeafAsync(RootSnapshotResult snapshot, string version, Uri leafSourceUri, CancellationToken ct = default) - { - ct.ThrowIfCancellationRequested(); - if (!TrySelect(snapshot, version, leafSourceUri, out var leaf, out var manifestId, out var error)) - return LeafUnavailable(leaf, error); - return Project(await EnsureLeafAsync(leaf, force: true, ct).ConfigureAwait(false), manifestId); - } + public Task GetCachedPackageAsync(RootSnapshotResult snapshot, PackageEntry package, CancellationToken ct = default) => + ReadPackageAsync(snapshot, package, cachedOnly: true, force: false, ct); - /// - /// 只从内存或磁盘读取已有的叶子清单，绝对不发 HTTP 请求，也不等待正在进行的网络任务。 - /// 如果本地没有缓存直接返回 Unavailable；关闭磁盘缓存时，若是本地 file:// 协议则直接读取源文件。 - /// - public Task GetCachedLeafAsync(RootSnapshotResult snapshot, string version, Uri leafSourceUri, CancellationToken ct = default) + public Task RefreshPackageAsync(RootSnapshotResult snapshot, PackageEntry package, CancellationToken ct = default) => + ReadPackageAsync(snapshot, package, cachedOnly: false, force: true, ct); + + private async Task ReadPackageAsync(RootSnapshotResult snapshot, PackageEntry package, bool cachedOnly, bool force, CancellationToken ct) { + ArgumentNullException.ThrowIfNull(snapshot); + ArgumentNullException.ThrowIfNull(package); ct.ThrowIfCancellationRequested(); - if (!TrySelect(snapshot, version, leafSourceUri, out var leaf, out var manifestId, out var error)) - return Task.FromResult(LeafUnavailable(leaf, error)); - if (TryMemoryLeaf(leaf) is { } memory && (_disk is not null || !leaf.IsFile)) - return Task.FromResult(ProjectCached(memory, manifestId)); - return LoadCachedLeafAsync(leaf, manifestId, ct); + ObjectDisposedException.ThrowIf(_disposed, this); + if (snapshot.Document is null || snapshot.OriginUri is null + || !EnumeratePackages(snapshot.Document).Any(candidate => ReferenceEquals(candidate.Raw, package.Raw))) + return LeafUnavailable(null, "Package does not belong to this snapshot"); + var registration = snapshot.Document.ManifestRegistration(package.ManifestId ?? ""); + if (registration is null) + return LeafUnavailable(null, "Package has no registered manifest"); + if (!IsStub(registration)) + { + if (!TryReadLeaf(snapshot.Document, package.ManifestId!, out var node, out var entry, out var error)) + return LeafUnavailable(null, error ?? "Invalid inline manifest"); + return new LeafResult(snapshot.Document, node, entry, null, snapshot.Status, snapshot.Error); + } + if (!TryResolvePackage(snapshot.Document, snapshot.OriginUri, package, out var canonical, out var manifestId)) + return LeafUnavailable(null, "Package has no valid manifest Source"); + var source = new Uri(canonical); + if (cachedOnly) + { + if (TryMemoryLeaf(source, SourceRoot(snapshot)) is { } memory && (_disk is not null || !source.IsFile)) + return ProjectCached(memory, manifestId); + return await LoadCachedLeafAsync(source, SourceRoot(snapshot), manifestId, ct).ConfigureAwait(false); + } + return Project(await EnsureLeafAsync(source, SourceRoot(snapshot), force, ct).ConfigureAwait(false), manifestId); } /// @@ -168,8 +175,17 @@ public async Task PreloadLeavesAsync(RootSnapshotResult root ct.ThrowIfCancellationRequested(); ObjectDisposedException.ThrowIf(_disposed, this); var leaves = EnumerateLeaves(rootSnapshot); + var inline = rootSnapshot.Document is null ? new List() : EnumeratePackages(rootSnapshot.Document) + .Where(package => rootSnapshot.Document.ManifestRegistration(package.ManifestId ?? "") is { } manifest && !IsStub(manifest)) + .DistinctBy(package => package.ManifestId, StringComparer.OrdinalIgnoreCase).ToList(); + var inlineFailed = 0; + foreach (var package in inline) + { + if (!TryReadLeaf(rootSnapshot.Document!, package.ManifestId!, out _, out _, out _)) + inlineFailed++; + } if (leaves.Count == 0) - return LeafPreloadSummary.Empty; + return new LeafPreloadSummary(inline.Count, inline.Count - inlineFailed, inlineFailed); var waits = new List>(leaves.Count); foreach (var leaf in leaves) @@ -178,7 +194,8 @@ public async Task PreloadLeavesAsync(RootSnapshotResult root Task flight; try { - flight = StartShared(_leafFlights, leaf.AbsoluteUri, token => FetchLeafDocumentAsync(leaf, token)); + var sourceRoot = SourceRoot(rootSnapshot); + flight = StartShared(_leafFlights, LeafKey(leaf, sourceRoot), token => FetchLeafDocumentAsync(leaf, sourceRoot, token)); } catch { @@ -196,8 +213,9 @@ public async Task PreloadLeavesAsync(RootSnapshotResult root } var docs = await Task.WhenAll(waits).ConfigureAwait(false); - var failed = docs.Count(doc => doc.Document is null); - return new LeafPreloadSummary(docs.Length, docs.Length - failed, failed); + var failed = docs.Count(doc => doc.Document is null) + inlineFailed; + var total = docs.Length + inline.Count; + return new LeafPreloadSummary(total, total - failed, failed); } /// @@ -336,9 +354,9 @@ public void Dispose() _transfer.Dispose(); } - private Task EnsureLeafAsync(Uri leaf, bool force, CancellationToken ct) + private Task EnsureLeafAsync(Uri leaf, string sourceRoot, bool force, CancellationToken ct) { - var key = leaf.AbsoluteUri; + var key = LeafKey(leaf, sourceRoot); if (!force) { lock (_gate) @@ -350,23 +368,23 @@ private Task EnsureLeafAsync(Uri leaf, bool force, CancellationToken ct return Task.FromResult(memory); } - return Share(_leafReads, key, token => ReadOrFetchLeafAsync(leaf, token), ct); + return Share(_leafReads, key, token => ReadOrFetchLeafAsync(leaf, sourceRoot, token), ct); } - return Share(_leafFlights, key, token => FetchLeafDocumentAsync(leaf, token), ct); + return Share(_leafFlights, key, token => FetchLeafDocumentAsync(leaf, sourceRoot, token), ct); } - private async Task ReadOrFetchLeafAsync(Uri leaf, CancellationToken ct) + private async Task ReadOrFetchLeafAsync(Uri leaf, string sourceRoot, CancellationToken ct) { - var disk = _disk?.TryReadLeaf(leaf); + var disk = _disk?.TryReadSnapshot(leaf, sourceRoot); if (disk is not null) { - var loaded = StoreParsedLeaf(leaf, disk.Bytes, disk.Digest, MetadataFreshness.Stale, error: null); + var loaded = StoreParsedLeaf(leaf, sourceRoot, disk.Bytes, disk.Digest, MetadataFreshness.Stale, error: null); if (loaded is not null) return loaded; } - return await Share(_leafFlights, leaf.AbsoluteUri, token => FetchLeafDocumentAsync(leaf, token), CancellationToken.None) + return await Share(_leafFlights, LeafKey(leaf, sourceRoot), token => FetchLeafDocumentAsync(leaf, sourceRoot, token), CancellationToken.None) .WaitAsync(ct) .ConfigureAwait(false); } @@ -388,11 +406,11 @@ private static LeafResult Project(LeafDoc doc, string manifestId) doc.Status == MetadataFreshness.Fresh ? null : doc.Error); } - private LeafDoc? TryMemoryLeaf(Uri leaf) + private LeafDoc? TryMemoryLeaf(Uri leaf, string sourceRoot) { lock (_gate) { - return _leaves.TryGetValue(leaf.AbsoluteUri, out var memory) && memory.Document is not null + return _leaves.TryGetValue(LeafKey(leaf, sourceRoot), out var memory) && memory.Document is not null ? memory : null; } @@ -406,135 +424,231 @@ private static LeafResult ProjectCached(LeafDoc doc, string manifestId) : projected; } - private async Task LoadCachedLeafAsync(Uri leaf, string manifestId, CancellationToken ct) + private async Task LoadCachedLeafAsync(Uri leaf, string sourceRoot, string manifestId, CancellationToken ct) { ObjectDisposedException.ThrowIf(_disposed, this); if (_disk is null) { if (!leaf.IsFile) return LeafUnavailable(leaf, "本地没有可用的叶子清单缓存"); - var live = await ReadLiveFileLeafAsync(leaf, ct).ConfigureAwait(false); + var live = await ReadLiveFileLeafAsync(leaf, sourceRoot, ct).ConfigureAwait(false); return live is null ? LeafUnavailable(leaf, "本地没有可用的叶子清单缓存") : ProjectCached(live, manifestId); } - var disk = _disk.TryReadLeaf(leaf); + var disk = _disk.TryReadSnapshot(leaf, sourceRoot); if (disk is null) return LeafUnavailable(leaf, "本地没有可用的叶子清单缓存"); - var loaded = RememberCachedLeaf(leaf, disk.Bytes, disk.Digest); + var loaded = StoreParsedLeaf(leaf, sourceRoot, disk.Bytes, disk.Digest, MetadataFreshness.Stale, error: null); return loaded is null ? LeafUnavailable(leaf, "本地叶子清单缓存无法解析") : ProjectCached(loaded, manifestId); } - private async Task ReadLiveFileLeafAsync(Uri leaf, CancellationToken ct) - { - var transfer = await ReadFileAsync(leaf, ct).ConfigureAwait(false); - if (!transfer.Ok || transfer.Bytes is null) - return null; - return ParseLeafDocument(leaf, transfer.Bytes, MetadataDiskCache.Sha256Hex(transfer.Bytes)); - } - - private LeafDoc? RememberCachedLeaf(Uri leaf, byte[] bytes, string digest) - { - if (TryMemoryLeaf(leaf) is { } existing) - return existing; - - var parsed = ParseLeafDocument(leaf, bytes, digest); - if (parsed is null) - return null; - - lock (_gate) - { - if (_leaves.TryGetValue(leaf.AbsoluteUri, out var current) && current.Document is not null) - return current; - _leaves[leaf.AbsoluteUri] = parsed; - return parsed; - } - } - - private static LeafDoc? ParseLeafDocument(Uri leaf, byte[] bytes, string digest) + private Task ReadLiveFileLeafAsync(Uri leaf, string sourceRoot, CancellationToken ct) { + ct.ThrowIfCancellationRequested(); try { - var document = ParseBytes(bytes); - if (!TryAcceptLeaf(document, out _)) - return null; - return new LeafDoc + var source = new OfflineSource(new CapturedSource(leaf, [])); + var read = MetadataReader.Read(leaf.AbsoluteUri, new MetadataSourceOptions { SourceRoot = sourceRoot, CustomSource = source }); + if (!TryAcceptLeaf(read.Root, out _)) + return Task.FromResult(null); + return Task.FromResult(new LeafDoc { Source = leaf, - Document = document, - Digest = digest, + Document = read.Root, Status = MetadataFreshness.Stale, - Error = null, - }; + }); } - catch (Exception ex) when (ex is XmlException or InvalidOperationException or IOException) + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or XmlException + or InvalidOperationException or ArgumentException) { - return null; + return Task.FromResult(null); } } + private Task OpenCore(Uri origin, CancellationToken ct) { ct.ThrowIfCancellationRequested(); _disk?.BindPrimary(origin); - var disk = _disk?.TryReadRoot(origin); - if (disk is null) - return Task.FromResult(RootUnavailable(origin, "本地没有可用的根清单缓存")); - - var memory = FindRoot(origin, disk.Digest) ?? ParseRootBytes(origin, disk.Bytes, disk.Digest); - if (memory is null) - return Task.FromResult(RootUnavailable(origin, "本地根清单缓存无法解析")); - return Task.FromResult(RootOf(memory, MetadataFreshness.Stale, error: null)); + if (_disk?.HasForeignOwner(origin) == true) + return Task.FromResult(RootUnavailable(origin, "Cache URI ownership mismatch")); + lock (_gate) + { + if (_rootDocs.TryGetValue(origin.AbsoluteUri, out var remembered)) + return Task.FromResult(RootOf(remembered, MetadataFreshness.Stale, error: null)); + } + var memory = ReadCompleteRoot(origin); + return Task.FromResult(memory is null + ? RootUnavailable(origin, "No complete cached metadata") + : RootOf(memory, MetadataFreshness.Stale, error: null)); } private async Task RefreshRootCore(Uri origin, CancellationToken ct) { _disk?.BindPrimary(origin); + Dictionary? captured = null; + try + { + var (read, documents) = await CaptureGraphAsync(origin, ct).ConfigureAwait(false); + captured = documents; + var catalogError = CatalogError(read.Root); + if (catalogError is not null) + return RootFallback(origin, catalogError); + var digest = GraphDigest(read); + await CommitGraphAsync(origin, read, documents, digest, ct).ConfigureAwait(false); + return RootOf(RememberRoot(origin, read.Root, digest), MetadataFreshness.Fresh, error: null); + + } + catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) + { + throw; + } + catch (Exception ex) when (ex is HttpRequestException or IOException or UnauthorizedAccessException + or XmlException or InvalidOperationException or ArgumentException) + { + return RootFallback(origin, ex.Message); + } + finally + { + DiscardGraph(captured); + } + } + + private async Task<(MetadataReadResult Read, Dictionary Documents)> CaptureGraphAsync(Uri origin, CancellationToken ct, string? sourceRoot = null) + { + var documents = new Dictionary(StringComparer.Ordinal); + var source = new CapturedSource(origin, documents); try { - var disk = _disk?.TryReadRoot(origin); - var transfer = await TransferAsync(origin, disk is { Verified: true } ? disk.Stamp : null, ct).ConfigureAwait(false); + sourceRoot ??= source.GetDirectoryName(origin.AbsoluteUri)!; + await CaptureAsync(origin, sourceRoot, ct).ConfigureAwait(false); + return (MetadataReader.Read(origin.AbsoluteUri, new MetadataSourceOptions { SourceRoot = sourceRoot, CustomSource = source }), documents); + } + catch + { + DiscardGraph(documents); + throw; + } + + async Task CaptureAsync(Uri uri, string boundary, CancellationToken token) + { + if (documents.ContainsKey(uri.AbsoluteUri)) + return; + var disk = _disk?.TryReadRoot(uri); + var transfer = await TransferAsync(uri, disk is { Verified: true } ? disk.Stamp : null, token).ConfigureAwait(false); + var retained = false; try { + XDocument? xml = null; if (transfer.NotModified) { - var again = _disk?.TryReadRoot(origin); - var memory = again is { Verified: true } - ? FindRoot(origin, again.Digest) ?? ParseRootBytes(origin, again.Bytes, again.Digest) - : null; - if (memory is null) - { - // 304 不带正文，直接换成无条件重取。 - transfer = await TransferAsync(origin, validators: null, ct).ConfigureAwait(false); - if (transfer.NotModified) - return RootFallback(origin, "HTTP 304 但本地根清单无法解析"); - } - else - { - return RootOf(memory, MetadataFreshness.Fresh, error: null); - } + disk = _disk?.TryReadRoot(uri); + if (disk is not { Verified: true } || (xml = TryReadXml(disk.Bytes)) is null) + transfer = await TransferAsync(uri, validators: null, token).ConfigureAwait(false); + } + var body = transfer.NotModified ? disk?.Bytes : transfer.Bytes; + if (!transfer.Ok || (body is null && transfer.Path is null)) + throw new IOException(transfer.Error ?? "Missing XML response body"); + if (transfer.Path is { } staged) + { + using var stream = File.OpenRead(staged); + xml = ReadXml(stream); + } + else + xml ??= ReadXml(body!); + var stamp = transfer.NotModified ? disk!.Stamp! : new CacheStamp + { + Uri = uri.AbsoluteUri, + Sha256 = transfer.Path is { } path ? MetadataDiskCache.Sha256FileHex(path) : MetadataDiskCache.Sha256Hex(body!), + ETag = transfer.ETag, + LastModified = transfer.LastModified, + }; + documents.Add(uri.AbsoluteUri, new CapturedXml(uri, body, transfer.Path, xml, stamp)); + retained = true; + foreach (var include in xml.Descendants().Where(element => element.Name.LocalName == "Include")) + { + var target = MetadataReader.ResolveInclude(source, uri.AbsoluteUri, + include.Attribute("Source")?.Value ?? "", boundary); + var child = MetadataDiskCache.CanonicalizeEntry(target); + await CaptureAsync(child, boundary, token).ConfigureAwait(false); } - - if (!transfer.Ok || (transfer.Bytes is null && transfer.Path is null)) - return RootFallback(origin, transfer.Error ?? "刷新根清单失败"); - return await PublishRootAsync(origin, transfer, ct).ConfigureAwait(false); } finally { - transfer.DiscardStagedFile(); + if (!retained) + transfer.DiscardStagedFile(); } } - catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) - { - throw; - } - catch (Exception ex) when (ex is HttpRequestException or IOException or UnauthorizedAccessException) + } + + private async Task CommitGraphAsync(Uri origin, MetadataReadResult read, Dictionary documents, string digest, CancellationToken ct) + { + if (_disk is null) + return; + _disk.ValidateGraphPaths(documents.Values.Select(document => document.Uri)); + foreach (var captured in documents.Values) { - return RootFallback(origin, ex.Message); + var path = _disk.RootBodyPath(captured.Uri); + try + { + if (captured.Path is { } staged) + _disk.StageFile(path, captured.Uri.AbsoluteUri, staged); + else + await _disk.StageAsync(path, captured.Uri.AbsoluteUri, captured.Bytes!, ct).ConfigureAwait(false); + _disk.Commit(path, captured.Stamp); + } + catch + { + _disk.Discard(path); + throw; + } } + _disk.CommitComposition(origin, read.SourceRoot, new XDocument(new XElement("Composition", + new XAttribute("Origin", origin.AbsoluteUri), new XAttribute("SourceRoot", read.SourceRoot), new XAttribute("Sha256", digest), + documents.Values.Select(item => new XElement("Document", new XAttribute("Uri", item.Uri.AbsoluteUri), + new XAttribute("Sha256", item.Stamp.Sha256))), + new XElement("Root", new XElement(read.Document.Root!))))); + } + + private static string GraphDigest(MetadataReadResult read) => + MetadataDiskCache.Sha256Hex(Encoding.UTF8.GetBytes(read.Document.Root!.ToString(SaveOptions.DisableFormatting))); + + private static void DiscardGraph(Dictionary? documents) + { + if (documents is null) + return; + foreach (var captured in documents.Values) + if (captured.Path is { } path) + TryDeleteFile(path); + } + + private static XDocument ReadXml(byte[] bytes) + { + using var stream = new MemoryStream(bytes, writable: false); + return ReadXml(stream); + } + + internal static XDocument ReadXml(Stream stream) + { + using var reader = XmlReader.Create(stream, new XmlReaderSettings { DtdProcessing = DtdProcessing.Prohibit, XmlResolver = null }); + return XDocument.Load(reader); + } + + private static XDocument? TryReadXml(byte[] bytes) + { + try { return ReadXml(bytes); } + catch (XmlException) { return null; } + } + + private RootMemory? ReadCompleteRoot(Uri origin) + { + var snapshot = _disk?.TryReadSnapshot(origin, new Uri(origin, ".").AbsoluteUri); + return snapshot is null ? null + : FindRoot(origin, snapshot.Digest) ?? ParseRootBytes(origin, snapshot.Bytes, snapshot.Digest); } private async Task FetchImageAsync(Uri uri, CancellationToken ct) @@ -923,152 +1037,85 @@ private static bool TryImageContentType(Uri uri, string? header, out string cont _ => null, }; - private async Task FetchLeafDocumentAsync(Uri leaf, CancellationToken ct) + private async Task FetchLeafDocumentAsync(Uri leaf, string sourceRoot, CancellationToken ct) { - var key = leaf.AbsoluteUri; + Dictionary? captured = null; try { - var disk = _disk?.TryReadLeaf(leaf); - var transfer = await TransferAsync(leaf, disk is { Verified: true } ? disk.Stamp : null, ct).ConfigureAwait(false); - try - { - if (transfer.NotModified) - { - var again = _disk?.TryReadLeaf(leaf); - var reused = again is { Verified: true } - ? FindLeaf(key, again.Digest) ?? StoreParsedLeaf(leaf, again.Bytes, again.Digest, MetadataFreshness.Fresh, null) - : null; - if (reused is null) - { - // 304 不带正文，直接换成无条件重取。 - transfer = await TransferAsync(leaf, validators: null, ct).ConfigureAwait(false); - if (transfer.NotModified) - return LeafDocFallback(leaf, "HTTP 304 但本地叶子清单无法解析"); - } - else - { - lock (_gate) - { - reused.Status = MetadataFreshness.Fresh; - reused.Error = null; - } - - return reused; - } - } - - if (!transfer.Ok || (transfer.Bytes is null && transfer.Path is null)) - return LeafDocFallback(leaf, transfer.Error ?? "读取叶子清单失败"); - return await PublishLeafDocumentAsync(leaf, transfer, ct).ConfigureAwait(false); - } - finally - { - transfer.DiscardStagedFile(); - } + var (read, documents) = await CaptureGraphAsync(leaf, ct, sourceRoot).ConfigureAwait(false); + captured = documents; + if (!TryAcceptLeaf(read.Root, out var error)) + return LeafDocFallback(leaf, sourceRoot, error ?? "Invalid manifest document"); + var digest = GraphDigest(read); + await CommitGraphAsync(leaf, read, documents, digest, ct).ConfigureAwait(false); + return StoreLeafDocument(leaf, sourceRoot, read.Root, digest, MetadataFreshness.Fresh, error: null); } catch (OperationCanceledException) when (_lifetime.IsCancellationRequested) { throw; } - catch (Exception ex) when (ex is HttpRequestException or IOException or UnauthorizedAccessException) + catch (Exception ex) when (ex is HttpRequestException or IOException or UnauthorizedAccessException + or XmlException or InvalidOperationException or ArgumentException) + { + return LeafDocFallback(leaf, sourceRoot, ex.Message); + } + finally { - return LeafDocFallback(leaf, ex.Message); + DiscardGraph(captured); } } - private async Task PublishRootAsync(Uri origin, Transfer transfer, CancellationToken ct) + private sealed record CapturedXml(Uri Uri, byte[]? Bytes, string? Path, XDocument Document, CacheStamp Stamp); + + private sealed class CapturedSource(Uri entry, Dictionary documents) : IMetadataXmlSource { - string? stagedPath = transfer.Path; - Metadata document; - try + public bool Exists(string path) => documents.ContainsKey(GetFullPath(path)); + public XDocument LoadDocument(string path) => documents[GetFullPath(path)].Document; + public string Combine(string basePath, string relativePath) => + new Uri(DirectoryUri(basePath), relativePath.Replace('\\', '/')).AbsoluteUri; + public string? GetDirectoryName(string path) => new Uri(new Uri(GetFullPath(path)), ".").AbsoluteUri; + public string GetRelativePath(string fromPath, string toPath) { - document = stagedPath is not null ? ParseFile(stagedPath) : ParseBytes(transfer.Bytes!); + var from = DirectoryUri(fromPath); + var to = new Uri(GetFullPath(toPath)); + if (from.IsFile && to.IsFile) + return Path.GetRelativePath(from.LocalPath, to.LocalPath).Replace('\\', '/'); + if (from.Scheme != to.Scheme || from.Authority != to.Authority) + return to.AbsoluteUri; + return from.MakeRelativeUri(to).ToString(); } - catch (Exception ex) when (ex is not OperationCanceledException) + public string GetFullPath(string path) => new Uri(entry, path.Replace('\\', '/')).AbsoluteUri; + public bool IsPathRooted(string path) => Uri.TryCreate(path, UriKind.Absolute, out _) || Path.IsPathRooted(path); + private Uri DirectoryUri(string path) { - return RootFallback(origin, "根清单解析失败：" + ex.Message); - } - - var catalogError = CatalogError(document); - if (catalogError is not null) - return RootFallback(origin, catalogError); - - var digest = stagedPath is not null - ? MetadataDiskCache.Sha256FileHex(stagedPath) - : MetadataDiskCache.Sha256Hex(transfer.Bytes!); - if (_disk is not null) - { - var bodyPath = _disk.RootBodyPath(origin); - try - { - if (stagedPath is not null) - _disk.StageFile(bodyPath, origin.AbsoluteUri, stagedPath); - else - await _disk.StageAsync(bodyPath, origin.AbsoluteUri, transfer.Bytes!, ct).ConfigureAwait(false); - _disk.Commit(bodyPath, new CacheStamp - { - Uri = origin.AbsoluteUri, - ETag = transfer.ETag, - LastModified = transfer.LastModified, - Sha256 = digest, - }); - } - catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) - { - _disk.Discard(bodyPath); - return RootFallback(origin, "根清单写入缓存失败：" + ex.Message); - } + var full = GetFullPath(path); + return new Uri(full.EndsWith('/') ? full : full + "/"); } - - return RootOf(RememberRoot(origin, document, digest), MetadataFreshness.Fresh, error: null); } - private async Task PublishLeafDocumentAsync(Uri leaf, Transfer transfer, CancellationToken ct) + private sealed class OfflineSource(IMetadataXmlSource paths) : IMetadataXmlSource { - string? stagedPath = transfer.Path; - Metadata document; - try + public bool Exists(string path) { - document = stagedPath is not null ? ParseFile(stagedPath) : ParseBytes(transfer.Bytes!); + var uri = MetadataDiskCache.CanonicalizeEntry(paths.GetFullPath(path)); + return uri.IsFile && File.Exists(uri.LocalPath); } - catch (Exception ex) when (ex is not OperationCanceledException) + public XDocument LoadDocument(string path) { - return LeafDocFallback(leaf, "叶子清单解析失败：" + ex.Message); + var uri = MetadataDiskCache.CanonicalizeEntry(paths.GetFullPath(path)); + if (!uri.IsFile) + throw new IOException("Offline source only reads local XML documents"); + using var stream = File.OpenRead(uri.LocalPath); + return ReadXml(stream); } - - if (!TryAcceptLeaf(document, out var reject)) - return LeafDocFallback(leaf, reject ?? "叶子清单无效"); - - var digest = stagedPath is not null - ? MetadataDiskCache.Sha256FileHex(stagedPath) - : MetadataDiskCache.Sha256Hex(transfer.Bytes!); - if (_disk is not null) - { - var bodyPath = _disk.LeafBodyPath(leaf); - try - { - if (stagedPath is not null) - _disk.StageFile(bodyPath, leaf.AbsoluteUri, stagedPath); - else - await _disk.StageAsync(bodyPath, leaf.AbsoluteUri, transfer.Bytes!, ct).ConfigureAwait(false); - _disk.Commit(bodyPath, new CacheStamp - { - Uri = leaf.AbsoluteUri, - ETag = transfer.ETag, - LastModified = transfer.LastModified, - Sha256 = digest, - }); - } - catch (Exception ex) when (ex is IOException or UnauthorizedAccessException) - { - _disk.Discard(bodyPath); - return LeafDocFallback(leaf, "叶子清单写入缓存失败：" + ex.Message); - } - } - - return StoreLeafDocument(leaf, document, digest, MetadataFreshness.Fresh, error: null); + public string Combine(string basePath, string relativePath) => paths.Combine(basePath, relativePath); + public string? GetDirectoryName(string path) => paths.GetDirectoryName(path); + public string GetRelativePath(string fromPath, string toPath) => paths.GetRelativePath(fromPath, toPath); + public string GetFullPath(string path) => paths.GetFullPath(path); + public bool IsPathRooted(string path) => paths.IsPathRooted(path); } + private async Task TransferAsync(Uri uri, CacheStamp? validators, CancellationToken ct) { if (uri.IsFile) @@ -1172,13 +1219,11 @@ private async Task ReadFileAsync(Uri uri, CancellationToken ct) private RootSnapshotResult RootFallback(Uri origin, string error) { - var disk = _disk?.TryReadRoot(origin); - if (disk is not null) - { - var parsed = FindRoot(origin, disk.Digest) ?? ParseRootBytes(origin, disk.Bytes, disk.Digest); - if (parsed is not null) - return RootOf(parsed, MetadataFreshness.Stale, error); - } + if (_disk?.HasForeignOwner(origin) == true) + return RootUnavailable(origin, error); + var parsed = ReadCompleteRoot(origin); + if (parsed is not null) + return RootOf(parsed, MetadataFreshness.Stale, error); lock (_gate) { @@ -1189,19 +1234,19 @@ private RootSnapshotResult RootFallback(Uri origin, string error) return RootUnavailable(origin, error); } - private LeafDoc LeafDocFallback(Uri leaf, string error) + private LeafDoc LeafDocFallback(Uri leaf, string sourceRoot, string error) { - var disk = _disk?.TryReadLeaf(leaf); + var disk = _disk?.TryReadSnapshot(leaf, sourceRoot); if (disk is not null) { - var loaded = StoreParsedLeaf(leaf, disk.Bytes, disk.Digest, MetadataFreshness.Stale, error); + var loaded = StoreParsedLeaf(leaf, sourceRoot, disk.Bytes, disk.Digest, MetadataFreshness.Stale, error); if (loaded is not null) return loaded; } lock (_gate) { - if (_leaves.TryGetValue(leaf.AbsoluteUri, out var memory)) + if (_leaves.TryGetValue(LeafKey(leaf, sourceRoot), out var memory)) { memory.Status = MetadataFreshness.Stale; memory.Error = error; @@ -1212,29 +1257,6 @@ private LeafDoc LeafDocFallback(Uri leaf, string error) return new LeafDoc { Source = leaf, Status = MetadataFreshness.Unavailable, Error = error }; } - private static bool TrySelect(RootSnapshotResult snapshot, string version, Uri leafSourceUri, out Uri leaf, out string manifestId, out string error) - { - ArgumentNullException.ThrowIfNull(snapshot); - if (string.IsNullOrWhiteSpace(version)) - throw new ArgumentException("版本不能为空", nameof(version)); - - leaf = RequireLeaf(leafSourceUri); - manifestId = ""; - if (snapshot.Document is null || snapshot.OriginUri is null) - { - error = "根快照没有可用文档"; - return false; - } - - var match = Match(snapshot.Document, snapshot.OriginUri, version, leaf.AbsoluteUri, out manifestId); - error = match switch - { - MatchKind.One => "", - MatchKind.Ambiguous => $"版本 {version} 在这份快照里登记了多份不同清单", - _ => $"版本 {version} 在这份快照里没有登记清单地址 {leaf.AbsoluteUri}", - }; - return match == MatchKind.One; - } private RootMemory? ParseRootBytes(Uri origin, byte[] bytes, string digest) { @@ -1283,9 +1305,9 @@ private RootMemory RememberRoot(Uri origin, Metadata document, string digest) } } - private LeafDoc? StoreParsedLeaf(Uri leaf, byte[] bytes, string digest, MetadataFreshness status, string? error) + private LeafDoc? StoreParsedLeaf(Uri leaf, string sourceRoot, byte[] bytes, string digest, MetadataFreshness status, string? error) { - var existing = FindLeaf(leaf.AbsoluteUri, digest); + var existing = FindLeaf(LeafKey(leaf, sourceRoot), digest); if (existing is not null) { lock (_gate) @@ -1302,7 +1324,7 @@ private RootMemory RememberRoot(Uri origin, Metadata document, string digest) var document = ParseBytes(bytes); if (!TryAcceptLeaf(document, out _)) return null; - return StoreLeafDocument(leaf, document, digest, status, error); + return StoreLeafDocument(leaf, sourceRoot, document, digest, status, error); } catch (Exception ex) when (ex is XmlException or InvalidOperationException or IOException) { @@ -1310,11 +1332,11 @@ private RootMemory RememberRoot(Uri origin, Metadata document, string digest) } } - private LeafDoc StoreLeafDocument(Uri leaf, Metadata document, string digest, MetadataFreshness status, string? error) + private LeafDoc StoreLeafDocument(Uri leaf, string sourceRoot, Metadata document, string digest, MetadataFreshness status, string? error) { lock (_gate) { - if (_leaves.TryGetValue(leaf.AbsoluteUri, out var existing) + if (_leaves.TryGetValue(LeafKey(leaf, sourceRoot), out var existing) && string.Equals(existing.Digest, digest, StringComparison.OrdinalIgnoreCase)) { existing.Status = status; @@ -1330,7 +1352,7 @@ private LeafDoc StoreLeafDocument(Uri leaf, Metadata document, string digest, Me Status = status, Error = error, }; - _leaves[leaf.AbsoluteUri] = memory; + _leaves[LeafKey(leaf, sourceRoot)] = memory; return memory; } } @@ -1346,6 +1368,10 @@ private LeafDoc StoreLeafDocument(Uri leaf, Metadata document, string digest, Me } } + private static string SourceRoot(RootSnapshotResult snapshot) => new Uri(snapshot.OriginUri!, ".").AbsoluteUri; + + private static string LeafKey(Uri leaf, string sourceRoot) => sourceRoot + "\n" + leaf.AbsoluteUri; + private Task Share(Dictionary> flights, string key, Func> work, CancellationToken callerCt) => Observe(StartShared(flights, key, work), callerCt); @@ -1427,29 +1453,6 @@ private static IEnumerable EnumeratePackages(Metadata document) } } - private static MatchKind Match(Metadata document, Uri origin, string version, string leafText, out string manifestId) - { - string? found = null; - foreach (var package in EnumeratePackages(document)) - { - if (!string.Equals(package.Version, version, StringComparison.Ordinal)) - continue; - if (!TryResolvePackage(document, origin, package, out var canonical, out var id)) - continue; - if (!string.Equals(canonical, leafText, StringComparison.Ordinal)) - continue; - if (found is not null && !string.Equals(found, id, StringComparison.OrdinalIgnoreCase)) - { - manifestId = ""; - return MatchKind.Ambiguous; - } - - found = id; - } - - manifestId = found ?? ""; - return found is null ? MatchKind.None : MatchKind.One; - } private static bool TryResolvePackage(Metadata document, Uri origin, PackageEntry package, out string canonical, out string manifestId) { @@ -1458,7 +1461,8 @@ private static bool TryResolvePackage(Metadata document, Uri origin, PackageEntr if (string.IsNullOrWhiteSpace(package.ManifestId)) return false; - var source = document.ManifestRegistration(package.ManifestId)?.Get("Source"); + var registration = document.ManifestRegistration(package.ManifestId); + var source = registration is not null && IsStub(registration) ? registration.Get("Source") : null; if (string.IsNullOrWhiteSpace(source)) return false; @@ -1476,8 +1480,8 @@ private static bool TryResolvePackage(Metadata document, Uri origin, PackageEntr private static bool TryReadLeaf(Metadata document, string expectedManifestId, out Metadata? node, out ManifestEntry? entry, out string? error) { - node = document.GetAllElements("Manifest") - .FirstOrDefault(item => string.Equals(item.Get("ID"), expectedManifestId, StringComparison.OrdinalIgnoreCase)); + var manifests = document.GetAllElements("Manifest").ToList(); + node = manifests.FirstOrDefault(item => string.Equals(item.Get("ID"), expectedManifestId, StringComparison.OrdinalIgnoreCase)); if (node is null) { entry = null; @@ -1522,12 +1526,7 @@ private static bool TryReadLeaf(Metadata document, string expectedManifestId, ou return true; } - private static bool IsStub(Metadata manifest) - { - var hasSource = !string.IsNullOrWhiteSpace(manifest.Get("Source")); - var hasBody = manifest.Children.Any(child => child.Name is "File" or "Skudef" or "Dependencies" or "SubManifest"); - return hasSource && !hasBody; - } + private static bool IsStub(Metadata manifest) => !string.IsNullOrWhiteSpace(manifest.Get("Source")); private static bool TryAcceptLeaf(Metadata document, out string? error) { @@ -1577,15 +1576,6 @@ private static Metadata ParseBytes(byte[] bytes) return MetadataParser.Load(stream); } - /// - /// 从传输方落盘的临时正文解析。与 是同一个解析入口，只是数据直接来自文件： - /// 不再把整份正文读回内存。解析与 SHA256 都针对同一个不可变文件，文档与摘要来自同一份快照。 - /// - private static Metadata ParseFile(string path) - { - using var stream = File.OpenRead(path); - return MetadataParser.Load(stream); - } private static Uri RequireEntry(string metadataUrl) { @@ -1594,13 +1584,6 @@ private static Uri RequireEntry(string metadataUrl) return MetadataDiskCache.CanonicalizeEntry(metadataUrl); } - private static Uri RequireLeaf(Uri leafSourceUri) - { - ArgumentNullException.ThrowIfNull(leafSourceUri); - if (!leafSourceUri.IsAbsoluteUri) - throw new ArgumentException("叶子地址必须是绝对地址", nameof(leafSourceUri)); - return MetadataDiskCache.Canonicalize(leafSourceUri); - } private static RootSnapshotResult RootOf(RootMemory memory, MetadataFreshness status, string? error) => new(memory.Document, memory.Catalog, memory.Origin, status, Guid.NewGuid().ToString("D"), DateTimeOffset.UtcNow, error); @@ -1608,15 +1591,9 @@ private static RootSnapshotResult RootOf(RootMemory memory, MetadataFreshness st private static RootSnapshotResult RootUnavailable(Uri origin, string error) => new(null, null, origin, MetadataFreshness.Unavailable, Guid.NewGuid().ToString("D"), DateTimeOffset.UtcNow, error); - private static LeafResult LeafUnavailable(Uri source, string error) => + private static LeafResult LeafUnavailable(Uri? source, string error) => new(null, null, null, source, MetadataFreshness.Unavailable, error); - private enum MatchKind - { - None, - One, - Ambiguous, - } private readonly record struct Transfer( bool Ok, diff --git a/Ra3.BattleNet.Metadata/Cache/MetadataDiskCache.cs b/Ra3.BattleNet.Metadata/Cache/MetadataDiskCache.cs index fb149d3..85120d8 100644 --- a/Ra3.BattleNet.Metadata/Cache/MetadataDiskCache.cs +++ b/Ra3.BattleNet.Metadata/Cache/MetadataDiskCache.cs @@ -3,15 +3,15 @@ using System.Text; using System.Text.Json; using System.Text.Json.Serialization.Metadata; +using System.Xml.Linq; namespace Ra3.BattleNet.Metadata.Cache; /// -/// 最后一份校验合法的 XML 与图片。正文按**发布文件树**落盘：第一次绑定的主发布基直接映射到缓存目录， -/// 其余发布基放在可读的 .sources 子树，带查询串的地址放进 .requests 保留区。每个正文旁的 -/// .etag 记录规范化 URI、正文 SHA256、图片响应类型与条件请求校验器。校验器缺失时正文仍可离线读取； -/// 路径归属写在一次成型的 .uri 旁挂文件里，所以同一磁盘路径上属于**另一个地址**的正文既不会被误读， -/// 也不会被覆盖。 +/// 最后一份完整规范化 XML 快照与图片。原始 XML 正文按发布文件树保存，仅用于条件重验； +/// 完整快照另存于 .compositions，不会直接把原始正文作为离线根或叶子返回。 +/// 首次绑定的发布基直接映射到缓存目录，其余发布基放在 .sources，图片查询串放在 .requests。 +/// .etag 将原始正文 SHA256 与响应校验器配对，.uri 保存路径归属，防止其他地址误读或覆盖。 /// /// /// @@ -20,6 +20,7 @@ namespace Ra3.BattleNet.Metadata.Cache; /// cacheDir/<相对 Source> 主发布基的叶子与图片（与发布文件树同名） /// cacheDir/.sources/<scheme>/<host_port>/<path> 其他发布基的正文 /// cacheDir/.requests/<请求键>/body 带查询串的图片正文（键 = 完整 URI 的 SHA256） +/// cacheDir/.compositions/<上下文键>.xml 完整规范化快照（键 = SourceRoot 与 URI 的 SHA256） /// /// 同目录还有 .etag（校验器、正文 SHA256 与图片类型）、.uri（一次成型的路径归属地址）和写入中的 /// .tmp。正文用替换发布，旧正文一直留到替换成功。只接受静态发布文件树：跳转/空路径段、 @@ -178,6 +179,36 @@ public void BindPrimary(Uri entry) public string LeafBodyPath(Uri leaf) => BodyPath(Canonicalize(leaf)); + public void ValidateGraphPaths(IEnumerable uris) + { + var comparer = OperatingSystem.IsWindows() ? StringComparer.OrdinalIgnoreCase : StringComparer.Ordinal; + var files = new HashSet(comparer); + foreach (var uri in uris) + { + if (HasForeignOwner(uri)) + throw new IOException("Cache URI ownership mismatch"); + var path = RootBodyPath(uri); + foreach (var file in new[] { path, path + ".uri", path + ".etag", path + ".tmp" }) + { + if (!files.Add(file) || Directory.Exists(file)) + throw new IOException("Captured XML cache paths collide"); + for (var parent = Path.GetDirectoryName(file); parent is not null; parent = Path.GetDirectoryName(parent)) + { + if (File.Exists(parent)) + throw new IOException("Captured XML cache path has a file ancestor"); + if (Directory.Exists(parent) && (File.GetAttributes(parent) & FileAttributes.ReparsePoint) != 0) + throw new IOException("Captured XML cache path cannot use filesystem links"); + } + if (File.Exists(file) && (File.GetAttributes(file) & FileAttributes.ReparsePoint) != 0) + throw new IOException("Captured XML cache path cannot use filesystem links"); + } + } + foreach (var file in files) + for (var parent = Path.GetDirectoryName(file); parent is not null; parent = Path.GetDirectoryName(parent)) + if (files.Contains(parent)) + throw new IOException("Captured XML cache paths overlap"); + } + /// /// 图片正文路径。相对 Source 图片与 XML 共用发布文件树；带查询串的地址按完整请求 URI 的 /// SHA-256 落到保留目录 .requests/<key>/body，不同查询串不会撞到同一份正文。 @@ -196,10 +227,80 @@ public string ImageBodyPath(Uri image) return TryReadBody(BodyPath(canonical), canonical.AbsoluteUri, requireValidators: true); } - public DiskBody? TryReadLeaf(Uri leaf) + public DiskBody? TryReadSnapshot(Uri leaf, string sourceRoot) { var canonical = Canonicalize(leaf); - return TryReadBody(BodyPath(canonical), canonical.AbsoluteUri, requireValidators: true); + if (HasForeignOwner(canonical)) + return null; + var record = TryReadComposition(canonical, sourceRoot)?.Root; + if (record?.Name != "Composition" || (string?)record.Attribute("Origin") != canonical.AbsoluteUri + || (string?)record.Attribute("SourceRoot") != sourceRoot + || !record.Elements("Document").Any() || record.Element("Root")?.Element("Metadata") is not { } root) + return null; + var bytes = Encoding.UTF8.GetBytes(root.ToString(SaveOptions.DisableFormatting)); + var digest = Sha256Hex(bytes); + return string.Equals(digest, (string?)record.Attribute("Sha256"), StringComparison.Ordinal) + ? new DiskBody(CompositionPath(canonical, sourceRoot), bytes, digest, null, false) + : null; + } + + + public bool HasForeignOwner(Uri origin) + { + var canonical = Canonicalize(origin); + var path = BodyPath(canonical); + var stamp = ReadStamp(path + ".etag"); + var (exists, owner) = ReadOwner(path + ".uri"); + return (stamp is not null && !string.IsNullOrEmpty(stamp.Uri) + && !string.Equals(stamp.Uri, canonical.AbsoluteUri, StringComparison.Ordinal)) + || (exists && !string.Equals(owner, canonical.AbsoluteUri, StringComparison.Ordinal)); + } + + public XDocument? TryReadComposition(Uri origin, string sourceRoot) + { + try + { + var path = CompositionPath(origin, sourceRoot); + if (!File.Exists(path)) + return null; + using var stream = File.OpenRead(path); + return MetadataClient.ReadXml(stream); + } + catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or System.Xml.XmlException) + { + return null; + } + } + + public void CommitComposition(Uri origin, string sourceRoot, XDocument document) + { + var path = CompositionPath(origin, sourceRoot); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + var temporary = path + "." + Guid.NewGuid().ToString("N") + ".tmp"; + try + { + File.WriteAllText(temporary, document.ToString(SaveOptions.DisableFormatting), new UTF8Encoding(false)); + if (File.Exists(path)) + File.Replace(temporary, path, destinationBackupFileName: null); + else + File.Move(temporary, path); + } + finally + { + TryDelete(temporary); + } + } + + private string CompositionPath(Uri origin, string sourceRoot) + { + var directory = Path.Combine(_root, ".compositions"); + var key = Sha256Hex(Encoding.UTF8.GetBytes(sourceRoot + "\n" + Canonicalize(origin).AbsoluteUri)); + var path = Path.Combine(directory, key + ".xml"); + foreach (var candidate in new[] { _root, directory, path, path + ".tmp" }) + if ((Directory.Exists(candidate) || File.Exists(candidate)) + && (File.GetAttributes(candidate) & FileAttributes.ReparsePoint) != 0) + throw new IOException("Composition cache cannot use filesystem links"); + return path; } /// @@ -424,6 +525,8 @@ private static void RejectPrimaryReserved(List relative) throw new ArgumentException($"相对 Source 不能占用缓存保留目录 {SourcesDirectoryName}"); if (string.Equals(first, RequestsDirectoryName, StringComparison.OrdinalIgnoreCase)) throw new ArgumentException($"相对 Source 不能占用缓存保留目录 {RequestsDirectoryName}"); + if (string.Equals(first, ".compositions", StringComparison.OrdinalIgnoreCase)) + throw new ArgumentException("Source occupies the composition cache directory"); } private string Combine(List segments) @@ -726,7 +829,7 @@ internal sealed class CacheStamp public string Sha256 { get; set; } = ""; - /// 图片响应类型。XML 正文不写这个字段，缺失时按旧缓存处理。 + /// 图片响应类型；XML 正文不写这个字段。 public string? ContentType { get; set; } } diff --git a/Ra3.BattleNet.Metadata/Exporting/MetadataExportOptions.cs b/Ra3.BattleNet.Metadata/Exporting/MetadataExportOptions.cs new file mode 100644 index 0000000..1ea9df3 --- /dev/null +++ b/Ra3.BattleNet.Metadata/Exporting/MetadataExportOptions.cs @@ -0,0 +1,17 @@ +namespace Ra3.BattleNet.Metadata; + +public enum MetadataExportLayout +{ + Flat, + Tree, + Mixed +} + +public sealed class MetadataExportOptions +{ + public MetadataExportLayout Layout { get; init; } = MetadataExportLayout.Flat; + public Func? Inline { get; init; } + public bool InlineManifests { get; init; } + public bool ConvertImages { get; init; } + public bool ResolveVariables { get; init; } +} diff --git a/Ra3.BattleNet.Metadata/Exporting/MetadataExporter.cs b/Ra3.BattleNet.Metadata/Exporting/MetadataExporter.cs new file mode 100644 index 0000000..1379cb7 --- /dev/null +++ b/Ra3.BattleNet.Metadata/Exporting/MetadataExporter.cs @@ -0,0 +1,377 @@ +using System.Xml.Linq; + +namespace Ra3.BattleNet.Metadata; + +public static class MetadataExporter +{ + public static IReadOnlyList Export(MetadataReadResult input, string outputDirectory, MetadataExportOptions? options = null) + { + ArgumentNullException.ThrowIfNull(input); + ArgumentException.ThrowIfNullOrWhiteSpace(outputDirectory); + var output = Path.GetFullPath(outputDirectory); + EnsureSeparate(input, output); + try + { + var session = new ExportSession(input, output, options ?? new MetadataExportOptions()); + session.Plan(); + ClearOutput(output); + session.Write(); + return session.XmlPaths; + } + catch + { + ClearOutput(output); + throw; + } + } + + + private static void EnsureSeparate(MetadataReadResult input, string output) => ValidateOutput(input.SourceRoot, output); + + internal static void ValidateOutput(string sourceRoot, string output) + { + output = Path.GetFullPath(output); + EnsureNoLinks(output); + if (Uri.TryCreate(sourceRoot, UriKind.Absolute, out var uri) && !uri.IsFile) + return; + var source = Path.GetFullPath(uri?.IsFile == true ? uri.LocalPath : sourceRoot); + if (IsWithin(source, output) || IsWithin(output, source)) + throw new InvalidOperationException("Export output must not overlap the input source root."); + } + + private static bool IsWithin(string path, string root) + { + var relative = Path.GetRelativePath(root, path); + return relative != ".." && !relative.StartsWith(".." + Path.DirectorySeparatorChar, StringComparison.Ordinal) + && !Path.IsPathRooted(relative); + } + + private static void EnsureNoLinks(string path) + { + for (var current = Path.GetFullPath(path); current is not null; current = Path.GetDirectoryName(current)) + { + if ((File.Exists(current) || Directory.Exists(current)) + && (File.GetAttributes(current) & FileAttributes.ReparsePoint) != 0) + throw new InvalidOperationException($"Export paths must not traverse links: {current}"); + } + } + + private static void ClearOutput(string output) + { + if (Directory.Exists(output)) + Directory.Delete(output, recursive: true); + } + + private sealed class ExportSession + { + private readonly MetadataReadResult input; + private readonly string output; + private readonly MetadataExportOptions options; + private readonly Dictionary documents; + private readonly Dictionary xml = new(StringComparer.OrdinalIgnoreCase); + private readonly Dictionary files = new(StringComparer.OrdinalIgnoreCase); + private readonly Dictionary owners = new(StringComparer.OrdinalIgnoreCase); + private readonly HashSet emitted; + private readonly Dictionary> manifestCandidates; + private readonly HashSet prepared = new(StringComparer.OrdinalIgnoreCase); + + internal ExportSession(MetadataReadResult input, string output, MetadataExportOptions options) + { + if (!Enum.IsDefined(options.Layout)) + throw new ArgumentOutOfRangeException(nameof(options)); + if (options.Layout == MetadataExportLayout.Mixed && options.Inline is null) + throw new ArgumentException("Mixed layout requires an Inline predicate.", nameof(options)); + this.input = input; + this.output = output; + this.options = options; + var comparer = MetadataReader.PathComparer; + documents = input.Documents.ToDictionary(document => document.Path, comparer); + emitted = new HashSet(comparer); + manifestCandidates = new Dictionary>(comparer); + } + internal IReadOnlyList XmlPaths => xml.Keys.Select(path => Path.Combine(output, path)).ToArray(); + + + internal void Plan() + { + var document = Compose(input.EntryPath, "metadata.xml", new HashSet(documents.Comparer)); + if (options.Layout == MetadataExportLayout.Flat) + MetadataInheritance.Resolve(document.Root!); + foreach (var attribute in input.Document.Root!.Attributes()) + document.Root!.SetAttributeValue(attribute.Name, attribute.Value); + AddXml("metadata.xml", input.EntryPath, document); + foreach (var (destination, planned) in xml.ToList()) + Prepare(planned.Root!, destination, manifestLeaf: destination != "metadata.xml" + && planned.Root!.Elements().All(element => element.Name.LocalName is "Manifest" or "Tags")); + foreach (var path in owners.Keys) + { + var parent = Path.GetDirectoryName(path); + while (!string.IsNullOrEmpty(parent)) + { + if (owners.ContainsKey(parent.Replace('\\', '/'))) + throw new InvalidOperationException($"Output file/directory collision: {path}"); + parent = Path.GetDirectoryName(parent); + } + } + } + + private XDocument Compose(string path, string destination, HashSet expanding) + { + if (!expanding.Add(path)) + throw new InvalidOperationException($"Include cycle during export: {path}"); + var info = documents[path]; + var document = new XDocument(info.NormalizedDocument); + var root = document.Root ?? throw new InvalidOperationException($"Missing Metadata root: {path}"); + foreach (var include in root.Descendants("Include").ToList()) + { + var target = (string?)include.Attribute("Source") + ?? throw new InvalidOperationException("Include has no Source."); + var child = documents[target]; + var inline = options.Layout == MetadataExportLayout.Flat + || options.Layout == MetadataExportLayout.Mixed && options.Inline!(child.RelativePath); + if (inline) + { + if (!emitted.Add(target)) + { + include.Remove(); + continue; + } + var nested = Compose(target, destination, expanding); + var anchor = include.Parent?.Name.LocalName == "Includes" ? include.Parent : include; + anchor.AddBeforeSelf(nested.Root!.Elements().Where(element => element.Name.LocalName != "Tags") + .Select(element => new XElement(element))); + include.Remove(); + } + else + { + var childDestination = RelativeSource(target); + if (emitted.Add(target)) + AddXml(childDestination, target, Compose(target, childDestination, expanding)); + include.SetAttributeValue("Source", RelativeTo(destination, childDestination)); + } + } + foreach (var includes in root.Descendants("Includes").Where(element => !element.HasElements).ToList()) + includes.Remove(); + expanding.Remove(path); + return document; + } + + private void Prepare(XElement root, string destination, bool manifestLeaf = false) + { + if (!prepared.Add(destination)) + return; + foreach (var registration in root.Descendants().Where(IsRegistration).ToList()) + { + if (registration.Name.LocalName == "Image" && !string.IsNullOrWhiteSpace((string?)registration.Attribute("Url"))) + continue; + var source = (string?)registration.Attribute("Source"); + if (registration.Name.LocalName == "Manifest" && !manifestLeaf) + { + var id = (string)registration.Attribute("ID")!; + XElement leaf; + if (IsFullManifest(registration)) + leaf = new XElement(registration); + else + leaf = Manifest(source!, id); + leaf.Attribute("Source")?.Remove(); + if (options.InlineManifests) + { + registration.ReplaceWith(leaf); + } + else + { + var originalSource = source; + if (string.IsNullOrWhiteSpace(originalSource)) + { + var definition = documents.Values.FirstOrDefault(document => + document.NormalizedDocument.Root!.Elements("Manifest").Any(element => + string.Equals((string?)element.Attribute("ID"), id, StringComparison.OrdinalIgnoreCase)) + && document.NormalizedDocument.Root.Elements().All(element => + element.Name.LocalName is "Manifest" or "Tags" + || element.Name.LocalName == "Includes" && !element.HasElements)); + originalSource = definition?.Path; + } + var leafDestination = string.IsNullOrWhiteSpace(originalSource) + ? "manifests/" + SafeSegment(id) + ".xml" : RelativeSource(originalSource); + if (string.Equals(leafDestination, destination, StringComparison.OrdinalIgnoreCase) + || xml.TryGetValue(leafDestination, out var existingDocument) + && existingDocument.Root!.Elements().Any(element => element.Name.LocalName is not ("Manifest" or "Tags"))) + leafDestination = "manifests/" + SafeSegment(id) + ".xml"; + if (!xml.TryGetValue(leafDestination, out var leafDocument)) + { + leafDocument = new XDocument(new XElement("Metadata", leaf)); + Prepare(leafDocument.Root!, leafDestination, manifestLeaf: true); + AddXml(leafDestination, originalSource ?? "manifest:" + id, leafDocument); + } + else + { + if (!leafDocument.Root!.Elements("Manifest").Any(element => + string.Equals((string?)element.Attribute("ID"), id, StringComparison.OrdinalIgnoreCase))) + { + var addition = new XDocument(new XElement("Metadata", leaf)); + prepared.Remove(leafDestination); + Prepare(addition.Root!, leafDestination, manifestLeaf: true); + leafDocument.Root.Add(addition.Root!.Elements()); + } + } + registration.RemoveNodes(); + registration.ReplaceAttributes(new XAttribute("ID", id), + new XAttribute("Source", RelativeTo(destination, leafDestination))); + } + } + else if (!string.IsNullOrWhiteSpace(source)) + { + var assetDestination = RelativeSource(source); + if (xml.ContainsKey(assetDestination)) + throw new InvalidOperationException($"Resource/XML path collision: {assetDestination}"); + Claim(assetDestination, source); + files.TryAdd(assetDestination, source); + registration.SetAttributeValue("Source", RelativeTo(destination, assetDestination)); + } + } + if (options.ResolveVariables) + new VariableResolver().ReplaceInElement(root); + } + + private static bool IsFullManifest(XElement manifest) => + string.IsNullOrWhiteSpace((string?)manifest.Attribute("Source")); + + private XElement Manifest(string source, string id) + { + if (!manifestCandidates.TryGetValue(source, out var full)) + { + IEnumerable candidates; + if (documents.TryGetValue(source, out var captured) && captured.Includes.Count == 0 + && captured.NormalizedDocument.Root!.Elements("Manifest").Any(IsFullManifest)) + candidates = captured.NormalizedDocument.Root!.Elements("Manifest"); + else + { + RelativeSource(source); + var leaf = MetadataReader.Read(source, new MetadataSourceOptions + { + SourceRoot = input.SourceRoot, + InlineManifests = true, + CustomSource = input.Source is HttpXmlSource or PhysicalFileXmlSource ? null : input.Source + }); + var normalized = leaf.Document; + foreach (var registration in normalized.Descendants().Where(IsRegistration)) + { + if (registration.Attribute("Source") is { } attribute) + { + var uri = MetadataResourceUri.Resolve(source, attribute.Value); + attribute.Value = uri.IsFile ? uri.LocalPath : uri.AbsoluteUri; + } + } + candidates = normalized.Root!.Elements("Manifest"); + } + full = candidates.Where(IsFullManifest).ToArray(); + manifestCandidates.Add(source, full); + } + var matches = full.Where(manifest => string.Equals((string?)manifest.Attribute("ID"), id, + StringComparison.OrdinalIgnoreCase)).ToList(); + if (matches.Count != 1) + throw new InvalidOperationException($"Expected one full manifest for {id} in {source}."); + var result = new XElement(matches[0]); + result.Attribute("Source")?.Remove(); + return result; + } + + + private string RelativeSource(string source) + { + string relative; + if (Uri.TryCreate(source, UriKind.Absolute, out var uri) && uri.Scheme is "http" or "https") + { + var root = Uri.TryCreate(input.SourceRoot, UriKind.Absolute, out var rootUri) ? rootUri : null; + var baseUri = root?.Scheme is "http" or "https" ? new Uri(root.AbsoluteUri.TrimEnd('/') + "/") : null; + if (baseUri?.IsBaseOf(uri) == true) + relative = baseUri.MakeRelativeUri(uri).ToString(); + else + relative = "remote/" + SafeSegment(uri.GetLeftPart(UriPartial.Authority)) + "/" + + uri.AbsolutePath.TrimStart('/'); + if (uri.Query.Length != 0) + relative += ".query-" + SafeSegment(uri.Query); + } + else + { + relative = input.Source.GetRelativePath(input.SourceRoot, source).Replace('\\', '/'); + EnsureNoLinks(uri?.IsFile == true ? uri.LocalPath : source); + } + ValidateRelative(relative); + return relative; + } + + private static string SafeSegment(string value) => + Uri.EscapeDataString(value).Replace("~", "~7E", StringComparison.Ordinal).Replace("%", "~", StringComparison.Ordinal); + + private static void ValidateRelative(string relative) + { + if (string.IsNullOrWhiteSpace(relative) || Path.IsPathRooted(relative) + || Uri.TryCreate(relative, UriKind.Absolute, out _) + || relative.Split('/').Any(segment => segment is "" or "." or ".." + || segment.IndexOfAny(Path.GetInvalidFileNameChars()) >= 0)) + throw new InvalidOperationException($"Unsafe export path: {relative}"); + } + + private void Claim(string destination, string owner) + { + ValidateRelative(destination); + if (owners.TryGetValue(destination, out var existing) && !documents.Comparer.Equals(existing, owner)) + throw new InvalidOperationException($"Output path collision: {destination}"); + owners[destination] = owner; + } + + private void AddXml(string destination, string owner, XDocument document) + { + Claim(destination, owner); + if (files.ContainsKey(destination)) + throw new InvalidOperationException($"Resource/XML path collision: {destination}"); + xml[destination] = document; + } + + internal void Write() + { + Directory.CreateDirectory(output); + using var client = new HttpClient(new SocketsHttpHandler { AllowAutoRedirect = false }); + foreach (var (relative, source) in files) + { + var target = Target(relative); + if (Uri.TryCreate(source, UriKind.Absolute, out var uri) && !uri.IsFile) + { + if (uri.Scheme is not ("http" or "https")) + throw new InvalidOperationException($"Unsupported resource URI: {source}"); + using var request = new HttpRequestMessage(HttpMethod.Get, uri); + using var response = client.Send(request, HttpCompletionOption.ResponseHeadersRead); + if ((int)response.StatusCode is >= 300 and < 400) + throw new InvalidOperationException($"Resource redirect refused: {source}"); + response.EnsureSuccessStatusCode(); + using var body = response.Content.ReadAsStream(); + using var file = File.Create(target); + body.CopyTo(file); + } + else + File.Copy(uri?.IsFile == true ? uri.LocalPath : source, target); + } + foreach (var (relative, document) in xml) + document.Save(Target(relative)); + if (options.ConvertImages) + ImagePostProcessor.ApplyWebP(output, xml.Keys); + } + + private string Target(string relative) + { + var path = Path.GetFullPath(Path.Combine(output, relative)); + if (!IsWithin(path, output)) + throw new InvalidOperationException($"Output path escaped root: {relative}"); + EnsureNoLinks(path); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + return path; + } + + private static string RelativeTo(string document, string resource) => + Path.GetRelativePath(Path.GetDirectoryName(document) is { Length: > 0 } directory ? directory : ".", resource).Replace('\\', '/'); + + private static bool IsRegistration(XElement element) => MetadataReader.IsRegistration(element); + + } +} diff --git a/Ra3.BattleNet.Metadata/MetadataSourceExporter.cs b/Ra3.BattleNet.Metadata/Exporting/MetadataSourceExporter.cs similarity index 94% rename from Ra3.BattleNet.Metadata/MetadataSourceExporter.cs rename to Ra3.BattleNet.Metadata/Exporting/MetadataSourceExporter.cs index 50cb917..513814e 100644 --- a/Ra3.BattleNet.Metadata/MetadataSourceExporter.cs +++ b/Ra3.BattleNet.Metadata/Exporting/MetadataSourceExporter.cs @@ -4,7 +4,7 @@ namespace Ra3.BattleNet.Metadata; /// -/// 把已展平的在线元数据导出成单个 Mod 的自包含作者源声明，是 的反向操作。 +/// 把在线元数据导出成单个 Mod 的自包含作者源声明。 /// /// 只抄这个 Mod 自己的节点和它直接引用的 Image / Markdown 登记，不夹带其他 Mod 或 Application。 /// 资源引用换成不含发布前缀的短 ID；登记节点的 Source 既可以是相对发布根的路径，也可以解析成绝对地址。 @@ -12,6 +12,17 @@ namespace Ra3.BattleNet.Metadata; /// public static class MetadataSourceExporter { + public static XElement ExportManifestElement(XElement manifest) + { + ArgumentNullException.ThrowIfNull(manifest); + var source = manifest.Name.LocalName == "Manifest" ? manifest + : manifest.Descendants("Manifest").SingleOrDefault() + ?? throw new ArgumentException("XML has no Manifest node.", nameof(manifest)); + var element = new XElement(source); + element.SetAttributeValue("ID", MetadataId.LocalId((string?)element.Attribute("ID"))); + return element; + } + /// /// 整份模组声明：保留该 Mod 的全部版本与线上 CurrentVersion，不挑版本、不改当前版本。 /// 每个版本都指向本目录的短清单叶子，资源引用与安装路径一致。此方法只做纯转换，不读写磁盘、不发网络请求。 @@ -147,7 +158,7 @@ private static string Build(Metadata root, Metadata modRaw, string? publishedMet return document.ToString(); } - private static XElement Copy(Metadata node) => XElement.Parse(MetadataXml.Write(node)); + private static XElement Copy(Metadata node) => MetadataXml.ToElement(node); private static void KeepSelectedPackage(XElement mod, string version) { @@ -268,10 +279,7 @@ private sealed class ShortIdGenerator public string Create(string oldId) { - var local = oldId; - var separator = local.LastIndexOf(':'); - if (separator >= 0 && separator < local.Length - 1) - local = local[(separator + 1)..]; + var local = MetadataId.LocalId(oldId); var slug = new StringBuilder(local.Length); foreach (var character in local) diff --git a/Ra3.BattleNet.Metadata/MetadataXml.cs b/Ra3.BattleNet.Metadata/Exporting/MetadataXml.cs similarity index 91% rename from Ra3.BattleNet.Metadata/MetadataXml.cs rename to Ra3.BattleNet.Metadata/Exporting/MetadataXml.cs index 6604938..f481615 100644 --- a/Ra3.BattleNet.Metadata/MetadataXml.cs +++ b/Ra3.BattleNet.Metadata/Exporting/MetadataXml.cs @@ -9,7 +9,7 @@ namespace Ra3.BattleNet.Metadata; public static class MetadataXml { /// 还原成 XML 文本。 - public static string Write(Metadata node) => new XElement(ToElement(node)).ToString(); + public static string Write(Metadata node) => ToElement(node).ToString(); /// 还原成 XML 节点。 public static XElement ToElement(Metadata node) diff --git a/Ra3.BattleNet.Metadata/ImagePostProcessor.cs b/Ra3.BattleNet.Metadata/ImagePostProcessor.cs deleted file mode 100644 index 32c59c0..0000000 --- a/Ra3.BattleNet.Metadata/ImagePostProcessor.cs +++ /dev/null @@ -1,71 +0,0 @@ -using System.Xml.Linq; - -namespace Ra3.BattleNet.Metadata; - -/// -/// 展平后：扫描 Image 登记节点，调用 Imaging CLI 转 WebP，由本类改写 XML 的 Source。 -/// 图片不写 Hash：只有 Manifest 与 Updater 记录带哈希。 -/// -public static class ImagePostProcessor -{ - private static readonly HashSet Convertible = new(StringComparer.OrdinalIgnoreCase) - { - ".png", ".jpg", ".jpeg", ".bmp", ".gif", ".tiff", ".tif" - }; - - /// - /// 处理 outputDir 下 metadata.xml；返回转换张数。转换成功后删除 Output 内原图（发布面只留被引用资源）。 - /// - public static int ApplyWebP(string outputDir, TextWriter? log = null) - { - log ??= Console.Out; - outputDir = Path.GetFullPath(outputDir); - var flatPath = Path.Combine(outputDir, "metadata.xml"); - if (!File.Exists(flatPath)) - throw new FileNotFoundException($"找不到展平文件: {flatPath}"); - - var doc = XDocument.Load(flatPath); - var converted = 0; - var removedSources = new List(); - - foreach (var image in doc.Descendants("Image")) - { - if (image.Attribute("ID") == null) - continue; - - var sourceAttr = image.Attribute("Source"); - if (sourceAttr == null || string.IsNullOrWhiteSpace(sourceAttr.Value)) - continue; - if (!string.IsNullOrWhiteSpace(image.Attribute("Url")?.Value)) - continue; - - var rel = sourceAttr.Value.Replace('\\', '/'); - var abs = Path.Combine(outputDir, rel); - if (!File.Exists(abs)) - continue; - - var ext = Path.GetExtension(abs); - if (!Convertible.Contains(ext)) - continue; - - var webpAbs = Path.ChangeExtension(abs, ".webp")!; - var webpRel = Path.ChangeExtension(rel, ".webp")!.Replace('\\', '/'); - - log.WriteLine($" convert {rel} -> {webpRel}"); - ImagingInvoker.ConvertToWebP(abs, webpAbs); - sourceAttr.Value = webpRel; - removedSources.Add(abs); - converted++; - } - - doc.Save(flatPath); - - // XML 已指向 webp，删除 Output 内原图（源树不受影响） - foreach (var source in removedSources) - { - try { File.Delete(source); } catch { /* 尽力清理 */ } - } - - return converted; - } -} diff --git a/Ra3.BattleNet.Metadata/Metadata.cs b/Ra3.BattleNet.Metadata/Metadata.cs index 34c0dc4..0526e7a 100644 --- a/Ra3.BattleNet.Metadata/Metadata.cs +++ b/Ra3.BattleNet.Metadata/Metadata.cs @@ -3,8 +3,7 @@ namespace Ra3.BattleNet.Metadata { /// - /// 展平元数据树（纯反序列化）。仅 Name/Value/Attributes/Children； - /// 不含 Include 展开、Defines、public/private 等构建期语义——构建期语义见 MetadataFlattener/MetadataInheritance。 + /// 元数据节点投影；组合读取由 MetadataReader 负责。 /// public class Metadata { @@ -20,11 +19,6 @@ public class Metadata /// 节点文本值（仅叶子节点有效）。 public string? Value => _value; - /// - /// 从已展平的 metadata.xml 加载。遇到 Include/Module 会报错——本入口只接受展平产物。 - /// - public static Metadata LoadFromFile(string path) => MetadataParser.LoadFromFile(path); - /// /// 获取属性值。 /// @@ -96,12 +90,6 @@ internal void ParseElement(XElement element) { Name = element.Name.LocalName; - if (Name is "Include" or "Includes" or "Module") - { - throw new InvalidOperationException( - $"Load 只接受展平后的 metadata.xml（发现 {Name} 元素，疑似源树文件）"); - } - foreach (var attr in element.Attributes()) _variables[attr.Name.LocalName] = attr.Value; diff --git a/Ra3.BattleNet.Metadata/MetadataFlattener.cs b/Ra3.BattleNet.Metadata/MetadataFlattener.cs deleted file mode 100644 index 807502e..0000000 --- a/Ra3.BattleNet.Metadata/MetadataFlattener.cs +++ /dev/null @@ -1,399 +0,0 @@ -using System.Xml.Linq; - -namespace Ra3.BattleNet.Metadata; - -/// -/// 将带 Include 的源 XML 展平为单一数据树（不内联资源文件正文）。 -/// 资源登记 ID 展平为全局唯一：{路径前缀}:{localId}，路径前缀为定义文件相对源根、去 .xml 的路径。 -/// -public static class MetadataFlattener -{ - /// - /// 从入口 metadata.xml 展平，Source 路径改写为相对 sourceRoot，ID 按定义文件加前缀。 - /// 入口、Include 与被登记资源一律限制在 sourceRoot 内，越界路径在读取目标前直接拒绝。 - /// - public static XDocument Flatten(string entryPath, string sourceRoot, string schemaVersion, string contentRevision) => - Flatten(entryPath, sourceRoot, schemaVersion, contentRevision, validateCompatibility: true); - - /// - /// 展平实现。 为 false 时跳过跨实体 Compatibility 校验： - /// 单独读一个模组目录时，Compatibility 引用的其他模组本来就不在这棵树里。 - /// - internal static XDocument Flatten( - string entryPath, - string sourceRoot, - string schemaVersion, - string contentRevision, - bool validateCompatibility) - { - if (string.IsNullOrWhiteSpace(entryPath)) - throw new ArgumentException("入口文件不能为空", nameof(entryPath)); - if (string.IsNullOrWhiteSpace(sourceRoot)) - throw new ArgumentException("源目录不能为空", nameof(sourceRoot)); - - var sourceRootFull = Path.GetFullPath(sourceRoot); - var entryFull = Path.GetFullPath(entryPath); - EnsureWithinRoot(entryFull, sourceRootFull, "入口文件"); - - var session = new FlattenSession(sourceRootFull); - var root = LoadAndExpand(entryFull, session); - - // Include 展开后、写出版本前：合并 Base/InheritFrom，发布物无继承痕迹 - MetadataInheritance.Resolve(root); - - ValidateEntityIds(session.EntitySources); - if (validateCompatibility) - ModCompatibility.Validate(root); - - root.SetAttributeValue("SchemaVersion", schemaVersion); - root.SetAttributeValue("ContentRevision", contentRevision); - - root.Descendants().Where(e => e.Name.LocalName is "Includes" or "Include").ToList() - .ForEach(e => e.Remove()); - - return new XDocument(new XDeclaration("1.0", "utf-8", null), root); - } - - /// 实体 ID（Mod/Application）全局唯一；含 ':' 一律拒绝（前缀由构建器生成）。 - private static void ValidateEntityIds(Dictionary> entitySources) - { - foreach (var kv in entitySources) - { - if (kv.Key.Contains(':', StringComparison.Ordinal)) - { - throw new InvalidOperationException( - $"实体 ID \"{kv.Key}\" 含 ':'——实体 ID 不加前缀，请写短名"); - } - } - - var duplicates = entitySources.Where(kv => kv.Value.Count > 1).ToList(); - if (duplicates.Count == 0) return; - - var lines = duplicates.SelectMany(kv => - kv.Value.Select(src => $" - {src} (ID=\"{kv.Key}\")")); - throw new InvalidOperationException("实体 ID 重复:\n" + string.Join("\n", lines)); - } - - /// 由定义文件绝对路径生成路径前缀（相对 sourceRoot、/ 分隔、无扩展名）。 - public static string PathPrefix(string absoluteFilePath, string sourceRoot) - { - var rel = ToRootRelative(Path.GetFullPath(absoluteFilePath), Path.GetFullPath(sourceRoot)); - if (rel.EndsWith(".xml", StringComparison.OrdinalIgnoreCase)) - rel = rel[..^4]; - return rel.Replace('\\', '/'); - } - - /// 将 localId 限定为 {prefix}:{localId}；含 ':' 一律拒绝（前缀由构建器生成）。 - public static string QualifyId(string pathPrefix, string localId) - { - if (string.IsNullOrWhiteSpace(localId)) - throw new InvalidOperationException("ID 不能为空"); - if (localId.Contains(':', StringComparison.Ordinal)) - throw new InvalidOperationException( - $"登记 ID \"{localId}\" 含 ':'——前缀由构建器自动生成，请写短名"); - if (string.IsNullOrWhiteSpace(pathPrefix)) - return localId; - return $"{pathPrefix}:{localId}"; - } - - /// 取限定 ID 的局部名（最后一个 ':' 之后）；无冒号则整串。 - public static string LocalId(string? id) - { - if (string.IsNullOrEmpty(id)) - return string.Empty; - var i = id.LastIndexOf(':'); - return i < 0 ? id : id[(i + 1)..]; - } - - /// 节点分类：Image/Markdown/Manifest 资源登记节点（是否带 @ID 由调用处再判断）。 - private static bool IsRegistrationNode(XElement el) => - el.Name.LocalName is "Image" or "Markdown" or "Manifest"; - - /// 节点分类：Mod/Application 实体节点。 - private static bool IsEntity(XElement el) => - el.Name.LocalName is "Mod" or "Application"; - - /// - /// 节点分类：文本内容是资源 ID 短名的引用节点。 - /// 新增引用类型（如新样式字段、新 Post 字段）只需在此处登记。 - /// - private static bool IsTextReference(XElement el) => el.Name.LocalName switch - { - "Icon" or "Logo" or "Content" or "HeadImage" => true, - // Package.Manifest 引用：无 ID 且无子元素 - "Manifest" => el.Attribute("ID") == null && !el.HasElements, - // Background 等处的 ID 文本引用；登记节点必有 ID - "Image" => el.Attribute("ID") == null, - _ => false - }; - - private static XElement LoadAndExpand(string full, FlattenSession session) - { - if (!session.Processing.Add(full)) - throw new InvalidOperationException($"检测到循环引用: {full}"); - - if (!File.Exists(full)) - throw new FileNotFoundException($"找不到文件: {full}"); - - var doc = XDocument.Load(full); - var root = doc.Root ?? throw new InvalidOperationException($"无效 XML: {full}"); - if (root.Name.LocalName != "Metadata") - throw new InvalidOperationException($"根节点必须是 Metadata: {full}"); - - var baseDir = Path.GetDirectoryName(full) - ?? throw new InvalidOperationException($"无法确定目录: {full}"); - var prefix = PathPrefix(full, session.SourceRootFull); - - // 展开前单趟：校验本文件手写 ID（登记/实体禁止 ':' 前缀）并记录顶层实体来源 - ValidateAndRecordOwnIds(root, full, session); - ExpandElement(root, baseDir, session); - // 展开后单趟：改写 Source、限定登记 ID，并同步建立引用作用域映射 - var scope = ProcessRegistrations(root, baseDir, prefix, session); - // 最后一趟独立改写引用：前向引用要求作用域映射已完整 - RewriteIdReferences(root, scope); - - session.Processing.Remove(full); - return root; - } - - /// - /// 展开前单趟处理本文件声明：拒绝手工前缀 ID（Image/Markdown/Manifest/Mod/Application）， - /// 并记录顶层实体 ID → 来源（展开会把 include 的实体并进来，不能算本文件的）。 - /// - private static void ValidateAndRecordOwnIds(XElement root, string fullPath, FlattenSession session) - { - var rel = ToRootRelative(fullPath, session.SourceRootFull); - foreach (var el in root.DescendantsAndSelf()) - { - var id = el.Attribute("ID")?.Value; - - if (el.Parent == root && IsEntity(el) && !string.IsNullOrWhiteSpace(id)) - { - if (!session.EntitySources.TryGetValue(id, out var list)) - session.EntitySources[id] = list = new List(); - list.Add($"{rel} ({el.Name.LocalName})"); - } - - if (string.IsNullOrWhiteSpace(id) || !id.Contains(':', StringComparison.Ordinal)) - continue; - if (IsRegistrationNode(el)) - { - throw new InvalidOperationException( - $"登记 ID \"{id}\" 含 ':'——前缀由构建器自动生成，请写短名"); - } - if (IsEntity(el)) - { - throw new InvalidOperationException( - $"实体 ID \"{id}\" 含 ':'——实体 ID 不加前缀，请写短名"); - } - } - } - - private static void ExpandElement(XElement element, string baseDir, FlattenSession session) - { - var includeHosts = element.Elements() - .Where(e => e.Name.LocalName is "Includes" or "Include") - .ToList(); - - foreach (var host in includeHosts) - { - if (host.Name.LocalName == "Include") - { - InsertInclude(host, baseDir, session); - host.Remove(); - } - else - { - foreach (var inc in host.Elements().Where(e => e.Name.LocalName == "Include").ToList()) - { - InsertInclude(inc, baseDir, session); - inc.Remove(); - } - if (!host.HasElements) - host.Remove(); - } - } - - foreach (var child in element.Elements().ToList()) - { - if (child.Name.LocalName is "Includes" or "Include") - continue; - ExpandElement(child, baseDir, session); - } - } - - private static void InsertInclude(XElement includeEl, string baseDir, FlattenSession session) - { - var rel = includeEl.Attribute("Source")?.Value; - if (string.IsNullOrWhiteSpace(rel)) - throw new InvalidOperationException("Include 缺少 Source"); - - var target = ResolveWithinRoot(baseDir, rel, session.SourceRootFull, "Include"); - var includedRoot = LoadAndExpand(target, session); - - var anchor = includeEl.Parent is { Name.LocalName: "Includes" } host - ? host - : includeEl; - - foreach (var child in includedRoot.Elements().ToList()) - { - if (child.Name.LocalName is "Tags" or "Includes") - continue; - - if (child.Name.LocalName == "Manifest") - { - var id = child.Attribute("ID")?.Value; - if (string.IsNullOrWhiteSpace(id)) - throw new InvalidOperationException($"Manifest 缺少 ID: {target}"); - - var existingSource = child.Attribute("Source")?.Value; - if (!string.IsNullOrWhiteSpace(existingSource)) - { - // 已是下层 stub，ID 应已限定 - anchor.AddBeforeSelf(new XElement(child)); - continue; - } - - var sourceRel = ToRootRelative(target, session.SourceRootFull); - // 叶子 Manifest 经自身展平已限定（构建期产物）；贡献者手写前缀由 QualifyId 拒绝 - var qualified = id.Contains(':', StringComparison.Ordinal) - ? id - : QualifyId(PathPrefix(target, session.SourceRootFull), id); - var stub = new XElement("Manifest", - new XAttribute("ID", qualified), - new XAttribute("Source", sourceRel)); - anchor.AddBeforeSelf(stub); - continue; - } - - anchor.AddBeforeSelf(new XElement(child)); - } - } - - /// - /// 展开后单趟处理全部登记节点：Source 改写为相对源根、本文件登记 ID 限定为 - /// {路径前缀}:{localId}，并按文档顺序同步建立 short localId → qualifiedId 作用域映射 - /// （完整限定 ID 始终可查；同文件作用域内短名冲突则移出短名键）。 - /// - private static Dictionary ProcessRegistrations(XElement root, string baseDir, string pathPrefix, FlattenSession session) - { - var map = new Dictionary(StringComparer.Ordinal); - foreach (var el in root.DescendantsAndSelf()) - { - if (!IsRegistrationNode(el)) - continue; - - var idAttr = el.Attribute("ID"); - var id = idAttr?.Value; - if (string.IsNullOrWhiteSpace(id)) - continue; - - // 已带前缀的登记节点来自 Include，源文件已按自己的目录改写并限定过： - // 再按当前文件目录解释一次会把 root 相对路径误读成 baseDir 相对路径。 - if (!id.Contains(':', StringComparison.Ordinal)) - { - RewriteSource(el, baseDir, session); - id = idAttr!.Value = QualifyId(pathPrefix, id); - } - - map[id] = id; - - var local = LocalId(id); - if (map.TryGetValue(local, out var existing) && - !string.Equals(existing, id, StringComparison.Ordinal)) - { - // 跨 Include 同短名：短名不可再解析（应在各自文件内已改写引用），移出短名键 - map.Remove(local); - continue; - } - map[local] = id; - } - return map; - } - - /// - /// 将登记节点（带 ID 的 Image/Markdown/Manifest）的 Source 改写为相对源根路径；Url 外链跳过。 - /// 先用 baseDir、再用源根解释；两条候选都必须在源根内，越界在探测前拒绝。 - /// - private static void RewriteSource(XElement el, string baseDir, FlattenSession session) - { - if (el.Name.LocalName is not ("Image" or "Markdown" or "Manifest")) - return; - - var sourceAttr = el.Attribute("Source"); - if (sourceAttr == null || string.IsNullOrWhiteSpace(sourceAttr.Value)) - return; - if (!string.IsNullOrWhiteSpace(el.Attribute("Url")?.Value)) - return; - - var raw = sourceAttr.Value; - var fromBase = ResolveWithinRoot(baseDir, raw, session.SourceRootFull, "资源 Source"); - if (File.Exists(fromBase)) - { - sourceAttr.Value = ToRootRelative(fromBase, session.SourceRootFull); - return; - } - - var fromRoot = ResolveWithinRoot(session.SourceRootFull, raw, session.SourceRootFull, "资源 Source"); - if (File.Exists(fromRoot)) - sourceAttr.Value = ToRootRelative(fromRoot, session.SourceRootFull); - } - - private static void RewriteIdReferences(XElement root, Dictionary scope) - { - foreach (var el in root.DescendantsAndSelf()) - { - if (IsTextReference(el)) - RewriteTextRef(el, scope); - } - } - - private static void RewriteTextRef(XElement el, Dictionary scope) - { - var text = el.Value?.Trim(); - if (string.IsNullOrEmpty(text)) - return; - if (scope.TryGetValue(text, out var qualified)) - el.Value = qualified; - // 找不到：留给 ValidateHard 报断引用（可能是笔误） - } - - private static string ToRootRelative(string absolutePath, string sourceRoot) - { - var rel = Path.GetRelativePath(sourceRoot, absolutePath).Replace('\\', '/'); - if (rel.StartsWith("..", StringComparison.Ordinal)) - throw new InvalidOperationException($"资源不在源目录内: {absolutePath}"); - return rel; - } - - /// - /// 把相对路径按 baseDir 解析，并要求结果仍在源根内。越界路径（..、绝对路径、跨盘） - /// 在此直接拒绝，调用方不会再去探测或读取目标。 - /// - private static string ResolveWithinRoot(string baseDir, string relativePath, string sourceRootFull, string kind) - { - var full = Path.GetFullPath(Path.Combine(baseDir, relativePath.Replace('\\', '/'))); - EnsureWithinRoot(full, sourceRootFull, kind); - return full; - } - - private static void EnsureWithinRoot(string fullPath, string sourceRootFull, string kind) - { - var relative = Path.GetRelativePath(sourceRootFull, fullPath); - // 跨盘时 GetRelativePath 原样返回绝对路径，IsPathRooted 拦下；同一磁盘只可能以 .. 开头。 - if (Path.IsPathRooted(relative) - || relative == ".." - || relative.StartsWith(".." + Path.DirectorySeparatorChar, StringComparison.Ordinal)) - throw new InvalidOperationException($"{kind} 越出源目录: {fullPath}"); - } - - /// 单次展平过程的共享状态：源根、展开栈（循环检测）、实体 ID 来源表。 - private sealed class FlattenSession - { - public FlattenSession(string sourceRootFull) => SourceRootFull = sourceRootFull; - - public string SourceRootFull { get; } - public HashSet Processing { get; } = new(StringComparer.OrdinalIgnoreCase); - public Dictionary> EntitySources { get; } = new(StringComparer.OrdinalIgnoreCase); - } -} diff --git a/Ra3.BattleNet.Metadata/MetadataSchema.cs b/Ra3.BattleNet.Metadata/MetadataSchema.cs index 49a1770..28c08d4 100644 --- a/Ra3.BattleNet.Metadata/MetadataSchema.cs +++ b/Ra3.BattleNet.Metadata/MetadataSchema.cs @@ -1,7 +1,7 @@ namespace Ra3.BattleNet.Metadata; /// -/// 发布物的契约版本（metadata.xml 根上的 SchemaVersion）。 +/// 数据契约版本（入口 metadata.xml 根上的 SchemaVersion，源树与发布物都写在这里）。 /// 客户端拿它判断远端数据能否被当前版本直接消费。 /// public static class MetadataSchema @@ -10,9 +10,9 @@ public static class MetadataSchema public const string Current = MetadataBuilder.DefaultSchemaVersion; /// - /// 判断发布物的 SchemaVersion 是否能被本库直接消费。 + /// 判断根清单的 SchemaVersion 是否能被本库直接消费。 /// - /// 发布物根上的 SchemaVersion；缺失时为 null。 + /// 入口根节点上的 SchemaVersion；缺失时为 null。 public static bool IsCompatible(string? version) { return string.Equals(version, Current, StringComparison.Ordinal); diff --git a/Ra3.BattleNet.Metadata/MetadataSourceManifest.cs b/Ra3.BattleNet.Metadata/MetadataSourceManifest.cs deleted file mode 100644 index adead5e..0000000 --- a/Ra3.BattleNet.Metadata/MetadataSourceManifest.cs +++ /dev/null @@ -1,51 +0,0 @@ -using System.Xml.Linq; - -namespace Ra3.BattleNet.Metadata; - -/// -/// 解析本地作者声明里的版本清单：登记节点既可能是内联清单，也可能只是指向叶子的 stub。 -/// -/// 把 Include 进来的独立清单留成登记 stub（ID 已限定、 -/// Source 相对源根）；手写 <Manifest ID Source> 的 stub 带入口文件前缀， -/// 单独加载叶子时前缀又来自叶子自身路径。两处前缀可以不同，因此按 -/// 对齐局部名后再取节点。 -/// -public static class MetadataSourceManifest -{ - /// - /// 取版本包声明的清单节点。登记节点已内联清单（带 Skudef 或 File 子节点）时直接返回； - /// 只有 stub 时按 Source 在声明目录内读取叶子，并按局部名对齐清单 ID。 - /// - /// 已展平的本地作者声明根。 - /// 声明文件路径；叶子清单与相对 Source 都相对它所在的目录。 - /// 版本包节点（声明里的 Package）。 - /// 清单节点；版本包没有声明 Manifest 时返回 null。 - public static Metadata? Resolve(Metadata root, string declarationPath, Metadata package) - { - ArgumentNullException.ThrowIfNull(root); - ArgumentException.ThrowIfNullOrWhiteSpace(declarationPath); - ArgumentNullException.ThrowIfNull(package); - - var id = package.Find("Manifest")?.Value; - if (string.IsNullOrWhiteSpace(id)) - return null; - - var registration = root.ManifestRegistration(id) - ?? throw new InvalidOperationException($"模组声明找不到文件清单：{id}"); - if (registration.Find("Skudef") is not null || registration.Children.Any(child => child.Name == "File")) - return registration; - - var source = registration.Get("Source") - ?? throw new InvalidOperationException($"模组文件清单缺少 Source：{id}"); - var folder = Path.GetDirectoryName(Path.GetFullPath(declarationPath))!; - var leaf = MetadataResourceUri.Resolve(declarationPath, source); - if (!leaf.IsFile) - throw new InvalidOperationException("本地模组文件清单必须在模组目录内"); - - var document = MetadataBuilder.LoadSource(leaf.LocalPath, folder); - var localId = MetadataFlattener.LocalId(id); - return document.GetAllElements("Manifest").SingleOrDefault(manifest => - string.Equals(MetadataFlattener.LocalId(manifest.Get("ID")), localId, StringComparison.OrdinalIgnoreCase)) - ?? throw new InvalidOperationException($"文件清单 ID 与模组声明不一致：{id}"); - } -} diff --git a/Ra3.BattleNet.Metadata/Program.cs b/Ra3.BattleNet.Metadata/Program.cs index f6f6381..3f19f1a 100644 --- a/Ra3.BattleNet.Metadata/Program.cs +++ b/Ra3.BattleNet.Metadata/Program.cs @@ -2,92 +2,93 @@ namespace Ra3.BattleNet.Metadata; internal static class Program { - private const string DefaultSrcFolder = "./Metadata"; - private const string DefaultDstFolder = "./Output"; - private static int Main(string[] args) { - var srcFolder = DefaultSrcFolder; - var dstFolder = DefaultDstFolder; - string? schemaVersion = null; - string? contentRevision = null; + var source = "./Metadata"; + var output = "./Output"; var command = "build"; - var withWebp = false; + var convertImages = false; + var layout = MetadataExportLayout.Flat; + var inlineManifests = false; + var resolveVariables = false; + var inlinePaths = new HashSet(StringComparer.Ordinal); - foreach (var arg in args) + try { - if (arg == "build") - command = "build"; - else if (arg == "--webp") - withWebp = true; - else if (arg.StartsWith("--src=", StringComparison.Ordinal)) - srcFolder = arg["--src=".Length..]; - else if (arg.StartsWith("--dst=", StringComparison.Ordinal)) - dstFolder = arg["--dst=".Length..]; - else if (arg.StartsWith("--schema-version=", StringComparison.Ordinal)) - schemaVersion = arg["--schema-version=".Length..]; - else if (arg.StartsWith("--content-revision=", StringComparison.Ordinal)) - contentRevision = arg["--content-revision=".Length..]; - else if (arg is "--help" or "-h") + foreach (var argument in args) { - PrintHelp(); - return 0; + if (argument is "build" or "export") + command = argument; + else if (argument == "--webp") + convertImages = true; + else if (argument.StartsWith("--src=", StringComparison.Ordinal)) + source = argument["--src=".Length..]; + else if (argument.StartsWith("--dst=", StringComparison.Ordinal)) + output = argument["--dst=".Length..]; + else if (argument.StartsWith("--layout=", StringComparison.Ordinal)) + layout = Enum.Parse(argument["--layout=".Length..], ignoreCase: true); + else if (argument.StartsWith("--inline=", StringComparison.Ordinal)) + inlinePaths.Add(argument["--inline=".Length..].Replace('\\', '/')); + else if (argument == "--inline-manifests") + inlineManifests = true; + else if (argument == "--resolve-variables") + resolveVariables = true; + else if (argument is "--help" or "-h") + { + PrintHelp(); + return 0; + } + else + throw new ArgumentException($"未知参数: {argument}"); } - } - Console.WriteLine($"工作目录: {Environment.CurrentDirectory}"); - Console.WriteLine($"源目录: {srcFolder}"); - Console.WriteLine($"输出目录: {dstFolder}"); - Console.WriteLine($"WebP: {(withWebp ? "开（展平后按图调 Imaging CLI）" : "关")}"); - Console.WriteLine(); - - try - { - if (command != "build") + Console.WriteLine($"源: {source}"); + Console.WriteLine($"输出目录: {output}"); + if (command == "export") { - Console.Error.WriteLine($"未知命令: {command}"); - return 1; + var entry = Directory.Exists(source) ? Path.Combine(source, "metadata.xml") : source; + MetadataExporter.Export(MetadataReader.Read(entry), output, new MetadataExportOptions + { + Layout = layout, + Inline = layout == MetadataExportLayout.Mixed ? inlinePaths.Contains : null, + InlineManifests = inlineManifests, + ConvertImages = convertImages, + ResolveVariables = resolveVariables + }); + Console.WriteLine("Export complete."); + return 0; } - Console.WriteLine(">>> 构建（展平 / 校验" + (withWebp ? " / Imaging" : "") + "）"); - MetadataBuilder.Build(srcFolder, dstFolder, schemaVersion, contentRevision, convertImages: withWebp); - Console.WriteLine("✓ 构建完成"); - - var flatPath = Path.Combine(dstFolder, "metadata.xml"); - var loaded = MetadataBuilder.Load(flatPath); - Console.WriteLine(); - Console.WriteLine("=== 展平结果 ==="); - Console.WriteLine($"SchemaVersion={loaded.Get("SchemaVersion")}"); - Console.WriteLine($"ContentRevision={loaded.Get("ContentRevision")}"); - foreach (var app in loaded.Applications()) + MetadataBuilder.Build(source, output, convertImages); + var root = MetadataReader.Read(Path.Combine(output, "metadata.xml")).Root; + Console.WriteLine($"SchemaVersion={root.Get("SchemaVersion")}"); + foreach (var app in root.Applications()) Console.WriteLine($"Application id={app.Id} version={app.Version}"); - foreach (var mod in loaded.Mods()) + foreach (var mod in root.Mods()) Console.WriteLine($"Mod id={mod.Id} version={mod.Version}"); - return 0; } - catch (Exception ex) + catch (Exception error) { - Console.ForegroundColor = ConsoleColor.Red; - Console.Error.WriteLine($"错误: {ex.Message}"); - Console.ResetColor(); + Console.Error.WriteLine($"错误: {error.Message}"); return 1; } } - private static void PrintHelp() - { - Console.WriteLine(""" - Ra3.BattleNet.Metadata — 核心构建 - - 用法: - build --src=./Metadata --dst=./Output - build --webp --src=./Metadata --dst=./Output + private static void PrintHelp() => Console.WriteLine(""" + Ra3.BattleNet.Metadata — 构建与导出 - --webp: 展平后对每张本地图调用 Imaging CLI（只产 webp）， - 由本程序改写 metadata.xml 的 Source。 + 用法: + build --src=./Metadata --dst=./Output + build --webp --src=./Metadata --dst=./Output + export --src=./Metadata/metadata.xml --dst=./Export --layout=tree + export --src=./Metadata --dst=./Export --layout=mixed --inline=mods/demo.xml --inline-manifests - NuGet 主包不含 Imaging；Imaging 为仓库编译 CLI。 - """); - } + Build: 入口与契约版本都来自源树，没有额外参数（SchemaVersion 写在源根 metadata.xml 上）。 + Export: --layout=flat|tree|mixed, + --inline= (repeatable for mixed), + --inline-manifests, --resolve-variables. + --webp: 调用仓库 Imaging CLI 转换本地图并改写 Source。 + NuGet 主包不含 Imaging。 + """); } diff --git a/Ra3.BattleNet.Metadata/Reading/HttpXmlSource.cs b/Ra3.BattleNet.Metadata/Reading/HttpXmlSource.cs new file mode 100644 index 0000000..8fad6f3 --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/HttpXmlSource.cs @@ -0,0 +1,207 @@ +using System.Net; +using System.Xml; +using System.Xml.Linq; + +namespace Ra3.BattleNet.Metadata; + +/// +/// 的 HTTP 只读实现：路径一律是绝对 URL，请求走内部 HttpClient， +/// 构造后无全局状态。读取全部流式进行，不先整包下载；XML 正文按 URL 缓存，同一个文件不发两次 GET。 +/// 不跟随重定向：展平器的越界检查只看即将发出的 URL，跟随 3xx 会把外站正文当源树内容读进来。 +/// +public sealed class HttpXmlSource : IMetadataXmlSource, IDisposable +{ + /// 单个 XML 请求的超时。 + private static readonly TimeSpan RequestTimeout = TimeSpan.FromSeconds(60); + + // 与 MetadataParser 口径一致：禁 DTD、不解析外部实体 + private static readonly XmlReaderSettings ReaderSettings = new() + { + DtdProcessing = DtdProcessing.Prohibit, + XmlResolver = null + }; + + private readonly Uri _baseUri; + private readonly HttpClient _client; + private readonly Dictionary _documents = new(StringComparer.Ordinal); + + /// + /// 视为源根目录，缺少尾部 / 时补齐，相对路径按它解析； + /// 为空时用默认处理器（关闭自动重定向）。 + /// + public HttpXmlSource(Uri baseUri, HttpMessageHandler? handler = null) + { + ArgumentNullException.ThrowIfNull(baseUri); + if (!baseUri.IsAbsoluteUri) + throw new ArgumentException("baseUri 必须是绝对 URL", nameof(baseUri)); + if (!IsHttpScheme(baseUri)) + throw new ArgumentException("baseUri 只能是 http/https URL", nameof(baseUri)); + + var root = baseUri.AbsoluteUri; + _baseUri = new Uri(root.EndsWith("/", StringComparison.Ordinal) ? root : root + "/"); + _client = handler is null + ? new HttpClient(new SocketsHttpHandler { AllowAutoRedirect = false }) + : new HttpClient(handler, disposeHandler: false); + _client.Timeout = RequestTimeout; + } + + /// 释放内部 HttpClient；注入的 handler 归调用方所有，不随本实例释放。 + public void Dispose() => _client.Dispose(); + + /// + public bool Exists(string path) + { + var full = GetFullPath(path); + using var response = Send(full); + if (response.StatusCode == HttpStatusCode.NotFound) + return false; + if (!response.IsSuccessStatusCode) + throw Failed(full, response); + + // 只缓存 XML 正文；图片等资源只判存在，不把整包读进内存 + if (LooksLikeXml(full, response)) + _documents[full] = ReadBody(response.Content); + return true; + } + + /// + public XDocument LoadDocument(string path) + { + var full = GetFullPath(path); + if (_documents.TryGetValue(full, out var body)) + { + using var cached = new MemoryStream(body, writable: false); + return Parse(cached); + } + + using var response = Send(full); + if (response.StatusCode == HttpStatusCode.NotFound) + throw new FileNotFoundException($"找不到文件: {full}"); + if (!response.IsSuccessStatusCode) + throw Failed(full, response); + + // 直接从响应流解析，不先落内存副本 + using var stream = response.Content.ReadAsStream(); + return Parse(stream); + } + + /// + public string Combine(string basePath, string relativePath) + { + ArgumentException.ThrowIfNullOrWhiteSpace(basePath); + ArgumentNullException.ThrowIfNull(relativePath); + + var baseDir = GetFullPath(basePath); + if (!baseDir.EndsWith("/", StringComparison.Ordinal)) + baseDir += "/"; + + // 作者可能写 \ 分隔的路径；先统一成 / 再交给 Uri 解析 + var relative = relativePath.Replace('\\', '/'); + return Normalize(new Uri(new Uri(baseDir), relative), relative); + } + + /// + // 文件 URL 返回所在目录，目录 URL 返回上一级；站点根没有父目录 + public string? GetDirectoryName(string path) + { + var full = GetFullPath(path); + if (new Uri(full).AbsolutePath == "/") + return null; + + var parent = new Uri(new Uri(full), full.EndsWith("/", StringComparison.Ordinal) ? ".." : "."); + var absolute = parent.AbsoluteUri; + return absolute.EndsWith("/", StringComparison.Ordinal) ? absolute : absolute + "/"; + } + + /// + // 同源地址使用相对路径；跨源地址保留绝对 URL。 + public string GetRelativePath(string fromPath, string toPath) + { + var from = GetFullPath(fromPath); + var to = GetFullPath(toPath); + + var fromUri = new Uri(from); + var toUri = new Uri(to); + if (!string.Equals(fromUri.Scheme, toUri.Scheme, StringComparison.OrdinalIgnoreCase) + || !string.Equals(fromUri.Host, toUri.Host, StringComparison.OrdinalIgnoreCase) + || fromUri.Port != toUri.Port) + return to; + + var dir = from.EndsWith("/", StringComparison.Ordinal) ? from : from + "/"; + return new Uri(dir).MakeRelativeUri(toUri).ToString(); + } + + /// + public string GetFullPath(string path) + { + ArgumentException.ThrowIfNullOrWhiteSpace(path); + var relative = path.Replace('\\', '/'); + return Normalize(new Uri(_baseUri, relative), relative); + } + + /// + public bool IsPathRooted(string path) + { + if (string.IsNullOrWhiteSpace(path)) + return false; + return Uri.TryCreate(path, UriKind.Absolute, out var uri) && IsHttpScheme(uri); + } + + private static bool IsHttpScheme(Uri uri) => + uri.Scheme == Uri.UriSchemeHttp || uri.Scheme == Uri.UriSchemeHttps; + + // 规范化：Uri 负责折叠 . / ..、统一 scheme/host 大小写与转义；这里补目录尾斜杠并拒绝非 http/https + private static string Normalize(Uri resolved, string input) + { + if (!IsHttpScheme(resolved)) + throw new InvalidOperationException($"只支持 http/https URL: {input}"); + + var absolute = resolved.AbsoluteUri; + return input.EndsWith("/", StringComparison.Ordinal) && !absolute.EndsWith("/", StringComparison.Ordinal) + ? absolute + "/" + : absolute; + } + + // 有 Content-Type 就看它含不含 xml；没有 Content-Type 时按 .xml 扩展名兜底 + private static bool LooksLikeXml(string url, HttpResponseMessage response) + { + var mediaType = response.Content.Headers.ContentType?.MediaType; + return string.IsNullOrEmpty(mediaType) + ? url.EndsWith(".xml", StringComparison.OrdinalIgnoreCase) + : mediaType.Contains("xml", StringComparison.OrdinalIgnoreCase); + } + + private static XDocument Parse(Stream stream) + { + using var reader = XmlReader.Create(stream, ReaderSettings); + return XDocument.Load(reader); + } + + private static byte[] ReadBody(HttpContent content) + { + using var stream = content.ReadAsStream(); + using var buffer = new MemoryStream(); + stream.CopyTo(buffer); + return buffer.ToArray(); + } + + private HttpResponseMessage Send(string fullUrl) + { + using var request = new HttpRequestMessage(HttpMethod.Get, fullUrl); + var response = _client.Send(request, HttpCompletionOption.ResponseHeadersRead); + + // 不读 Location、不跟随：越界检查只覆盖即将发出的 URL，跟随重定向等于绕开源根约束 + if ((int)response.StatusCode is >= 300 and < 400) + { + var status = (int)response.StatusCode; + var reason = response.ReasonPhrase; + response.Dispose(); + throw new InvalidOperationException($"GET {fullUrl} 返回重定向 {status} {reason}"); + } + + return response; + } + + private static HttpRequestException Failed(string full, HttpResponseMessage response) => + new($"GET {full} 返回 {(int)response.StatusCode} {response.ReasonPhrase}", null, response.StatusCode); +} diff --git a/Ra3.BattleNet.Metadata/Reading/IMetadataXmlSource.cs b/Ra3.BattleNet.Metadata/Reading/IMetadataXmlSource.cs new file mode 100644 index 0000000..28ee733 --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/IMetadataXmlSource.cs @@ -0,0 +1,34 @@ +using System.Xml.Linq; + +namespace Ra3.BattleNet.Metadata; + +/// +/// 组合读取的只读 XML 源。路径是不透明字符串，由实现负责读取与路径运算。 +/// +public interface IMetadataXmlSource +{ + /// 目标是否存在。 + bool Exists(string path); + + /// 读取并解析 XML 文档。 + XDocument LoadDocument(string path); + + /// 拼接目录与相对路径。 + string Combine(string basePath, string relativePath); + + /// 取所在目录；无法确定时返回 null。 + string? GetDirectoryName(string path); + + /// + /// 把 表示为相对 的路径； + /// 两者不同根（跨盘、跨 URL 主机）时原样返回 的绝对形式。 + /// URL 路径必须保留 URI 转义，不能解码百分号或已转义的分隔符；本地路径保留文件名字符。 + /// + string GetRelativePath(string fromPath, string toPath); + + /// 规范化为绝对形式（本地实现是绝对路径，URL 实现是绝对 URL）。 + string GetFullPath(string path); + + /// 是否是根路径（本地绝对路径、绝对 URL）。 + bool IsPathRooted(string path); +} diff --git a/Ra3.BattleNet.Metadata/Reading/MetadataDocumentInfo.cs b/Ra3.BattleNet.Metadata/Reading/MetadataDocumentInfo.cs new file mode 100644 index 0000000..c55250b --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/MetadataDocumentInfo.cs @@ -0,0 +1,51 @@ +using System.Xml.Linq; + +namespace Ra3.BattleNet.Metadata; + +public sealed class MetadataIncludeInfo +{ + internal MetadataIncludeInfo(string sourcePath, string targetPath) + { + SourcePath = sourcePath; + TargetPath = targetPath; + } + public string SourcePath { get; } + public string TargetPath { get; } +} + +public sealed class MetadataDocumentInfo +{ + internal MetadataDocumentInfo(string path, string relativePath, XDocument document) + { + Path = path; + RelativePath = relativePath; + Document = new XDocument(document); + NormalizedDocument = new XDocument(document); + } + public string Path { get; } + public string RelativePath { get; } + public XDocument Document { get; } + public XDocument NormalizedDocument { get; } + public IReadOnlyList Includes => IncludeList; + internal List IncludeList { get; } = []; +} + +public sealed class MetadataReadResult +{ + internal MetadataReadResult(XDocument document, string entryPath, string sourceRoot, + IReadOnlyList documents, IMetadataXmlSource source) + { + Document = document; + Root = MetadataParser.Load(document); + EntryPath = entryPath; + SourceRoot = sourceRoot; + Documents = documents; + Source = source; + } + public Metadata Root { get; } + public XDocument Document { get; } + public string EntryPath { get; } + public string SourceRoot { get; } + public IReadOnlyList Documents { get; } + internal IMetadataXmlSource Source { get; } +} diff --git a/Ra3.BattleNet.Metadata/Reading/MetadataId.cs b/Ra3.BattleNet.Metadata/Reading/MetadataId.cs new file mode 100644 index 0000000..480a2ae --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/MetadataId.cs @@ -0,0 +1,33 @@ +namespace Ra3.BattleNet.Metadata; + +public static class MetadataId +{ + public static string Qualify(string scope, string localId) + { + ArgumentException.ThrowIfNullOrWhiteSpace(localId); + if (localId.Contains('/') || localId.Contains(':')) return localId; + return string.IsNullOrEmpty(scope) ? Escape(localId) : scope + "/" + Escape(localId); + } + + public static string LocalId(string? id) + { + if (string.IsNullOrEmpty(id)) return string.Empty; + return Uri.UnescapeDataString(id[(id.LastIndexOf('/') + 1)..]); + } + + public static string PathPrefix(string absoluteFilePath, string sourceRoot) => + PathPrefix(new PhysicalFileXmlSource(), absoluteFilePath, sourceRoot); + + internal static string PathPrefix(IMetadataXmlSource source, string absoluteFilePath, string sourceRoot) + { + var full = source.GetFullPath(absoluteFilePath); + var root = source.GetFullPath(sourceRoot); + MetadataReader.EnsureWithinRoot(source, full, root, "Document"); + var relative = source.GetRelativePath(root, full).Replace('\\', '/'); + if (relative.EndsWith(".xml", StringComparison.OrdinalIgnoreCase)) relative = relative[..^4]; + return string.Join("/", relative.Split('/').Select(segment => Escape(MetadataReader.IsHttp(full) + ? Uri.UnescapeDataString(segment) : segment))); + } + private static string Escape(string value) => value is "." or ".." + ? value.Replace(".", "%2E", StringComparison.Ordinal) : Uri.EscapeDataString(value); +} diff --git a/Ra3.BattleNet.Metadata/MetadataParser.cs b/Ra3.BattleNet.Metadata/Reading/MetadataParser.cs similarity index 63% rename from Ra3.BattleNet.Metadata/MetadataParser.cs rename to Ra3.BattleNet.Metadata/Reading/MetadataParser.cs index 34901f2..1fefd54 100644 --- a/Ra3.BattleNet.Metadata/MetadataParser.cs +++ b/Ra3.BattleNet.Metadata/Reading/MetadataParser.cs @@ -5,8 +5,7 @@ namespace Ra3.BattleNet.Metadata; /// -/// 展平 metadata.xml 的纯反序列化：不展开 Include、不做构建期处理。 -/// 构建期（展平/继承/变量/XSD）见 MetadataFlattener / MetadataInheritance / VariableResolver / SchemaValidator。 +/// XML 节点的纯模型投影；路径、组合与资源身份由 MetadataReader 解释。 /// public static class MetadataParser { @@ -18,33 +17,7 @@ public static class MetadataParser IgnoreWhitespace = true }; - /// - /// 从已展平的 metadata.xml 加载元数据树。 - /// - public static Metadata LoadFromFile(string path) - { - if (string.IsNullOrWhiteSpace(path)) - throw new ArgumentException("文件路径不能为空", nameof(path)); - - var fullPath = Path.IsPathRooted(path) - ? path - : Path.Combine(Environment.CurrentDirectory, path); - - if (!File.Exists(fullPath)) - throw new FileNotFoundException($"找不到文件: {fullPath}"); - - try - { - using var stream = File.OpenRead(fullPath); - return Load(stream); - } - catch (XmlException ex) - { - throw new XmlException($"XML 解析失败: {fullPath}", ex); - } - } - - /// 解析已读入的正文。DTD 禁止、不解析外部实体，Include 仍由节点解析拒绝。 + /// 解析 XML 正文，禁止 DTD 和外部实体。 internal static Metadata Load(Stream stream) { using var reader = XmlReader.Create(stream, ReaderSettings); @@ -52,11 +25,7 @@ internal static Metadata Load(Stream stream) return Load(doc); } - /// - /// 从内存中的 XML 文档解析元数据树。根必须是 Metadata； - /// 与文件入口一样拒绝 Include/Includes/Module，只接受已展平的文档。 - /// 调用方可以先把内存节点包进新的 Metadata 根再解析，不需要写临时文件。 - /// + /// 将 Metadata 根文档投影为模型，不执行读取或组合处理。 public static Metadata Load(XDocument document) { if (document == null) diff --git a/Ra3.BattleNet.Metadata/Reading/MetadataReader.cs b/Ra3.BattleNet.Metadata/Reading/MetadataReader.cs new file mode 100644 index 0000000..987939a --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/MetadataReader.cs @@ -0,0 +1,295 @@ +using System.Xml.Linq; + +namespace Ra3.BattleNet.Metadata; + +public static class MetadataReader +{ + internal static StringComparer PathComparer => DocumentPathComparer.Instance; + public static MetadataReadResult Read(string pathOrUrl, MetadataSourceOptions? options = null) + { + ArgumentException.ThrowIfNullOrWhiteSpace(pathOrUrl); + var settings = options ?? new MetadataSourceOptions(); + if (settings.CustomSource is { } custom) + return ReadCore(custom, pathOrUrl, settings); + if (IsHttp(pathOrUrl)) + { + using var source = new HttpXmlSource(new Uri(new Uri(pathOrUrl), ".")); + return ReadCore(source, pathOrUrl, settings); + } + var path = Uri.TryCreate(pathOrUrl, UriKind.Absolute, out var uri) && uri.IsFile + ? uri.LocalPath : pathOrUrl; + return ReadCore(new PhysicalFileXmlSource(), path, settings); + } + + internal static string ResolveInclude(IMetadataXmlSource source, string documentPath, string includeSource, string sourceRoot) + { + ArgumentNullException.ThrowIfNull(source); + ArgumentException.ThrowIfNullOrWhiteSpace(includeSource); + if (IsHttp(documentPath) && Uri.TryCreate(includeSource, UriKind.Absolute, out var uri) + && uri.Scheme != Uri.UriSchemeHttp && uri.Scheme != Uri.UriSchemeHttps) + throw new InvalidOperationException("HTTP metadata cannot include local documents"); + var directory = source.GetDirectoryName(documentPath) + ?? throw new InvalidOperationException($"Cannot determine document directory: {documentPath}"); + var target = source.GetFullPath(source.Combine(directory, includeSource)); + EnsureWithinRoot(source, target, sourceRoot, "Include"); + return target; + } + + internal static MetadataReadResult ReadCore(IMetadataXmlSource source, string entryPath, MetadataSourceOptions options) + { + var entry = source.GetFullPath(entryPath); + var root = source.GetFullPath(options.SourceRoot ?? source.GetDirectoryName(entry) + ?? throw new InvalidOperationException($"Cannot determine source root: {entry}")); + EnsureWithinRoot(source, entry, root, "Entry"); + var session = new Session(source, entry, root, options); + return session.Read(); + } + + internal static bool IsHttp(string path) => Uri.TryCreate(path, UriKind.Absolute, out var uri) + && (uri.Scheme == Uri.UriSchemeHttp || uri.Scheme == Uri.UriSchemeHttps); + + internal static void EnsureWithinRoot(IMetadataXmlSource source, string path, string root, string kind) + { + var relative = source.GetRelativePath(root, path).Replace('\\', '/'); + if (source.IsPathRooted(relative) || relative == ".." || relative.StartsWith("../", StringComparison.Ordinal)) + throw new InvalidOperationException($"{kind} 越出源目录: {path}"); + } + + internal static bool IsRegistration(XElement element) => element.Attribute("ID") != null + && element.Name.LocalName is "Image" or "Markdown" or "Manifest"; + + internal static bool IsExternalManifest(XElement element) => element.Name.LocalName == "Manifest" + && element.Attribute("ID") != null + && !string.IsNullOrWhiteSpace(element.Attribute("Source")?.Value); + + internal static bool IsReference(XElement element) => element.Name.LocalName switch + { + "Icon" or "Logo" or "Content" or "HeadImage" => !element.HasElements, + "Manifest" or "Image" => element.Attribute("ID") == null && !element.HasElements, + _ => false + }; + + private sealed class Session + { + private readonly IMetadataXmlSource _source; + private readonly string _entry; + private readonly string _root; + private readonly MetadataSourceOptions _options; + private readonly Dictionary _documents; + private readonly HashSet _processing; + private readonly HashSet _registrations = new(StringComparer.OrdinalIgnoreCase); + private readonly Dictionary> _globalAliases = new(StringComparer.OrdinalIgnoreCase); + private readonly Dictionary _entities = new(StringComparer.OrdinalIgnoreCase); + private readonly List _ordered = []; + private readonly StringComparer _pathComparer; + private readonly HashSet _manifestProcessing = new(StringComparer.OrdinalIgnoreCase); + + internal Session(IMetadataXmlSource source, string entry, string root, MetadataSourceOptions options) + { + _source = source; + _entry = entry; + _root = root; + _options = options; + _pathComparer = DocumentPathComparer.Instance; + _documents = new Dictionary(_pathComparer); + _processing = new HashSet(_pathComparer); + } + + internal MetadataReadResult Read() + { + var entry = Capture(_entry); + foreach (var document in _ordered.ToArray()) RewriteReferences(document); + var expanded = Expand(entry, new HashSet(_pathComparer)); + MetadataInheritance.Resolve(expanded); + var baseDirectory = _source.GetDirectoryName(_entry)!; + foreach (var element in expanded.Descendants().Where(IsRegistration)) + { + if (element.Attribute("Source") is not { } source || string.IsNullOrWhiteSpace(source.Value)) continue; + if (element.Name.LocalName == "Image" && !string.IsNullOrWhiteSpace(element.Attribute("Url")?.Value)) continue; + source.Value = _source.GetRelativePath(baseDirectory, source.Value).Replace('\\', '/'); + } + var result = new MetadataReadResult(new XDocument(new XDeclaration("1.0", "utf-8", null), expanded), + _entry, _root, _ordered.AsReadOnly(), _source); + if (_options.ValidateReferences) MetadataValidator.EnsureReferences(result.Root); + return result; + } + + private string DocumentScope(string path) => MetadataId.PathPrefix(_source, path, _root); + + private MetadataDocumentInfo Capture(string path, bool manifestResource = false) + { + if (_processing.Contains(path)) throw new InvalidOperationException($"检测到循环引用: {path}"); + if (_documents.TryGetValue(path, out var existing)) return existing; + _processing.Add(path); + var raw = _source.LoadDocument(path); + if (raw.Root?.Name.LocalName != "Metadata") + throw new InvalidOperationException($"根节点必须是 Metadata: {path}"); + var info = new MetadataDocumentInfo(path, _source.GetRelativePath(_root, path).Replace('\\', '/'), raw); + _documents.Add(path, info); + _ordered.Add(info); + var root = info.NormalizedDocument.Root!; + var prefix = DocumentScope(path); + var ownIds = new HashSet(StringComparer.OrdinalIgnoreCase); + foreach (var element in root.Descendants().Where(IsRegistration)) + { + var id = element.Attribute("ID")!; + if (IsExternalManifest(element) && (!string.IsNullOrWhiteSpace(element.Attribute("HashAlgorithm")?.Value) + || element.HasElements)) + throw new InvalidOperationException($"External manifest cannot also contain a body: {id.Value} ({path})"); + NormalizeSource(element, path); + var scope = IsExternalManifest(element) + ? DocumentScope(element.Attribute("Source")!.Value) : prefix; + id.Value = MetadataId.Qualify(scope, id.Value); + if (!ownIds.Add(id.Value)) throw new InvalidOperationException($"登记 ID 重复: {id.Value} ({path})"); + if (!(manifestResource && element.Name.LocalName == "Manifest") && !_registrations.Add(id.Value)) + throw new InvalidOperationException($"登记 ID 重复: {id.Value} ({path})"); + AddAlias(_globalAliases, id.Value); + } + foreach (var entity in root.Elements().Where(e => e.Name.LocalName is "Mod" or "Application")) + { + var id = entity.Attribute("ID")?.Value; + if (string.IsNullOrWhiteSpace(id)) continue; + if (!_entities.TryAdd(id, path)) + throw new InvalidOperationException($"实体 ID 重复: {id} ({_entities[id]}; {path})"); + } + foreach (var include in root.Descendants().Where(e => e.Name.LocalName == "Include").ToArray()) + { + var target = ResolveInclude(_source, path, include.Attribute("Source")?.Value ?? string.Empty, _root); + include.SetAttributeValue("Source", target); + info.IncludeList.Add(new MetadataIncludeInfo(path, target)); + Capture(target, manifestResource); + } + _processing.Remove(path); + return info; + } + + private void NormalizeSource(XElement element, string path) + { + if (element.Attribute("Source") is not { } attribute || string.IsNullOrWhiteSpace(attribute.Value)) return; + if (element.Name.LocalName == "Image" && !string.IsNullOrWhiteSpace(element.Attribute("Url")?.Value)) return; + var raw = attribute.Value; + if (IsHttp(raw)) + { + attribute.Value = new Uri(raw).AbsoluteUri; + return; + } + if (IsHttp(path) && (raw.StartsWith("\\\\", StringComparison.Ordinal) + || !raw.StartsWith('/') && Uri.TryCreate(raw, UriKind.Absolute, out var uri) + && uri.Scheme != Uri.UriSchemeHttp && uri.Scheme != Uri.UriSchemeHttps)) + throw new InvalidOperationException("HTTP metadata cannot register local resources"); + var directory = _source.GetDirectoryName(path)!; + var target = _source.GetFullPath(_source.Combine(directory, raw)); + EnsureWithinRoot(_source, target, _root, "资源 Source"); + attribute.Value = target; + } + + private void RewriteReferences(MetadataDocumentInfo document) + { + var reachable = new HashSet(_pathComparer); + var ownAliases = new Dictionary>(StringComparer.OrdinalIgnoreCase); + foreach (var registration in document.NormalizedDocument.Descendants().Where(IsRegistration)) + AddAlias(ownAliases, registration.Attribute("ID")!.Value); + var full = new HashSet(StringComparer.OrdinalIgnoreCase); + var aliases = new Dictionary>(StringComparer.OrdinalIgnoreCase); + void Add(MetadataDocumentInfo current) + { + if (!reachable.Add(current.Path)) return; + foreach (var registration in current.NormalizedDocument.Descendants().Where(IsRegistration)) + { + var id = registration.Attribute("ID")!.Value; + full.Add(id); + AddAlias(aliases, id); + } + foreach (var edge in current.Includes) Add(_documents[edge.TargetPath]); + } + Add(document); + foreach (var reference in document.NormalizedDocument.Descendants().Where(IsReference)) + { + var id = reference.Value.Trim(); + if (id.Length == 0) continue; + if (full.Contains(id) || _registrations.Contains(id)) + { + reference.Value = id; + continue; + } + if (!id.Contains('/') && ownAliases.TryGetValue(id, out var own)) + { + if (own.Count != 1) throw new InvalidOperationException($"Ambiguous resource ID: {id} ({document.Path})"); + reference.Value = own.Single(); + continue; + } + if (!ownAliases.TryGetValue(id, out var matches) && !aliases.TryGetValue(id, out matches)) + _globalAliases.TryGetValue(id, out matches); + if (matches is { Count: > 1 }) + throw new InvalidOperationException($"Ambiguous resource ID: {id} ({document.Path})"); + reference.Value = matches is { Count: 1 } ? matches.Single() : id; + } + } + + private static void AddAlias(Dictionary> aliases, string id) + { + var local = MetadataId.LocalId(id); + if (!aliases.TryGetValue(local, out var values)) + aliases[local] = values = new HashSet(StringComparer.OrdinalIgnoreCase); + values.Add(id); + } + + private XElement Expand(MetadataDocumentInfo document, HashSet expanded) + { + expanded.Add(document.Path); + var root = new XElement(document.NormalizedDocument.Root!); + foreach (var include in root.Descendants().Where(e => e.Name.LocalName == "Include").ToArray()) + { + var target = include.Attribute("Source")!.Value; + if (!expanded.Add(target)) { include.Remove(); continue; } + var child = Expand(_documents[target], expanded); + include.ReplaceWith(child.Elements().Where(e => e.Name.LocalName != "Tags").ToArray()); + } + foreach (var host in root.Descendants().Where(e => e.Name.LocalName == "Includes").ToArray()) + host.ReplaceWith(host.Elements().ToArray()); + if (_options.InlineManifests) + { + foreach (var manifest in root.Descendants().Where(IsExternalManifest).ToArray()) InlineManifest(manifest); + } + return root; + } + + private void InlineManifest(XElement registration) + { + var target = registration.Attribute("Source")!.Value; + var id = registration.Attribute("ID")!.Value; + var key = target + "#" + id; + if (!_manifestProcessing.Add(key)) throw new InvalidOperationException($"Manifest cycle: {target}"); + if (!_documents.TryGetValue(target, out var captured)) + { + EnsureWithinRoot(_source, target, _root, "Manifest Source"); + var count = _ordered.Count; + captured = Capture(target, true); + foreach (var added in _ordered.Skip(count)) RewriteReferences(added); + } + var body = SelectManifest(Expand(captured, new HashSet(_pathComparer)), id); + var clone = new XElement(body); + clone.Attribute("Source")?.Remove(); + registration.ReplaceWith(clone); + _manifestProcessing.Remove(key); + } + + private static XElement SelectManifest(XElement document, string id) + { + var manifests = document.Descendants().Where(e => e.Name.LocalName == "Manifest" && e.Attribute("ID") != null).ToArray(); + var exact = manifests.Where(e => string.Equals(e.Attribute("ID")!.Value, id, StringComparison.OrdinalIgnoreCase)).ToArray(); + if (exact.Length == 1) return exact[0]; + throw new InvalidOperationException($"Manifest not found: {id}"); + } + } + + private sealed class DocumentPathComparer : StringComparer + { + internal static readonly DocumentPathComparer Instance = new(); + private static StringComparer For(string? path) => IsHttp(path ?? string.Empty) || !OperatingSystem.IsWindows() + ? StringComparer.Ordinal : StringComparer.OrdinalIgnoreCase; + public override int Compare(string? x, string? y) => For(x).Compare(x, y); + public override bool Equals(string? x, string? y) => For(x).Equals(x, y); + public override int GetHashCode(string obj) => For(obj).GetHashCode(obj); + } +} diff --git a/Ra3.BattleNet.Metadata/Reading/MetadataSourceManifest.cs b/Ra3.BattleNet.Metadata/Reading/MetadataSourceManifest.cs new file mode 100644 index 0000000..1905b5e --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/MetadataSourceManifest.cs @@ -0,0 +1,31 @@ +namespace Ra3.BattleNet.Metadata; + +/// Resolves a package manifest in the declaration's shared source-root context. +public static class MetadataSourceManifest +{ + public static Metadata? Resolve(MetadataReadResult declaration, Metadata package) + { + ArgumentNullException.ThrowIfNull(declaration); + ArgumentNullException.ThrowIfNull(package); + var id = package.Find("Manifest")?.Value; + if (string.IsNullOrWhiteSpace(id)) + return null; + + var registration = declaration.Root.ManifestRegistration(id) + ?? throw new InvalidOperationException($"模组声明找不到文件清单：{id}"); + var source = registration.Get("Source"); + if (string.IsNullOrWhiteSpace(source)) + return registration; + + var leaf = MetadataResourceUri.Resolve(declaration.EntryPath, source); + if (!leaf.IsFile) + throw new InvalidOperationException("本地模组文件清单必须在模组目录内"); + var document = MetadataReader.Read(leaf.LocalPath, new MetadataSourceOptions + { + SourceRoot = declaration.SourceRoot, + InlineManifests = true, + }).Root; + return document.ManifestRegistration(id) + ?? throw new InvalidOperationException($"Manifest identity does not match registration: {id}"); + } +} diff --git a/Ra3.BattleNet.Metadata/Reading/MetadataSourceOptions.cs b/Ra3.BattleNet.Metadata/Reading/MetadataSourceOptions.cs new file mode 100644 index 0000000..5957dbb --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/MetadataSourceOptions.cs @@ -0,0 +1,26 @@ +namespace Ra3.BattleNet.Metadata; + +/// +/// Options for the unified reader. Reading never performs build-time substitutions. +/// +public sealed class MetadataSourceOptions +{ + /// + /// Include 与相对资源路径的允许根目录；缺省时用入口文件所在目录。 + /// 显式指定后入口、Include 与资源路径都必须落在该目录内。 + /// + public string? SourceRoot { get; init; } + + /// 读取完成后是否做登记引用完整性校验：断引用抛 。 + public bool ValidateReferences { get; init; } + + /// + /// 自定义只读源；非空时入口路径交给该源解释，不再当成本地路径或 URL。 + /// 未注入时本地路径用 ，HTTP(S) 入口用 。 + /// + public IMetadataXmlSource? CustomSource { get; init; } + + /// Resolve external manifest bodies. Included full manifests are always retained. + public bool InlineManifests { get; init; } + +} diff --git a/Ra3.BattleNet.Metadata/Reading/MetadataValidator.cs b/Ra3.BattleNet.Metadata/Reading/MetadataValidator.cs new file mode 100644 index 0000000..9c3a1cc --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/MetadataValidator.cs @@ -0,0 +1,113 @@ +namespace Ra3.BattleNet.Metadata; + +/// +/// 元数据引用完整性校验：登记节点（带 ID 的 Image / Markdown / Manifest）是唯一合法引用目标。 +/// 规则与 MetadataBuilder.ValidateHard 的引用部分一致： +/// Icon / Package.Manifest / Post.Content 查登记全集； +/// Post.HeadImage / Style.Logo / Style.Background.Image 只查图片登记。 +/// 只负责断引用，不负责字段缺失、资源文件是否存在等其它校验。 +/// +public static class MetadataValidator +{ + /// + /// 找出全部断引用，每条格式为 {实体/节点} (ID): {字段} -> {引用值}。 + /// ID 比较用 OrdinalIgnoreCase，引用值先 Trim，空值跳过。 + /// + public static IReadOnlyList FindMissingReferences(Metadata metadata) + { + ArgumentNullException.ThrowIfNull(metadata); + + var registrations = metadata.GetAllElements("Image") + .Concat(metadata.GetAllElements("Markdown")) + .Concat(metadata.GetAllElements("Manifest")) + .Where(node => !string.IsNullOrWhiteSpace(node.Get("ID"))); + var allIds = new HashSet(StringComparer.OrdinalIgnoreCase); + var imageIds = new HashSet(StringComparer.OrdinalIgnoreCase); + var missing = new List(); + foreach (var registration in registrations) + { + var id = registration.Get("ID")!; + if (!allIds.Add(id)) + missing.Add($"登记 ID 重复: {id}"); + if (registration.Name == "Image") + imageIds.Add(id); + } + foreach (var entity in metadata.GetAllElements("Application").Concat(metadata.GetAllElements("Mod"))) + { + var entityId = entity.Get("ID"); + var owner = string.IsNullOrWhiteSpace(entityId) ? entity.Name : $"{entity.Name} ({entityId})"; + + AddIfMissing(entity.Find("Icon")?.Value, "Icon", allIds, owner, missing); + + var packages = entity.Find("Packages"); + if (packages != null) + { + foreach (var package in packages.Children.Where(c => c.Name == "Package")) + AddIfMissing(package.Find("Manifest")?.Value, "Manifest", allIds, owner, missing); + } + + var posts = entity.Find("Posts"); + if (posts != null) + { + foreach (var post in posts.Children.Where(c => c.Name == "Post")) + { + // Post 封面只认图片登记：引用 Markdown / Manifest 或不存在都算断引用。 + AddIfMissing(post.Find("HeadImage")?.Value, "Post HeadImage", imageIds, owner, missing); + + var contents = post.Find("Contents"); + if (contents == null) + continue; + + foreach (var content in contents.Children.Where(c => c.Name == "Content")) + AddIfMissing(content.Value, "Post Content", allIds, owner, missing); + } + } + + var links = entity.Find("Links"); + if (links is not null) + { + foreach (var link in links.Children.Where(child => child.Name == "Link")) + AddIfMissing(link.Find("Icon")?.Value, "Link Icon", imageIds, owner, missing); + } + + var style = entity.Find("Style"); + if (style == null) + continue; + + var logo = style.Find("Logo"); + if (logo?.Value != null) + AddIfMissing(logo.Value, "Logo", imageIds, owner, missing); + + var background = style.Find("Background"); + if (background == null) + continue; + + foreach (var image in background.Children.Where(c => c.Name == "Image")) + AddIfMissing(image.Value, "Background Image", imageIds, owner, missing); + } + + return missing; + } + + /// + /// 存在断引用时抛 ，消息首行固定，之后每条断引用一行。 + /// + public static void EnsureReferences(Metadata metadata) + { + var missing = FindMissingReferences(metadata); + if (missing.Count == 0) + return; + + throw new InvalidOperationException("元数据引用校验失败:\n- " + string.Join("\n- ", missing)); + } + + private static void AddIfMissing(string? reference, string field, HashSet registry, string owner, List missing) + { + var value = reference?.Trim(); + if (string.IsNullOrEmpty(value)) + return; + + if (!registry.Contains(value)) + missing.Add($"{owner}: {field} -> {value}"); + } +} diff --git a/Ra3.BattleNet.Metadata/Reading/PhysicalFileXmlSource.cs b/Ra3.BattleNet.Metadata/Reading/PhysicalFileXmlSource.cs new file mode 100644 index 0000000..6452030 --- /dev/null +++ b/Ra3.BattleNet.Metadata/Reading/PhysicalFileXmlSource.cs @@ -0,0 +1,75 @@ +using System.Xml.Linq; + +namespace Ra3.BattleNet.Metadata; + +/// +/// 的本地文件实现：逐个转发 System.IO，构造无副作用、无全局状态。 +/// +public sealed class PhysicalFileXmlSource : IMetadataXmlSource +{ + /// + public bool Exists(string path) + { + if (!MetadataReader.IsHttp(path)) return File.Exists(path); + using var source = new HttpXmlSource(new Uri(new Uri(path), ".")); + return source.Exists(path); + } + + /// + public XDocument LoadDocument(string path) + { + if (!MetadataReader.IsHttp(path)) + { + using var stream = File.OpenRead(path); + using var reader = System.Xml.XmlReader.Create(stream, new System.Xml.XmlReaderSettings + { + DtdProcessing = System.Xml.DtdProcessing.Prohibit, + XmlResolver = null + }); + return XDocument.Load(reader); + } + using var source = new HttpXmlSource(new Uri(new Uri(path), ".")); + return source.LoadDocument(path); + } + + /// + // 作者可能写 \ 分隔的路径；先统一成 / 再交给 Path.Combine，跨平台口径一致 + public string Combine(string basePath, string relativePath) + { + if (MetadataReader.IsHttp(relativePath)) return new Uri(relativePath).AbsoluteUri; + var relative = relativePath.Replace('\\', '/'); + if (!MetadataReader.IsHttp(basePath)) return Path.Combine(basePath, relative); + var directory = basePath.EndsWith('/') ? basePath : basePath + "/"; + return new Uri(new Uri(directory), relative).AbsoluteUri; + } + + /// + public string? GetDirectoryName(string path) => MetadataReader.IsHttp(path) + ? new Uri(new Uri(path), ".").AbsoluteUri : Path.GetDirectoryName(path); + + /// + public string GetRelativePath(string fromPath, string toPath) + { + if (!MetadataReader.IsHttp(fromPath)) + return MetadataReader.IsHttp(toPath) ? toPath : Path.GetRelativePath(fromPath, toPath); + if (!MetadataReader.IsHttp(toPath)) return toPath; + var from = new Uri(fromPath); + var to = new Uri(toPath); + if (!string.Equals(from.Scheme, to.Scheme, StringComparison.OrdinalIgnoreCase) + || !string.Equals(from.Host, to.Host, StringComparison.OrdinalIgnoreCase) || from.Port != to.Port) + return to.AbsoluteUri; + var directory = from.AbsoluteUri.EndsWith('/') ? from.AbsoluteUri : from.AbsoluteUri + "/"; + return new Uri(directory).MakeRelativeUri(to).ToString(); + } + + /// + public string GetFullPath(string path) + { + if (MetadataReader.IsHttp(path)) return new Uri(path).AbsoluteUri; + if (Uri.TryCreate(path, UriKind.Absolute, out var uri) && uri.IsFile) return Path.GetFullPath(uri.LocalPath); + return Path.GetFullPath(path); + } + + /// + public bool IsPathRooted(string path) => MetadataReader.IsHttp(path) || Path.IsPathRooted(path); +} diff --git a/build.sh b/build.sh index bf19b16..4e99694 100644 --- a/build.sh +++ b/build.sh @@ -4,11 +4,9 @@ set -euo pipefail # 默认仅核心；--webp 时核心 CLI 内部串联 Imaging WEBP_FLAG=() -BUILD_ARGS=() for arg in "$@"; do case "$arg" in --webp|--publish|--with-webp) WEBP_FLAG=(--webp) ;; - --content-revision=*) BUILD_ARGS+=("$arg") ;; --help|-h) cat <<'EOF' 用法: bash build.sh [--webp] @@ -16,7 +14,6 @@ for arg in "$@"; do (无参数) 仅核心构建 → ./Output --webp 核心 + Imaging（WebP 与 Hash 更新） --publish 同 --webp - --content-revision=VALUE 指定发布内容版本 EOF exit 0 ;; @@ -34,4 +31,4 @@ export PATH="$DOTNET_ROOT:$PATH" dotnet --version dotnet run --project ./Ra3.BattleNet.Metadata --no-launch-profile -- \ - build --src="./Metadata" --dst="./Output" "${WEBP_FLAG[@]}" "${BUILD_ARGS[@]}" + build --src="./Metadata" --dst="./Output" "${WEBP_FLAG[@]}"