Skip to content

fix(download): send User-Agent on outbound HTTP - ModelScope CDN 403 on every model download - #245

Merged
Atom112 merged 1 commit into
devfrom
dev-fix-download-user-agent
Oct 30, 2026
Merged

Atom112 merged 1 commit into
devfrom
dev-fix-download-user-agent

Conversation

@Atom112

@Atom112 Atom112 commented Oct 30, 2026

Copy link
Copy Markdown
Owner

问题

本地模型下载页对 ModelScope 模型全部失败:

下载失败:下载 DeepCoder-14B-Preview-Q4_K_M.gguf 失败(已尝试所有来源): HTTP 403 Forbidden

同一台机器网络正常,浏览器从 modelscope.cn 网页下载同一文件毫无问题。

根因

不是网络,是请求缺 User-Agent:

  1. /models//resolve// 302 到对象存储 CDN cdn-lfs-cn-1.modelscope.cn;
  2. 该 CDN 前置阿里云 WAF(响应带 acw_tc cookie)对缺失 UA 的取流请求直接 403;
  3. reqwest 默认不发 User-Agent(curl、浏览器都发——所以网页下载正常);
  4. ModelScope 的 JSON API(/api/v1/dolphin/models、/repo/files)不校验 UA,所以市场能搜到、能列文件,只有真正取文件那一跳全灭——现象因此非常像"网络/权限"问题。

实测对证(同一 URL、同一 Range,本机)

客户端 结果
裸 reqwest::Client(无 UA,即修复前行为) 403 Forbidden
新客户端(UA AIO-Desktop/1.0.0-alpha.6) 206 Partial Content,Content-Range: bytes 0-1023/8988111168

改动

  • 新增 core/http.rs:统一出站身份 user_agent() = AIO-Desktop/<包版本> + client_builder() 起手式;模块头注释记录这个坑与「新增出站客户端必须带 UA」的约定
  • 接入三处此前裸起的客户端:本地 GGUF 下载器、市场元数据客户端(HF 镜像 / ModelScope 列表)、嵌入+重排模型清单下载器
  • 错误可见性:失败原因改为按来源逐条保留(此前只留最后一个来源,无法判断是哪个源在拒);403 单独给出「该来源拒绝访问(通常是 CDN/WAF 拦截或仓库受限)」双语文案;请求里既无 hfRepo 也无 msRepo 时给出「没有可用的下载源」而不是空原因
  • 文档:AGENTS.md 后端目录树补 core/http.rs;CHANGELOG 记入 ## [Unreleased]

测试

  • core::http +2:UA 自报产品名与版本;client_builder 起手的客户端确实发出 UA(本地最小 HTTP 服务器抓请求头,不外网)
  • local_model_download +1:失败原因按来源保留、相邻重复合并
  • 全量 npm run verify 全绿:i18n 1195 键、docs 37 文件、gates rgba=173/any=77 持平、prettier/eslint/tsc 干净、93 FE 测试、vite build;cargo fmt + clippy -D warnings + 473 Rust 测试通过 / 0 失败

行为影响

  • ModelScope 下载恢复;同一修复对 HF 系(hf-mirror / huggingface.co)与嵌入/重排模型下载同样生效(这两处原本也可能被上游 WAF 拒)
  • 无 UI 文案、无命令/事件契约变更

Fixes 用户实装反馈(本地模型下载 403)

…on every download

Every ModelScope model download failed with "403 Forbidden" while the browser
downloaded the same file fine. Root cause is not the network: reqwest sends NO
User-Agent header by default, and the object-store CDN that ModelScope 302s to
(cdn-lfs-cn-1.modelscope.cn, Aliyun WAF, acw_tc cookie) rejects UA-less stream
requests with 403. ModelScope's JSON APIs do not check the UA, so market search
and file listing worked and only the actual byte transfer died.

Measured with the same URL and Range on this machine:
- bare reqwest::Client (no UA) -> 403 Forbidden
- via the new client (AIO-Desktop/1.0.0-alpha.6) -> 206 Partial Content,
  Content-Range: bytes 0-1023/8988111168

Changes:
- core/http.rs: single outbound identity (user_agent() = AIO-Desktop/<version>,
  client_builder() preloaded with it); module doc records the trap
- wire it into the three previously bare clients: local GGUF downloader, market
  metadata client (HF mirror / ModelScope lists), embedding+rerank manifest
  downloader
- failure reporting: keep one reason per source instead of only the last one,
  give 403 its own actionable bilingual message, and say "no usable source"
  when the request carries neither hfRepo nor msRepo (was an empty reason)
- tests: core::http +2 (UA content; a local minimal HTTP server proves the
  client really sends it - no network), local_model_download +1 (failure
  aggregation keeps each source once)

Verification: npm run verify green (i18n 1195 keys, docs 37 files, gates
rgba=173/any=77 held, prettier/eslint/tsc clean, 93 FE tests, vite build;
cargo fmt + clippy -D warnings + 473 Rust tests passed / 0 failed).
@Atom112
Atom112 merged commit b24c095 into dev Oct 30, 2026
8 checks passed
@Atom112
Atom112 deleted the dev-fix-download-user-agent branch October 30, 2026 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant