fix: correct LOC calculation, cache integrity, and release health evaluation - #1
Conversation
Two independent problems made verbose mode unsafe to use: - `release:analyze -v` dumped the entire parsed flags object, which includes `--token`, so the plaintext GitLab token landed in logs. Only the token is redacted; other flags keep their debugging value. - `logger.debug()` and `logger.info()` used `console.debug`/`console.info`, which write to **stdout** in Node, not stderr. Running `-v --json` therefore produced output that no JSON parser could read. Both now go to stderr, leaving stdout for command output only. `warn`/`error` were already on stderr and are unchanged. `logger.info` had zero call sites, and all 47 `logger.debug` calls are diagnostic, so no user-visible output depended on the old stream. The logger test asserts which console method is used rather than spying on the streams: vitest replaces `console`, so stream-level spies never fire. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`getMergeRequestChanges` parsed GitLab's `changes_count` as if it were a `"+50 -20"` line-count string. It is not — it is the number of **changed files**. The regex never matched, so the else branch stored the file count in `additions` and left `deletions` permanently 0. Every LOC figure the release analysis reported was therefore wrong, often by one to two orders of magnitude, and the batch-size thresholds were being compared against file counts. The fix reuses what the codebase already had right: `allDiffs()` plus `calculateTotalChanges()` from `src/utils/diff-parser.ts`, which `size-analyzer.ts` has been using correctly all along. The 36 lines of `changes_count` parsing are gone. Signature and return type are unchanged, so no caller needed adjusting, and the API call count is the same as before — only the payload is larger. Truncated diffs now throw instead of silently under-counting. The check keys on GitLab's `collapsed`/`too_large` flags rather than "the diff is empty", because binary files legitimately produce empty diffs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The symptom was one command contradicting itself: the table reported 43 MRs for a release while `--json` reported 0. Root-causing that turned up several independent defects in the release analyzer. Health evaluation never ran at all. `calculateHealthLevelIfNeeded` looked up `config.release_types` by the `name` field while `classifyReleaseType` returns the key, so the lookup always missed and `health_level` was permanently null. Not a configuration problem — the code could not read it. Releases with MRs silently disappeared. The cache stores JSON, so `merged_at` came back as a string; `calculateFreezeDays` called `.getTime()` on it and threw, and the batch processor's `skip` strategy swallowed the error. Dates are now revived on read, and the failures the processor collects are logged with their tag name instead of discarded. Degraded results were cached as if they had succeeded, at two levels, so a transient API failure kept returning wrong numbers for the whole TTL. The inner calls now throw and the outer layer degrades, which keeps `cache.set` on the success path only. Cache keys also carry a schema version so entries written by the old LOC algorithm cannot be read back. Additional fixes in the same area: - Batch size can now be judged by `loc_additions`, which takes priority over `loc_changes`. Summing additions and deletions rates a large technical-debt cleanup the same as a large feature batch; the risk being measured is how much new code shipped. - The overall `metrics.level` considered only MR count, ignoring the LOC dimension entirely. It now takes the worst of the dimensions in play. - Time filtering removed the tag needed as a predecessor anchor, so the oldest release in range lost its MR window. Predecessor lookup now uses an unfiltered pool. - Release readiness assessed synthesized freeze periods. With no predecessor tag, `lastMergeDate` defaults to the tag date, giving `freeze_days = 0`, which `assessFreezePeriod` reports as "same-day release, insufficient test time" — a false alarm. Those releases are now skipped. - `init` silently dropped `loc_additions` when writing config from a template. - Recommendations name the dimension that triggered them instead of always citing MR count. Two contract changes, both deliberate: - `health_level` can now be `null`, meaning not evaluated — either the release type has `evaluate_batch_size` off, or no predecessor tag exists so the MR window cannot be defined. Reporting `healthy` for an unmeasured release is worse than reporting nothing. - Threshold docs now say `<` rather than `<=`. The implementation has always used `< healthy`, so a value exactly equal to `healthy` counts as a warning. The docs were aligned to the code rather than the reverse, since changing the comparison would alter existing `mr_count` behaviour. Boundary tests pin this down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review of this PR turned up three ways the "unmeasured" state still leaked into confident output. All three share a root cause: `lastMergeDate` defaults to the tag date and is only overwritten when MRs are found, so an empty MR list is indistinguishable from a same-day release. **A failed MR list query no longer reports `healthy`.** This one was made worse by the previous commit, not merely left unfixed. Before the key/name lookup was repaired, `calculateHealthLevelIfNeeded` always returned null, so a release whose MR query failed showed `N/A`. With the lookup working, the same degraded path produced 0 MRs, 0 LOC, and a confident `healthy` — and since the error goes to stderr while `--json` goes to stdout, a consumer received a clean, parseable, entirely wrong report. The list fetch now reports whether it failed, and a failed fetch marks the release unevaluated with a warning naming the tag. The release itself is still emitted: the tag existing is a fact, only its batch size is unknown. `getMergeRequestsBetweenReleases` keeps its signature and delegates to a private method that returns the failure flag alongside the events, so the public contract and its existing callers are untouched. **Readiness now skips releases with no MRs in range.** The previous guard covered only "no predecessor tag", but a range containing no merges reaches the same synthesized `freeze_days = 0` — a retagged release, a tag cut from the same commit as its predecessor, or the failed-fetch case above. All of them were reported as "same-day release, insufficient test time". The guard keys on `mr_count`, not on `freeze_days === 0`, so a genuine same-day release still reports critical. The test covering that case now uses a non-zero `mr_count`, which is also what it means: a real zero-day freeze had MRs merged on tag day. **Truncation detection is now tested.** It was the only data-integrity check in the diff-based line counting and had no coverage at all — deleting the entire check left every test green. Added cases for `collapsed` and `too_large`, plus one pinning that an empty diff *without* those flags must not throw, since binary files legitimately produce empty diffs. All three fixes were mutation-tested: reverting each one turns its tests red (1, 1, and 2 failures respectively). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Code review 記錄(
|
變更說明
同一條
release:analyze指令在不同輸出模式下互相矛盾(table 報 43 個 MR、--json報 0),追查後發現十二個獨立缺陷,橫跨安全性與資料正確性。分三個 commit:1. 安全性 —
-v會把整包 flags(含--token)dump 進日誌,GitLab 憑證明文外流;logger.debug/info使用console.debug/info,兩者在 Node 都寫 stdout,導致-v --json的輸出無法被任何 JSON parser 解析。2. LOC 計算 —
getMergeRequestChanges把 GitLab 的changes_count當成"+50 -20"行數格式解析,但它其實是變更檔案數。regex 永不匹配,else 分支把檔案數存進additions、deletions恆為 0。所有 LOC 數字都錯,常低估一到兩個數量級,而批量閾值一直在跟檔案數比較。改用allDiffs()+ 既有的calculateTotalChanges()(size-analyzer.ts早就用對了)。3. 健康度評估與資料完整性 — 三個最嚴重的:
evaluate_batch_size從未生效:用release_types的name欄位比對classifyReleaseType回傳的 key,永遠不匹配,health_level恆為 null。merged_at讀回是字串,calculateFreezeDays對它呼叫.getTime()拋錯,又被 batch 的skip策略吞掉。其餘:
metrics.level只看 MR 數、時間過濾切掉 predecessor anchor、readiness 對合成的freeze_days = 0誤報「當天發布,測試時間不足」、init靜默丟掉loc_additions、新增loc_additions閾值(加總會把清技術債判成和大批新功能同級)。兩個對外契約變更(刻意)
releases[].health_level新增null— 表示未評估:發布類型未啟用evaluate_batch_size,或缺前一個標籤而無法界定 MR 區間。對未測量的 release 回報healthy比回報「不知道」更糟。<=改為<— 實作一直是< healthy,剛好等於 healthy 的值算 warning。這裡是改文件對齊實作,不是改行為(改實作會變更既有mr_count邊界),並加了邊界測試釘住。已知未修
gitlab-client.ts的 MR 查詢是perPage: 100, maxPages: 10搭配orderBy: 'updated_at'——先抓整條分支最近 1000 筆 merged MR,才用merge_commit_sha對 tag 區間過濾。後果是 (a) 分支累積破千後舊 release 靜默低估、(b) 舊 MR 被更新就撞回視窗,同指令不同天可能回不同結果、(c) 錯誤會被(fromSha, toSha)快取固化。這是成功路徑產生錯數字,現有以錯誤為訊號的降級機制偵測不到。實測目前餘裕約 2.3 倍(14 個月區間 440 筆 / 上限 1000),觸發門檻約 32 個月的查詢區間,故本 PR 不處理,另案追蹤。
安全性檢查清單
在提交此 PR 前,請確認:
.env管理所有配置資訊./scripts/check-secrets.sh並通過檢查npm test補充說明(本 repo 是 public,故逐項複驗過):
dev-a、MR IID101、分支dev-a/i18n、hostgitlab.example.com、專案group/project。原本引用的真實 LOC 數字已改為維持相同大小關係的合成值(測試斷言的是「看新增」與「看加總」判斷不同這個行為,不是特定數值)。check-secrets.sh的檔案內容掃描 ✅ 通過。它另報「6 個可疑 commit」,經逐一核對是兩類假陽性:commit 1 的訊息文字描述「修掉 token 洩漏」必然包含 token 一詞,以及.env.example既有的# 格式範例: glpat-...佔位說明。三個 commit 的 diff 對真實密鑰模式命中為 0。glpat-前綴——那會讓check-secrets.sh對該檔案永遠報警,久了會訓練大家忽略掃描器。遮蔽是按欄位名做的,不看 token 格式。.gitlab-analysis.yml已由.gitignore忽略(git check-ignore驗過)。測試計畫
新增 7 個測試檔(44 個測試),全部針對「revert 掉修復也不會被抓到」的缺陷設計:
release-analyzer-regression.test.ts— 走analyzeBatchSize完整路徑,涵蓋 key/name 混淆、快取 JSON hydration、降級不得寫快取、schemaVersion、predecessor poolrelease-health-level.test.ts— 多維取最差、loc_additions優先、閾值邊界(釘住< healthy)release-readiness-unmeasured.test.ts— 未測量發布不進凍結期評估,含一條防過度過濾的守衛gitlab-client-mr-changes.test.ts— 行數必須來自 difflogger.test.ts— 斷言用了哪個 console 方法(vitest 會接管 console,stream 層的 spy 不會觸發)token-redaction.test.ts— 走命令真實流程,確認哨兵值不入日誌且 dump 真的發生過thresholds-schema.test.ts—loc_additionsschema 與 refine已對三個關鍵修復做突變測試(把修復 revert 掉確認測試會紅):key/name 修復 revert 後 regression test 紅而既有 health-level 測試仍綠;readiness 守衛 revert 後 3 紅 2 綠(綠的兩條是防過度過濾的守衛,本來就該不敏感)。
LOC 修復另有 golden acceptance:用獨立的 Python 腳本自行分頁並解析 unified diff,與工具輸出完全吻合。
相關 Issue
無