Skip to content

feat(studio): split a picture into regions with KohakuGBC - #82

Merged
xycld merged 1 commit into
mainfrom
feat/gbc-regions
Nov 10, 2026
Merged

xycld merged 1 commit into
mainfrom
feat/gbc-regions

Conversation

@xycld

@xycld xycld commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

What and why

KohakuGBC's panel was removed in the 11-04 UI rebuild (6ebea87). Its services and IPC stayed in the app, but nothing called them. This brings it back in the shape of the approved design (https://claude.ai/artifact/Q8dumU4DBFpTmneaqvipwB). All five decisions there were taken as recommended.

  • 拆成区域 in a picture's menu (an imported 图片, a result, or a run on a 画面's face). A new 画面 appears to the right, shaped like the picture and not wired to it. It shows the picture faintly with 正在看图… and 停止 (or 正在下载看图模型 on the first run), then fills with the regions and the picture's caption as its description.
    • 停止, a failure, or deleting the 画面 ends the run, and a 画面 still untouched is removed again.
    • At most 12 regions are kept, the largest first. They come from the objects the picture's caption names (top-level entity vertices).
    • While it cannot run, the entry stays in the menu and names why (先给助手加一个模型 / 看图还没装好); choosing it opens 设置 › 功能.
  • 设置 › 功能 › 看图拆区域 (off by default):
    • The 助手 model that reads pictures (only OpenAI-compatible or Anthropic with a key).
    • 安装: the newest commit of the default branch, its .venv, PyTorch from the trainer's index, then pip install -e .[torch]. KohakuGBC is not on PyPI.
    • 高级: how objects are found (OWLv2 / SAM 3) and a version picker (branch + commit), as in 训练.
  • Main process. GBC processes now have an owner. 停止 (gbc:cancel) and quitting (stopGbcProcesses on before-quit) end them; the install is abortable too. Progress goes to the card (event:gbc-parse, from Hugging Face download bars), and the status check is cached. runStream gains signal and splitCarriageReturn.

How it was verified

  • pnpm lint (0 errors, the same 12 warnings as main), pnpm format:check, pnpm typecheck, pnpm test:unit (3544 passed). New tests:
    • shared/gbc.test.ts: graph to regions, and the reasons it can't run.
    • main/services/gbc.test.ts: a fake gbc process covering the config, progress, 停止, quit and failures.
    • renderer/studio/pictureRegions.test.ts: the new 画面's size and placement, and how regions are applied.
    • run.test.ts: abort and carriage-return splitting.
  • pnpm build && pnpm test:e2e (345 passed). New e2e/specs/gbc.e2e.ts covers the menu with the feature off and on, filling a 画面, 停止, a failure, the not-ready entry, and setup in 设置. The visual audit now includes the GBC section in 设置 › 功能.
  • Light and dark checked from the E2E screenshots: gbc-picture-menu-light, gbc-regions-light, gbc-reading-dark, gbc-settings-light, gbc-settings-dark.
  • Docs updated: docs/reference/architecture.md (GBC section), docs/reference/studio-node-board.md (picture actions), docs/how-to/setup.md (「功能」 row), docs/reference/glossary.md.

Platforms affected

  • Platform-independent (the install uses each platform's venv layout; PyTorch's CUDA index on Windows/Linux with NVIDIA)

Notes for the reviewer

  • Not run against real KohakuGBC on a GPU. The archived notes say this was never done before either. Two parts follow GBC's README and ml-gbc's graph format rather than a real run: which vertices become regions, and which captions are used (short for regions, detail for the description).
  • At rest, regions on the board show as boxes without their words. That is the board's current region style, not this change.
  • gbc:run, gbc:list-images and gbc:thumbnails (folders and sidecar files, for training captions) still have no UI. Those are a later round, as decided.

拆成区域 in a picture's menu reads the picture with KohakuGBC and fills a new 画面 beside it with the regions and description it finds. 设置 › 功能 gets a 看图拆区域 switch (off by default) with the 助手 model that reads pictures, a one-step install, and a version picker under 高级.

GBC runs are now owned: 停止 and quitting end them, progress reaches the card, and the install runs PyTorch from the trainer's index before the version itself, since KohakuGBC is not on PyPI.
@xycld
xycld added this pull request to the merge queue Nov 10, 2026
Merged via the queue into main with commit ae28573 Nov 10, 2026
5 checks passed
@xycld
xycld deleted the feat/gbc-regions branch November 10, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant