Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/development/i18n.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ npm run check:i18n
| --- | --- | --- |
| 通用、错误 | [common.ts](../../src/i18n/messages/zh-CN/common.ts) | [common.ts](../../src/i18n/messages/en-US/common.ts) |
| 设置、主题 | [settings.ts](../../src/i18n/messages/zh-CN/settings.ts) | [settings.ts](../../src/i18n/messages/en-US/settings.ts) |
| 首次设置 | [firstRun.ts](../../src/i18n/messages/zh-CN/firstRun.ts) | [firstRun.ts](../../src/i18n/messages/en-US/firstRun.ts) |
| 画布与项目 | [canvas.ts](../../src/i18n/messages/zh-CN/canvas.ts) | [canvas.ts](../../src/i18n/messages/en-US/canvas.ts) |
| 模型与服务 | [models.ts](../../src/i18n/messages/zh-CN/models.ts) | [models.ts](../../src/i18n/messages/en-US/models.ts) |
| 画布外壳 | [canvasShell.ts](../../src/i18n/messages/zh-CN/canvasShell.ts) | [canvasShell.ts](../../src/i18n/messages/en-US/canvasShell.ts) |
Expand Down
1 change: 1 addition & 0 deletions docs/development/ui-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Design for the time between a user action and the point when they can continue w

- **Keep direct manipulation responsive.** Update typing, selection, dragging, panning, and scrubbing on the next available frame. Keep synchronous I/O and expensive work out of input handlers. Coalesce transient updates and persist edits at meaningful boundaries.
- **Keep navigation focused.** Load the data needed for the destination to become usable. Defer previews, secondary panels, and capability discovery until needed. Reuse valid results, combine duplicate requests, and cancel view-specific work when the view changes.
- **Avoid redundant clicks.** If a user action already identifies what to open or use, go directly there. Do not show an intermediate page whose only forward action repeats the same intent, or require a second click to select the single item the user just saved for that action. Keep steps that collect input, offer a meaningful choice, or confirm a consequential action.
- **Protect foreground actions.** Limit background concurrency so discovery, polling, and media loading leave capacity for navigation and edits. Preserve focus, selection, and drafts across background updates.
- **Show meaningful progress.** Acknowledge actions promptly. For work that takes around a second or longer, show the current stage and keep unrelated controls usable. Offer cancellation when the operation supports it.
- **Preserve correctness.** Measure readiness after required state is loaded and controls are actionable. Keep save, conflict, and task status tied to authoritative receipts.
Expand Down
2 changes: 2 additions & 0 deletions docs/guides/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

## Connect models

On the welcome screen, select **Get started**. Set up a language model for the built-in assistant or choose an existing harness. Then check one or more tasks. Each checked task shows its saved models or opens its setup form if you have none. **Start creating** places a node for each checked task on the same canvas. **Import media** opens the file picker. **Open an example canvas** opens an editable example project directly, with download progress in step two. Cancelling keeps your task selections. You can add more models in Settings anytime.

For external agent harnesses (Codex, OpenCode, Claude Code, DeepSeek Harness and JiuwenSwarm), OpenDirector checks the backend's `PATH` first, then common CLI installation directories. These include user bins, npm/nvm, Bun, Volta, Windows Scoop and Python Scripts, and macOS Homebrew and user Python bins. It also checks Codex desktop installs, OpenCode's `.opencode/bin`, Claude's `.claude/local`, and configured npm, pipx and uv bin directories. It checks known directories without scanning the whole disk. Finding a CLI does not establish authentication or configuration readiness. For other locations, add the CLI directory to the backend's `PATH` and restart the backend. The Default agent continues to use the bundled OpenCode.

Click the gear at the bottom left to open **Settings**, then open **Model connections**. Add models for chat, image, video, or audio generation as needed.
Expand Down
2 changes: 2 additions & 0 deletions docs/guides/user-guide.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

## 接入模型

首次打开时,在欢迎页选择「开始设置」。为内置助手配置一个语言模型,或选择已有的 Harness。接着勾选一项或多项任务。每个勾选项可以选用已有模型,缺模型时直接显示对应的配置表单。点击「开始创作」后,画布会为每项任务添加一个节点。点「导入素材」可直接选文件,点「打开示例画布」可直接进入可编辑的示例项目,下载进度在第二步显示。取消时会保留已勾选的任务。之后可随时在设置中添加其他模型。

选择外部 Agent harness(Codex、OpenCode、Claude Code、DeepSeek Harness 和 JiuwenSwarm)时,OpenDirector 优先检查后端的 `PATH`,再查找常见 CLI 安装目录,包括用户命令目录、npm/nvm、Bun、Volta、Windows Scoop 与 Python Scripts,以及 macOS Homebrew 和用户 Python 命令目录。同时检查 Codex 桌面版、OpenCode 的 `.opencode/bin`、Claude 的 `.claude/local`,以及配置的 npm、pipx 和 uv 命令目录。检测仅访问已知目录,不扫描整个磁盘;找到 CLI 后仍需检查认证与配置是否就绪。其他安装位置可加入后端的 `PATH`,然后重启后端。Default Agent 仍使用应用内置的 OpenCode。

点击左下角齿轮进入“设置”,打开“模型接入”,按用途添加对话、图片、视频或音频模型。
Expand Down
4 changes: 2 additions & 2 deletions scripts/check-browser-manifest.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ for (const entry of entries) {
const FIXTURE_LANES = ["extendedFixture", "extendedHttpFixture"];
const KNOWN_LANES = new Set(["gate", "extended", ...FIXTURE_LANES, "production", "perf", "manual"]);
// Static harness pages do not open the workspace, so they do not see the welcome dialog.
// The bilingual defaults test exercises the welcome dialog itself.
const welcomeExempt = new Set(["assistant_activity.mjs", "bilingual_defaults.mjs"]);
// These scenarios exercise the welcome dialog itself.
const welcomeExempt = new Set(["assistant_activity.mjs", "bilingual_defaults.mjs", "first_run.mjs"]);
for (const entry of entries) {
const text = entry.file === undefined ? "" : fs.readFileSync(path.join(browserDir, entry.file), "utf8");
const importsSrc = text.includes('"/src/');
Expand Down
1 change: 1 addition & 0 deletions scripts/run-browser-suite.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ const workflows = [
args: ["--full-creation"],
env: { PYTHON: "python" },
},
{ name: "first-run setup", script: "tests/browser/first_run.mjs" },
];

const failures = [];
Expand Down
3 changes: 3 additions & 0 deletions src/assets/welcome/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Welcome artwork

These are two screenshots of the actual OpenDirector canvas. For each locale, six clips from [*We Can Still Win*](../../../docs/guides/assets/dota2-short-film.mp4) were imported into an isolated sample project and added to its timeline. The screenshots are encoded as WebP without compositing and bundled to work offline.
Binary file added src/assets/welcome/canvas-en-US.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added src/assets/welcome/canvas-zh-CN.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 6 additions & 1 deletion src/backend/creationProjection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ export class CreationProjection {
private importHandles = new Map<string, string>();
private timelineImportSources = new Map<string, string>();
private masks = new Map<string, Promise<string>>();
private pendingSpecIds = new Map<string, string>();
readonly client: CreationClient;
constructor(client: CreationClient) {
this.client = client;
Expand Down Expand Up @@ -936,8 +937,11 @@ export class CreationProjection {
node: CanvasNode,
current?: GenerationSpec,
): GenerationSpecInput {
const id = current?.id ?? this.pendingSpecIds.get(node.id) ?? crypto.randomUUID();
// Preserve the batch payload when an unacknowledged draft save is retried.
if (!current) this.pendingSpecIds.set(node.id, id);
const common = {
id: current?.id ?? crypto.randomUUID(),
id,
projectId: this.client.id,
revision: current?.revision ?? 1,
purpose: current?.purpose ?? "draft",
Expand Down Expand Up @@ -2109,6 +2113,7 @@ export class CreationProjection {
// Business acknowledgement cannot be undone by a view preference failure.
this.basis = this.client.snapshot;
this.baseline = document;
this.pendingSpecIds.clear();
if (options.saveView === false) return operations.length > 0;
await this.saveView(document);
return operations.length > 0;
Expand Down
6 changes: 5 additions & 1 deletion src/backend/models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ export function useBackendModels() {
const [models, setModels] = useState<ModelConfig[]>([]);
const modelsRef = useRef<ModelConfig[]>([]);
const epoch = useRef(0);
const latestRefresh = useRef(0);
const querySequence = useRef(0);
const latestQuery = useRef(new Map<string, number>());
const alive = useRef(true);
Expand Down Expand Up @@ -136,6 +137,8 @@ export function useBackendModels() {
}, []);
const refresh = useCallback(async () => {
const initialEpoch = ++epoch.current;
latestRefresh.current = initialEpoch;
setLoading(true);
try {
const [list, general] = await Promise.all([
request<ModelConfig[]>("/model-configs").then((list) => {
Expand All @@ -156,7 +159,7 @@ export function useBackendModels() {
setConnected(false);
setError(errorMessage(failure));
} finally {
if (alive.current) setLoading(false);
if (alive.current && initialEpoch === latestRefresh.current) setLoading(false);
}
}, [storeModels]);
const refreshAvailability = useCallback(
Expand Down Expand Up @@ -302,6 +305,7 @@ export function useBackendModels() {
);
await refresh();
void refreshAvailability(saved.map((model) => model.id));
return saved;
};
const removeService = async (id: string) => {
epoch.current += 1;
Expand Down
20 changes: 12 additions & 8 deletions src/canvas/ModelPicker.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { ModelLogo } from "./ModelLogo";
import { useState, type ReactNode } from "react";
import { useId, useState, type ReactNode } from "react";
import { CanvasIcon } from "./CanvasIcon";
import { useT } from "../i18n/I18nProvider.tsx";

Expand Down Expand Up @@ -40,6 +40,7 @@ export function ModelPicker({
fetchLabel?: string;
}) {
const t = useT();
const manualFieldId = useId();
const [query, setQuery] = useState("");
const [onlySelected, setOnlySelected] = useState(false);
const [manualOpen, setManualOpen] = useState(false);
Expand Down Expand Up @@ -114,12 +115,12 @@ export function ModelPicker({
</div>
{manualOpen && (
<div className="cw-model-manual">
<label htmlFor="manual-model-id">
<label htmlFor={manualFieldId}>
{t("generation.picker.modelId")}
</label>
<div>
<input
id="manual-model-id"
id={manualFieldId}
autoFocus
value={manualId}
maxLength={200}
Expand Down Expand Up @@ -269,14 +270,17 @@ export function ModelPicker({
type="button"
className="cw-settings-button cw-settings-button-quiet cw-model-row-test"
disabled={disabled || !canFetch}
aria-busy={results[id]?.status === "running" || undefined}
aria-label={t("generation.picker.testAria", { id })}
onClick={() => onTest(id)}
>
{results[id]?.status === "running"
? t("generation.picker.testing")
: results[id]?.status === "passed"
? results[id].message
: t("generation.picker.test")}
<span key={results[id]?.status ?? "idle"} className="cw-model-row-result">
{results[id]?.status === "running"
? t("generation.picker.testing")
: results[id]?.status === "passed"
? results[id].message
: t("generation.picker.test")}
</span>
</button>
)}
{selected.includes(id) && renderDetails?.(id)}
Expand Down
69 changes: 43 additions & 26 deletions src/canvas/ModelServiceForm.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { DraftAvailabilityCheck } from "./DraftAvailabilityCheck";
import { ProviderPicker } from "./ProviderPicker";
import { useId, useState, type FormEvent } from "react";
import { useId, useLayoutEffect, useState, type FormEvent, type Ref } from "react";
import { errorMessage } from "../backend/api.ts";
import {
MODEL_KINDS,
Expand All @@ -25,6 +25,7 @@ import { CanvasIcon } from "./CanvasIcon";
import { ModelPicker, type ModelTestResult } from "./ModelPicker";
import { useT } from "../i18n/I18nProvider";
import type { MessageKey } from "../i18n/types";
import { firstRunModelKind, type FirstRunCapability } from "./firstRunModels";

export type SettingsModelKind = Exclude<ModelKind, "vision">;
const PROTOCOLS: Record<
Expand Down Expand Up @@ -58,15 +59,25 @@ export function ModelServiceForm({
kind,
existing,
catalog,
capability,
onClose,
onSaved,
cancelButtonRef,
onLockChange,
}: {
kind: SettingsModelKind;
existing?: ModelConfig;
catalog: BackendModels;
capability?: FirstRunCapability;
onClose: () => void;
onSaved?: (models: ModelConfig[]) => void;
cancelButtonRef?: Ref<HTMLButtonElement>;
onLockChange?: (capability: FirstRunCapability, locked: boolean) => void;
}) {
const t = useT();
const connectionFieldId = useId();
const audioPreset = capability === "lyrics_to_music" ? "vllm-omni-music" : "vllm-omni-speech";
const initialProtocol = capability === "lyrics_to_music" ? "vllm_omni_music" : PROTOCOLS[kind][0][0];
const [connection, setConnection] = useState(existing);
const [presetId, setPresetId] = useState(
existing
Expand All @@ -76,7 +87,7 @@ export function ModelServiceForm({
: kind === "image"
? "vllm-omni-image"
: kind === "audio"
? "vllm-omni-speech"
? audioPreset
: "custom",
);
const [modelOptions, setModelOptions] = useState<
Expand All @@ -93,7 +104,7 @@ export function ModelServiceForm({
protocol:
existing && isSegmentationModel(existing)
? "vllm_omni_segmentation"
: (existing?.protocol ?? PROTOCOLS[kind][0][0]),
: (existing?.protocol ?? initialProtocol),
authMode: existing?.authMode ?? "none",
proxyMode: existing ? (existing.proxyMode ?? "system") : "direct",
videoAdapter: existing?.videoAdapter ?? "auto",
Expand Down Expand Up @@ -124,6 +135,13 @@ export function ModelServiceForm({
const [showKey, setShowKey] = useState(false);
const [connectionOpen, setConnectionOpen] = useState(!existing);
const locked = busy || discovering || testing;
useLayoutEffect(() => {
if (!capability) return;
onLockChange?.(capability, locked);
return () => onLockChange?.(capability, false);
}, [capability, locked, onLockChange]);
const continuesSetup = capability === "text_response" && Boolean(onSaved);
const selectedTestFailed = continuesSetup && selectedModels.some((id) => results[id]?.status === "failed");
const segmentation = value.protocol === "vllm_omni_segmentation";
const depth = value.protocol === "vllm_omni_depth";
const score = value.protocol === "vllm_omni_score";
Expand All @@ -136,6 +154,8 @@ export function ModelServiceForm({
value.url.trim() &&
(value.authMode === "none" || value.apiKey.trim() || hasSavedKey),
);
const canSave = canFetch && selectedModels.length > 0 && selectedModels.length <= 200 &&
!selectedTestFailed && !selectedModels.some((id) => modelErrors[id]);
const update = (patch: Partial<ModelConfigInput>) => {
setValue((current) => ({
...current,
Expand All @@ -148,18 +168,22 @@ export function ModelServiceForm({
} as const)
: {}),
}));
if (
const connectionChanged =
patch.url !== undefined ||
patch.apiKey !== undefined ||
patch.protocol !== undefined ||
patch.connectionType !== undefined ||
patch.authMode !== undefined
) {
patch.authMode !== undefined;
if (connectionChanged) {
setModelIds(selectedModels);
setLoaded(false);
setDiscoveryError("");
setResults({});
}
if (
connectionChanged ||
patch.proxyMode !== undefined ||
patch.capabilityAutoCheck !== undefined
) setResults({});
if (patch.protocol !== undefined && patch.protocol !== value.protocol) {
setModelOptions(
Object.fromEntries(
Expand Down Expand Up @@ -223,26 +247,22 @@ export function ModelServiceForm({
};
const submit = async (event: FormEvent) => {
event.preventDefault();
if (
!selectedModels.length ||
locked ||
selectedModels.some((id) => modelErrors[id])
)
return;
if (locked || !canSave) return;
setBusy(true);
setError("");
try {
const { inputModalities, videoAdapter, videoCapabilities, ...shared } =
value;
await catalog.saveMany(
const saved = await catalog.saveMany(
shared,
selectedModels,
connection?.id,
Object.fromEntries(
selectedModels.map((id) => [id, optionsFor(id)]),
),
);
onClose();
if (onSaved) onSaved(saved);
else onClose();
} catch (failure) {
setError(errorMessage(failure));
} finally {
Expand Down Expand Up @@ -354,6 +374,7 @@ export function ModelServiceForm({
<div className="cw-model-editor-scroll">
<ProviderPicker
kind={kind}
allowProtocol={capability ? (protocol) => firstRunModelKind({ kind, protocol }, capability) : undefined}
value={presetId}
disabled={locked}
onChange={chooseProvider}
Expand Down Expand Up @@ -449,7 +470,7 @@ export function ModelServiceForm({
disabled={locked}
>
<div className="cw-model-key-heading">
<label htmlFor="service-api-key">API Key</label>
<label htmlFor={`${connectionFieldId}-key`}>API Key</label>
<label className="cw-model-no-auth">
<input
type="checkbox"
Expand All @@ -465,7 +486,7 @@ export function ModelServiceForm({
</div>
<div className="cw-model-key-input">
<input
id="service-api-key"
id={`${connectionFieldId}-key`}
type={showKey ? "text" : "password"}
disabled={value.authMode === "none"}
required={value.authMode === "bearer" && !hasSavedKey}
Expand Down Expand Up @@ -555,9 +576,10 @@ export function ModelServiceForm({
)}
</div>
<footer className="cw-model-editor-footer">
<div className="cw-model-save-status" role={error ? "alert" : "status"}>
<div className="cw-model-save-status" role={error || selectedTestFailed ? "alert" : "status"}>
{error ||
selectedModels.map((id) => modelErrors[id]).find(Boolean) ||
(selectedTestFailed ? t("firstRun.modelTestFailed") : "") ||
(selectedModels.length > 200
? t("models.service.tooManyModels")
: selectedModels.length
Expand All @@ -567,6 +589,7 @@ export function ModelServiceForm({
: t("models.service.selectAtLeastOne"))}
</div>
<button
ref={cancelButtonRef}
className="cw-settings-button"
type="button"
disabled={locked}
Expand All @@ -577,15 +600,9 @@ export function ModelServiceForm({
<button
className="cw-settings-button cw-settings-button-primary"
type="submit"
disabled={
locked ||
!canFetch ||
!selectedModels.length ||
selectedModels.length > 200 ||
selectedModels.some((id) => modelErrors[id])
}
disabled={locked || !canSave}
>
{busy ? t("models.service.saving") : t("models.service.save")}
{t(busy ? "models.service.saving" : continuesSetup ? "models.service.saveAndContinue" : "models.service.save")}
</button>
</footer>
</form>
Expand Down
Loading
Loading