Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ MindWork AI Studio is a free desktop app for macOS, Windows, and Linux. It provi
- [IONOS](https://cloud.ionos.com/managed/ai-model-hub) (AI Model Hub running open-source models in Germany)
- [LiteLLM](https://www.litellm.ai/) (an AI gateway you run yourself, in front of models from many providers)
- [Hugging Face](https://huggingface.co/) using their [inference providers](https://huggingface.co/docs/inference-providers/index) such as Cerebras, Nebius, Sambanova, Novita, Hyperbolic, Together AI, Fireworks, Hugging Face
- Self-hosted models using [llama.cpp](https://github.com/ggerganov/llama.cpp), [ollama](https://github.com/ollama/ollama), [LM Studio](https://lmstudio.ai/), and [vLLM](https://github.com/vllm-project/vllm)
- Self-hosted models using [llama.cpp](https://github.com/ggerganov/llama.cpp), [ollama](https://github.com/ollama/ollama), [LM Studio](https://lmstudio.ai/), [vLLM](https://github.com/vllm-project/vllm), and [llmman](https://github.com/llmmanorg/llmman)
- [Groq](https://groq.com/)
- [Fireworks](https://fireworks.ai/)
- For scientists and employees of research institutions, we also support [Helmholtz](https://helmholtz.cloud/services/?serviceID=d7d5c597-a2f6-4bd1-b71e-4d6499d98570) and [GWDG](https://gwdg.de/services/application-services/ai-services/) AI services. These are available through federated logins like eduGAIN to all 18 Helmholtz Centers, the Max Planck Society, most German, and many international universities.
Expand Down
4 changes: 2 additions & 2 deletions app/MindWork AI Studio/Plugins/configuration/plugin.lua
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ CONFIG["LLM_PROVIDERS"] = {}
-- ["InstanceName"] = "<user-friendly name for the combination of server and model>",
-- ["UsedLLMProvider"] = "SELF_HOSTED",
--
-- -- Allowed values for Host are: LM_STUDIO, LLAMACPP, OLLAMA, and VLLM
-- -- Allowed values for Host are: LM_STUDIO, LLAMACPP, OLLAMA, VLLM, and LLMMAN
-- ["Host"] = "OLLAMA",
-- ["Hostname"] = "<https address of the server>",
--
Expand Down Expand Up @@ -218,7 +218,7 @@ CONFIG["EMBEDDING_PROVIDERS"] = {}
-- ["Name"] = "<user-friendly name for the embedding provider>",
-- ["UsedLLMProvider"] = "SELF_HOSTED",
--
-- -- Allowed values for Host are: LM_STUDIO, LLAMACPP, OLLAMA, and VLLM
-- -- Allowed values for Host are: LM_STUDIO, LLAMACPP, OLLAMA, VLLM, and LLMMAN
-- ["Host"] = "OLLAMA",
-- ["Hostname"] = "<https address of the server>",
--
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -546,6 +546,7 @@ public static bool CanLoadModels(this LLMProviders provider, Host host, string?
case Host.OLLAMA:
case Host.LM_STUDIO:
case Host.VLLM:
case Host.LLMMAN:
return true;
}
}
Expand Down
3 changes: 3 additions & 0 deletions app/MindWork AI Studio/Provider/SelfHosted/Host.cs
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,7 @@ public enum Host
WHISPER_CPP,
OLLAMA,
VLLM,

// llmman, local runner with OpenAI-compatible /v1/ routes: https://github.com/llmmanorg/llmman
LLMMAN,
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ public static class HostExtensions
Host.WHISPER_CPP => "whisper.cpp",
Host.OLLAMA => "ollama",
Host.VLLM => "vLLM",
Host.LLMMAN => "llmman",

_ => "Unknown",
};
Expand Down Expand Up @@ -47,6 +48,7 @@ public static bool IsChatSupported(this Host host)
case Host.VLLM:
case Host.LM_STUDIO:
case Host.LLAMA_CPP:
case Host.LLMMAN:
return true;
}
}
Expand All @@ -58,6 +60,7 @@ public static bool IsEmbeddingSupported(this Host host)
case Host.LM_STUDIO:
case Host.OLLAMA:
case Host.VLLM:
case Host.LLMMAN:
return true;

default:
Expand All @@ -78,6 +81,7 @@ public static bool IsTranscriptionSupported(this Host host)
default:
case Host.LM_STUDIO:
case Host.LLAMA_CPP:
case Host.LLMMAN:
return false;
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ namespace AIStudio.Provider.SelfHosted;
public override string InstanceName { get; set; } = "Self-hosted";

/// <inheritdoc />
public override bool HasModelLoadingCapability => host is Host.OLLAMA or Host.LM_STUDIO or Host.VLLM or Host.LLAMA_CPP;
public override bool HasModelLoadingCapability => host is Host.OLLAMA or Host.LM_STUDIO or Host.VLLM or Host.LLAMA_CPP or Host.LLMMAN;

/// <inheritdoc />
public override async IAsyncEnumerable<ContentStreamChunk> StreamChatCompletion(Provider.Model chatModel, ChatThread chatThread, SettingsManager settingsManager, [EnumeratorCancellation] CancellationToken token = default)
Expand Down Expand Up @@ -102,6 +102,7 @@ public override async Task<ModelLoadResult> GetTextModels(string? apiKeyProvisio
case Host.LM_STUDIO:
case Host.OLLAMA:
case Host.VLLM:
case Host.LLMMAN:
var result = await this.LoadModels(SecretStoreType.LLM_PROVIDER, apiKeyProvisional, token);
return result with
{
Expand Down Expand Up @@ -133,6 +134,7 @@ public override async Task<ModelLoadResult> GetEmbeddingModels(string? apiKeyPro
case Host.LM_STUDIO:
case Host.OLLAMA:
case Host.VLLM:
case Host.LLMMAN:
var result = await this.LoadModels(SecretStoreType.EMBEDDING_PROVIDER, apiKeyProvisional, token);
return result with
{
Expand Down
1 change: 1 addition & 0 deletions app/MindWork AI Studio/wwwroot/changelog/v26.10.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- Added a way for IT departments to allow mailboxes only on the mail servers of your organization. AI Studio then connects to no other server, and the AI no longer reads a mailbox somebody added elsewhere.
- Added a setting to share which features of AI Studio you use with the operators of self-hosted AI servers, usually your organization. When you switch it on, requests to self-hosted providers and LiteLLM name the feature that sent them, such as the chat, an assistant, or an agent, together with the version of AI Studio and your operating system. Cloud providers never receive these details, and AI Studio sends nothing to its developers. The setting is off by default, and you find it in the app settings. Thanks, Dominic Neuburg (`donework`), for this contribution.
- Added a way for IT departments to let requests to the self-hosted AI servers of your organization name the feature of AI Studio that sent them, either for everybody or as a default that users may change. This shows which features are used. The Enterprise IT documentation explains what is sent and what to clarify before you switch it on, for example, with your works council.
- Added [llmman](https://github.com/llmmanorg/llmman) as a host option for self-hosted chat and embedding providers. llmman is a local model runner; use `http://localhost:17434` as the hostname.
- Improved the Read Web Page tool. When you have not allowed the AI to choose web addresses freely, the tool now refuses every address that does not appear in your chat or in what the tools returned, instead of only asking the AI not to read it.
- Improved the installation on macOS: AI Studio is now signed and checked by Apple, so it opens like any other app. You no longer need the Terminal.
- Improved the installation on Windows: AI Studio is now signed, so Windows shows who published it. For a while, Windows may still ask you to confirm the installation until it knows the app better.
Expand Down
Loading