Echo - Simple LLM Client Library

A lightweight Go library for interacting with various LLM providers with a simple, unified API.

Supported Providers

OpenAI
Anthropic
Google
OpenRouter (via OpenAI-compatible API)
xAI (Grok)
TypeSafe (Jev) - decision calls only, see Decision Calls
Local CLI tools (Claude Code, Codex, Gemini) - opt-in, see Local CLI Providers

Installation

go get github.com/mkozhukh/echo

Quick Start

Universal Client

The NewCommonClient function creates a client that auto-configures providers from API keys:

package main

import (
    "context"
    "fmt"
    "github.com/mkozhukh/echo"
)

func main() {
    ctx := context.Background()

    // Create client with explicit API keys
    client, err := echo.NewCommonClient(map[string]string{
        "openai": "your-openai-key",
    }, echo.WithModel("openai/gpt-6.1-sol"))
    if err != nil {
        panic(err)
    }

    // Simple call using QuickMessage helper
    resp, err := client.Complete(ctx, echo.QuickMessage("Hello, how are you?"))
    if err != nil {
        panic(err)
    }
    fmt.Println(resp.Text)
}

Pass nil as keys to auto-detect API keys from environment variables:

client, err := echo.NewCommonClient(nil, echo.WithModel("openai/gpt-6.1-sol"))

Provider-Specific Clients

If you only need a single provider, use the dedicated constructors:

// OpenAI
client := echo.NewOpenAIClient("your-api-key", "gpt-6.1-sol")

// Anthropic
client := echo.NewAnthropicClient("your-api-key", "claude-sonnet-5-5")

// Google
client := echo.NewGoogleClient("your-api-key", "gemini-4-argon")

// xAI (Grok)
client := echo.NewXAIClient("your-api-key", "grok-4.7")

// Voyage AI (embeddings & reranking)
client := echo.NewVoyageClient("your-api-key", "voyage-4-large")

// TypeSafe (Jev decision calls)
client := echo.NewTypeSafeClient("your-api-key", "jev-latest")

These accept the same CallOption options as NewCommonClient:

client := echo.NewOpenAIClient("your-api-key", "gpt-6.1-sol",
    echo.WithSystemMessage("You are a helpful assistant."),
    echo.WithTemperature(0.7),
)

Model Aliases

Use convenient aliases instead of full model names:

// Quality tiers available for each provider:
// - best: Highest quality model
// - balanced: Good balance of quality and speed
// - light: Fast and economical

client, _ := echo.NewCommonClient(nil, echo.WithModel("openai/best"))        // Uses gpt-6-astra
client, _ := echo.NewCommonClient(nil, echo.WithModel("anthropic/balanced")) // Uses claude-sonnet-5-5
client, _ := echo.NewCommonClient(nil, echo.WithModel("google/light"))       // Uses gemini-3.5-flash-lite
client, _ := echo.NewCommonClient(nil, echo.WithModel("xai/best"))           // Uses grok-4.7

Provider	`best`	`balanced`	`light`
openai	gpt-6-astra	gpt-6.1-sol	gpt-6-luna
anthropic	claude-fable-5-1	claude-sonnet-5-5	claude-haiku-5-5
google	gemini-4-argon	gemini-3.8-flash	gemini-3.5-flash-lite
xai	grok-4.7	grok-4.3	grok-4.20-0309-non-reasoning
openrouter	openai/gpt-6-astra	openai/gpt-6.1-sol	openai/gpt-6-luna
voyage (embeddings)	voyage-4-large	voyage-4	voyage-4-lite
typesafe	jev-latest	jev-latest	jev-latest

Environment Variables

The library supports flexible environment variable configuration:

// Set default model and API key
os.Setenv("ECHO_MODEL", "anthropic/balanced")
os.Setenv("ECHO_KEY", "your-api-key")

// Create client without parameters - uses env vars
client, _ := echo.NewCommonClient(nil)

// Or use provider-specific API keys
os.Setenv("OPENAI_API_KEY", "your-openai-key")
os.Setenv("ANTHROPIC_API_KEY", "your-anthropic-key")
os.Setenv("GOOGLE_API_KEY", "your-google-key")
os.Setenv("XAI_API_KEY", "your-xai-key")
os.Setenv("TYPESAFE_API_KEY", "your-typesafe-key")

// API key is automatically selected based on provider
client, _ := echo.NewCommonClient(nil, echo.WithModel("openai/gpt-6.1-sol"))

Message Chains

The library supports three ways to create message chains for conversations:

1. QuickMessage - Simple Single Messages

For basic single-message prompts:

resp, _ := client.Complete(ctx, echo.QuickMessage("Tell me a joke"))

2. TemplateMessage - Multi-Message Templates

For readable multi-turn conversations using a text template:

messages := echo.TemplateMessage(`
@system:
You are a helpful math tutor.

@user:
What is 2+2?

@agent:
2+2 equals 4.

@user:
Can you explain why?
`)

resp, err := client.Complete(ctx, messages)

Template format:

@role: markers separate messages (system, user, agent)
Content follows until the next marker or end of template
Content can be on the same line: @user: Hello there!
Multiline content is supported
Whitespace is automatically trimmed

3. Manual Message Construction

For programmatic message building:

messages := []echo.Message{
    {Role: echo.System, Content: "You are a helpful assistant."},
    {Role: echo.User, Content: "Hello"},
    {Role: echo.Agent, Content: "Hi! How can I help you today?"},
    {Role: echo.User, Content: "What's the weather like?"},
}

resp, err := client.Complete(ctx, messages)

Message Roles

echo.System - System instructions (must be first if present, only one allowed)
echo.User - User messages
echo.Agent - Assistant/model messages (maps to "assistant" for OpenAI/Anthropic, "model" for Gemini)

Options and Configuration

Client Creation with Options

// Set defaults at client creation time
client, _ := echo.NewCommonClient(nil,
    echo.WithModel("google/best"),
    echo.WithSystemMessage("You are a creative assistant."),
    echo.WithTemperature(0.8),
)

// Use client defaults
resp, _ := client.Complete(ctx, echo.QuickMessage("Tell me a joke"))

// Override defaults for specific calls
resp, _ = client.Complete(ctx, echo.QuickMessage("Write a formal email"),
    echo.WithTemperature(0.2), // More deterministic
)

Dynamic Provider Switching

The library supports switching providers on a per-call basis using WithModel:

// Create a client with a default provider
client, _ := echo.NewCommonClient(nil, echo.WithModel("openai/gpt-6.1-sol"))

// Use different providers for different calls
resp1, _ := client.Complete(ctx, echo.QuickMessage("Analyze this text"),
    echo.WithModel("anthropic/claude-opus-5-5"), // Use Anthropic for analysis
)

resp2, _ := client.Complete(ctx, echo.QuickMessage("Generate an image description"),
    echo.WithModel("google/gemini-4-argon"), // Use Google for creative tasks
)

resp3, _ := client.Complete(ctx, echo.QuickMessage("Quick calculation"),
    echo.WithModel("openai/gpt-6-luna"), // Use a lighter model for simple tasks
)

Per-Call Options

resp, err := client.Complete(ctx, echo.QuickMessage("Write a story"),
    echo.WithTemperature(0.7),
    echo.WithMaxTokens(100),
    echo.WithSystemMessage("You are a creative writer."),
)

Available Options

WithModel(string) - Override model for this call
WithTemperature(float32) - Control randomness (0.0 - 1.0)
WithMaxTokens(int) - Limit response length
WithSystemMessage(string) - Set or override system prompt (overrides any system message in the message chain)
WithBaseURL(string) - Override the API base URL (useful for custom endpoints)
WithEndPoint(string) - Specify endpoint routing (primarily for OpenRouter provider selection)
WithStoreData(bool) - Control server-side storage (xAI only, defaults to false for privacy)

Token Usage

Every response carries a normalized Usage struct (resp.Usage), or nil when the provider does not report usage (Google embeddings, local CLI providers):

resp, _ := client.Complete(ctx, echo.QuickMessage("Hello"))
if u := resp.Usage; u != nil {
    fmt.Println(u.InputTokens, u.OutputTokens, u.TotalTokens)
}

Field	Meaning
`Model`	Resolved `provider/model` used for the call (aliases expanded)
`InputTokens`	All input tokens, including cached ones
`CachedInputTokens`	Input tokens read from cache (subset of `InputTokens`)
`CacheWriteTokens`	Input tokens written to cache, e.g. Anthropic cache creation (subset of `InputTokens`)
`OutputTokens`	All output tokens, including reasoning/thinking
`ReasoningTokens`	Reasoning/thinking tokens (subset of `OutputTokens`)
`TotalTokens`	Total tokens
`CostUSD`	Charged amount, only when the provider reports it (OpenRouter); otherwise `nil`

The core package does not compute prices; see Cost Calculation for the optional pricing package. Use Usage.Add to accumulate token totals across calls. The same struct is available on EmbeddingResponse, RerankResponse and DecisionResponse. When streaming, usage arrives on a dedicated chunk near the end of the stream:

for chunk := range stream.Stream {
    if chunk.Usage != nil {
        total.Add(chunk.Usage)
    }
}

Cost Calculation

The optional github.com/mkozhukh/echo/pricing package turns Usage into USD. It ships with a built-in price table: a models.dev snapshot for OpenAI, Anthropic, Google, xAI and OpenRouter, plus Voyage AI and TypeSafe Jev prices. Prices are USD per 1M tokens and take cached input, cache writes, separately priced reasoning and context tiers (e.g. Gemini over 200k tokens) into account. Free tiers, batch discounts and account-specific rates are not.

import "github.com/mkozhukh/echo/pricing"

resp, _ := client.Complete(ctx, echo.QuickMessage("Hello"))
if cost, ok := pricing.Default().Cost(resp.Usage); ok {
    fmt.Printf("$%.6f (input $%.6f, output $%.6f)\n", cost.Total, cost.Input, cost.Output)
}

ok is false when no price is known for the model (local llama.cpp / CLI providers, fine-tunes, models missing from the table). A cost reported by the provider itself (OpenRouter) is used as is.

Cost in every response

Pass a price table to the client and Usage.CostUSD is filled on every response, including the usage chunk of streams, so results can be served or logged with their cost attached:

client, _ := echo.NewCommonClient(nil,
    echo.WithModel("anthropic/balanced"),
    echo.WithPricer(pricing.Default()),
)

resp, _ := client.Complete(ctx, echo.QuickMessage("Hello"))
json.NewEncoder(w).Encode(resp) // {"text": "...", "usage": {"model": "...", ..., "cost_usd": 0.0012}}

WithPricer accepts any echo.Pricer, so you can plug in your own price source. To enrich a Usage manually, use prices.Enrich(&usage).

Fresh or custom prices

prices, err := pricing.Fetch(ctx) // latest models.dev data; on error returns the built-in table and err
if err != nil {
    log.Println("using built-in prices:", err)
}

prices.Save(file)                    // cache it...
prices, err = pricing.Load(file)     // ...and load it on the next start

prices.Set("openai/my-finetune", pricing.Price{Rates: pricing.Rates{Input: 3, Output: 12}})

pricing.Default() is shared by the whole process; call Clone() before Set if you need an isolated table. The built-in snapshot is refreshed with go generate ./pricing.

Streaming Responses

For real-time streaming of responses, use the StreamComplete method:

Basic Streaming

streamResp, err := client.StreamComplete(ctx, echo.QuickMessage("Write a short story"))
if err != nil {
    panic(err)
}

StreamComplete vs Complete

Complete: Returns complete response after generation finishes
StreamComplete: Returns chunks as they're generated for real-time display

Both methods support the same options

Using in Tests

The "mock" provider can be used for tests, it will return combined string of all incoming messages

client, _ := echo.NewCommonClient(nil, echo.WithModel("mock/any"))
mockResp, err := client.Complete(ctx, echo.QuickMessage("test"))
if err != nil {
    panic(err)
}
// outputs: `[user]: test`

Using OpenRouter

OpenRouter provides access to multiple LLM providers through a single API:

// Basic usage with any OpenRouter model
client, _ := echo.NewCommonClient(map[string]string{
    "openrouter": "your-openrouter-key",
}, echo.WithModel("openrouter/anthropic/claude-sonnet-5.5"))

you can specify which underlying provider infrastructure to use:

// Specify provider routing with @ syntax in model name
client, _ := echo.NewCommonClient(nil, echo.WithModel("openrouter/anthropic/claude-sonnet-5.5@amazon-bedrock"))

// Multiple providers for fallback (comma-separated)
client, _ := echo.NewCommonClient(nil, echo.WithModel("openrouter/openai/gpt-6.1-sol@azure,openai"))

Using xAI (Grok)

xAI provides access to Grok models:

// Basic usage
client, _ := echo.NewCommonClient(map[string]string{
    "xai": "your-xai-key",
}, echo.WithModel("xai/grok-4.7"))

// Server-side storage is disabled by default for privacy
// To explicitly enable storage:
resp, _ := client.Complete(ctx, echo.QuickMessage("Hello"),
    echo.WithStoreData(true),
)

Decision Calls (TypeSafe Jev)

TypeSafe Jev answers typed questions about a state instead of generating text. Use Decide with three question types:

Choice - picks one option; returns the choice, probabilities and confidence
Score - scores the state on ordered levels; returns a score (can fall between levels), probabilities and confidence
Noul - probability (0..1) that a statement is true

client, _ := echo.NewCommonClient(nil, echo.WithModel("typesafe/jev-latest"))

resp, err := client.Decide(ctx, ticketText, echo.Questions{
    "type": echo.Choice("What is the main request?",
        echo.Option{Name: "refund", Description: "wants money back"},
        echo.Option{Name: "bug", Description: "reports a defect"},
    ),
    "anger":    echo.Score("How frustrated is the customer?", "calm", "annoyed", "furious"),
    "escalate": echo.Noul("The customer threatens to cancel"),
})
if err != nil {
    panic(err)
}

if kind, conf := resp.Choice("type"); kind == "refund" && conf > 0.8 {
    // ...
}
fmt.Println(resp.Score("anger"), resp.Noul("escalate"))

// Full answer details: probabilities, legend, confidence
fmt.Println(resp.Answers["anger"].Probabilities)

The state can be a string or any JSON-serializable value. Options are sent in the declared order. Use WithCriteria to clarify a Noul question, or build a Question literal for object/array instructions. Token usage is in resp.Usage (see Token Usage).

Other providers return a "does not support decisions" error for Decide; the mock provider returns deterministic answers (first option, lowest level, 0.5) for tests. TypeSafe supports only Decide.

E2E Testing

Run the e2e smoke test against live provider APIs. Requires API keys set as environment variables — providers without keys are skipped automatically.

go run ./cmd/e2e/              # all providers
go run ./cmd/e2e/ openai       # specific provider
go run ./cmd/e2e/ openai voyage # multiple providers

Local CLI Providers

If you have an agentic CLI installed locally (Claude Code, OpenAI Codex, Gemini CLI) and want to reuse its subscription for plain text-in/text-out completions, the library can route requests through the local binary instead of hitting a remote API. The CLI is launched in non-interactive mode, the assembled prompt is piped in, and stdout is returned as the response.

Because this executes local binaries, CLI providers are not registered automatically. You must opt in with an explicit call:

client, _ := echo.NewCommonClient(nil)
echo.EnableLocalCLI(client) // registers claude-cli, codex-cli, gemini-cli

resp, err := client.Complete(ctx, echo.QuickMessage("explain channels"),
    echo.WithModel("claude-cli/opus"),
)

Model naming follows the usual provider/model convention, where the second segment is forwarded to the CLI as the model flag:

claude-cli/<model> - runs claude -p --output-format text --model <model>
codex-cli/<model> - runs codex exec -m <model> <prompt>
gemini-cli/<model> - runs gemini -p -m <model> <prompt>

Binary paths can be overridden via environment variables when the tools aren't on PATH or a specific version is required:

ECHO_CLAUDE_CLI_PATH
ECHO_CODEX_CLI_PATH
ECHO_GEMINI_CLI_PATH

Only the WithModel and WithSystemMessage options have any effect on a CLI call - the CLIs do not expose a uniform way to set temperature, max tokens, or structured output, so those options are silently ignored. Embeddings and reranking are likewise unsupported.

For full control (custom flags, alternative binaries) instantiate CLIProvider directly and register it with any provider name you like:

client.SetProvider("claude-cli", &echo.CLIProvider{
    Binary:    "/opt/claude/claude",
    ModelFlag: "--model",
    ExtraArgs: []string{"-p", "--output-format", "text"},
})

License

MIT

Name		Name	Last commit message	Last commit date
cmd/ec		cmd/ec
pricing		pricing
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
README.md		README.md
Taskfile.yml		Taskfile.yml
aliases_test.go		aliases_test.go
anthropic.go		anthropic.go
cli.go		cli.go
cli_test.go		cli_test.go
common.go		common.go
decision.go		decision.go
decision_test.go		decision_test.go
export_test.go		export_test.go
go.mod		go.mod
go.sum		go.sum
google.go		google.go
http.go		http.go
lib.go		lib.go
message.go		message.go
message_test.go		message_test.go
mock.go		mock.go
mock_test.go		mock_test.go
openai.go		openai.go
todo.md		todo.md
typesafe.go		typesafe.go
usage_test.go		usage_test.go
voyage.go		voyage.go
xai.go		xai.go

Latest commit

History

Folders and files

Repository files navigation