feat: ollama-proxy — multi-account Ollama Cloud reverse proxy
Go reverse proxy for ollama.com that balances requests across multiple API keys with round-robin and failover on 429/5xx. Exposes both native Ollama API (/api/*) and OpenAI-compatible (/v1/*) passthrough. - Round-robin balancer with per-account cooldown (60s default) - Pre-stream failover: 429 → cooldown + next account; 5xx → next account - Streaming invariant: once 2xx starts streaming, no account switch - SSE (text/event-stream) and NDJSON passthrough with http.Flusher - CLI: accounts add/list/remove/set-base-url, serve, version - Accounts stored in ~/.config/ollama-proxy/accounts.json (chmod 0600) - systemd unit (User=dueattendant149, 127.0.0.1:11435, Restart=always) - 43 tests (unit + integration with httptest upstream) - opencode integration: custom provider 'ocp' with explicit model list - Requires NO_PROXY=127.0.0.1,localhost when HTTP_PROXY is set
This commit is contained in:
commit
3963eede70
23 changed files with 2534 additions and 0 deletions
59
internal/cli/root.go
Normal file
59
internal/cli/root.go
Normal file
|
|
@ -0,0 +1,59 @@
|
|||
// Package cli implements the ollama-proxy command-line interface: account
|
||||
// management subcommands and the `serve` subcommand that starts the proxy.
|
||||
package cli
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
)
|
||||
|
||||
// Run dispatches to a subcommand based on args[1]. It returns the process exit
|
||||
// code, or an error that the caller should print before exiting 1.
|
||||
func Run(args []string) int {
|
||||
if len(args) < 2 {
|
||||
printRootUsage()
|
||||
return 1
|
||||
}
|
||||
switch args[1] {
|
||||
case "serve":
|
||||
return RunServe(args[2:])
|
||||
case "accounts":
|
||||
return RunAccounts(args[2:])
|
||||
case "version":
|
||||
fmt.Println("ollama-proxy", Version)
|
||||
return 0
|
||||
case "-h", "--help", "help":
|
||||
printRootUsage()
|
||||
return 0
|
||||
default:
|
||||
fmt.Fprintf(os.Stderr, "unknown subcommand: %s\n\n", args[1])
|
||||
printRootUsage()
|
||||
return 2
|
||||
}
|
||||
}
|
||||
|
||||
// Version is overwritten at build time via -ldflags "-X .../cli.Version=...".
|
||||
var Version = "dev"
|
||||
|
||||
func printRootUsage() {
|
||||
fmt.Print(`ollama-proxy — multi-account Ollama Cloud reverse proxy
|
||||
|
||||
Usage:
|
||||
ollama-proxy serve [flags] Start the proxy server
|
||||
ollama-proxy accounts <subcommand> Manage Ollama Cloud API keys
|
||||
ollama-proxy version Print version
|
||||
|
||||
Accounts subcommands:
|
||||
accounts add [API_KEY] [--name <alias>] Add an account (prompts for key if not given)
|
||||
accounts list List configured accounts
|
||||
accounts remove <id|name> Remove an account
|
||||
accounts set-base-url <url> Set the default upstream URL
|
||||
|
||||
Environment variables (serve):
|
||||
OLLAMA_PROXY_ADDR listen address (default 127.0.0.1:11435)
|
||||
OLLAMA_PROXY_BASE_URL upstream root (default https://ollama.com)
|
||||
OLLAMA_PROXY_COOLDOWN 429 cooldown window (default 60s)
|
||||
OLLAMA_PROXY_RETRIES max failover attempts (default 3)
|
||||
OLLAMA_PROXY_LOG_LEVEL debug|info|warn|error (default info)
|
||||
`)
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue