Model Providers

DB-GPT supports API providers and local runtimes. For a first run, use an API provider unless you specifically want local inference.

Quick pick

Not sure which provider to choose? Start with OpenAI or DeepSeek for the fastest setup (API proxy, no GPU needed). Use Ollama if you want to run models locally without complex setup.

Provider comparison

Provider	Type	GPU Required	Best for
OpenAI	API proxy	No	Production quality, fastest setup
DeepSeek	API proxy	No	Cost-effective, strong reasoning
Qwen (Tongyi)	API proxy	No	Chinese language, Alibaba Cloud users
SiliconFlow	API proxy	No	China-hosted, multiple model choices
Ollama	Local proxy	Optional	Easy local models, privacy-first
vLLM	Local	Yes (NVIDIA)	High-throughput production inference

How model configuration works

All models are configured in TOML files under configs/. Each config file defines:

• LLM(s) — The language model(s) for chat and reasoning
• Embedding(s) — The embedding model(s) for RAG and knowledge search
• Reranker(s) — Optional re-ranking models for better retrieval

[models]

# Language model
[[models.llms]]
name = "model-name"
provider = "provider-type"
api_key = "your-api-key"

# Embedding model
[[models.embeddings]]
name = "embedding-model-name"
provider = "provider-type"
api_key = "your-api-key"

Environment variables

You can use environment variable syntax in TOML configs: "${env:VARIABLE_NAME:-default_value}". This keeps secrets out of config files.

Provider guides

• OpenAI — fastest default for a first setup
• DeepSeek — strong reasoning, OpenAI-compatible proxy pattern
• Qwen (Tongyi) — Alibaba Cloud / DashScope
• SiliconFlow — China-hosted API options
• Ollama — simple local model runtime
• vLLM — GPU-backed local inference for heavier workloads

More providers

DB-GPT also supports additional providers through its proxy system. See Advanced LLM Configuration for:

• Azure OpenAI
• Google Gemini
• Anthropic Claude
• Baichuan
• Spark (iFlyTek)
• And more