ProxyLLMs
DB-GPT can be deployed on servers with lower hardware through proxy LLMs, and now dbgpt support many proxy llms, such as OpenAI、Azure、Wenxin、Tongyi、Zhipu and so on.
Proxy model
- Open AI
- Azure
- Qwen
- ChatGLM
- WenXin
Install dependencies
pip install -e ".[openai]"
Download embedding model
cd DB-GPT
mkdir models and cd models
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the .env
file
# .env
LLM_MODEL=chatgpt_proxyllm
PROXY_API_KEY={your-openai-sk}
PROXY_SERVER_URL=https://api.openai.com/v1/chat/completions
# If you use gpt-4
# PROXYLLM_BACKEND=gpt-4
Install dependencies
pip install -e ".[openai]"
Download embedding model
cd DB-GPT
mkdir models and cd models
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese # change this to other embedding model if needed.
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the .env
file
# .env
LLM_MODEL=proxyllm
PROXY_API_KEY=xxxx
PROXY_API_BASE=https://xxxxxx.openai.azure.com/
PROXY_API_TYPE=azure
PROXY_SERVER_URL=xxxx
PROXY_API_VERSION=2023-05-15
PROXYLLM_BACKEND=gpt-35-turbo
API_AZURE_DEPLOYMENT=xxxx[deployment_name]
Install dependencies
pip install dashscope
Download embedding model
cd DB-GPT
mkdir models and cd models
# embedding model
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
or
git clone https://huggingface.co/moka-ai/m3e-large
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the .env
file
# .env
# Aliyun tongyiqianwen
LLM_MODEL=tongyi_proxyllm
TONGYI_PROXY_API_KEY={your-tongyi-sk}
PROXY_SERVER_URL={your_service_url}
Install dependencies
pip install zhipuai
Download embedding model
cd DB-GPT
mkdir models and cd models
# embedding model
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
or
git clone https://huggingface.co/moka-ai/m3e-large
Configure the proxy and modify LLM_MODEL, PROXY_API_URL and API_KEY in the .env
file
# .env
LLM_MODEL=zhipu_proxyllm
PROXY_SERVER_URL={your_service_url}
ZHIPU_MODEL_VERSION={version}
ZHIPU_PROXY_API_KEY={your-zhipu-sk}
Download embedding model
cd DB-GPT
mkdir models and cd models
# embedding model
git clone https://huggingface.co/GanymedeNil/text2vec-large-chinese
or
git clone https://huggingface.co/moka-ai/m3e-large
Configure the proxy and modify LLM_MODEL, MODEL_VERSION, API_KEY and API_SECRET in the .env
file
# .env
LLM_MODEL=wenxin_proxyllm
WEN_XIN_MODEL_VERSION={version} # ERNIE-Bot or ERNIE-Bot-turbo
WEN_XIN_API_KEY={your-wenxin-sk}
WEN_XIN_API_SECRET={your-wenxin-sct}
⚠️ Be careful not to overwrite the contents of the .env
configuration file