Cluster Deployment
Install command line tools
All the following operations are completed through the dbgpt
command. To use the dbgpt
command, you first need to install the DB-GPT
project. You can install it through the following command
$ pip install -e ".[default]"
It can also be used in script mode
$ python pilot/scripts/cli_scripts.py
Start Model Controller
$ dbgpt start controller
View log
$ docker logs db-gpt-webserver-1 -f
By default, Model Server
will start on port 8000
Start Model Worker
提示
Start glm-4-9b-chat
model Worker
dbgpt start worker --model_name glm-4-9b-chat \
--model_path /app/models/glm-4-9b-chat \
--port 8001 \
--controller_addr http://127.0.0.1:8000
提示
Start vicuna-13b-v1.5
model Worker
dbgpt start worker --model_name vicuna-13b-v1.5 \
--model_path /app/models/vicuna-13b-v1.5 \
--port 8002 \
--controller_addr http://127.0.0.1:8000
note
⚠️ Make sure to use your own model name and model path.
Start Embedding Model Worker
dbgpt start worker --model_name text2vec \
--model_path /app/models/text2vec-large-chinese \
--worker_type text2vec \
--port 8003 \
--controller_addr http://127.0.0.1:8000
note
⚠️ Make sure to use your own model name and model path.
Start Reranking Model Worker
dbgpt start worker --worker_type text2vec \
--rerank \
--model_path /app/models/bge-reranker-base \
--model_name bge-reranker-base \
--port 8004 \
--controller_addr http://127.0.0.1:8000
note
⚠️ Make sure to use your own model name and model path.
提示
View and inspect deployed models
$ dbgpt model list
+-------------------+------------+------------+------+---------+---------+-----------------+----------------------------+
| Model Name | Model Type | Host | Port | Healthy | Enabled | Prompt Template | Last Heartbeat |
+-------------------+------------+------------+------+---------+---------+-----------------+----------------------------+
| glm-4-9b-chat | llm | 172.17.0.2 | 8001 | True | True | | 2023-09-12T23:04:31.287654 |
| WorkerManager | service | 172.17.0.2 | 8001 | True | True | | 2023-09-12T23:04:31.286668 |
| WorkerManager | service | 172.17.0.2 | 8003 | True | True | | 2023-09-12T23:04:29.845617 |
| WorkerManager | service | 172.17.0.2 | 8002 | True | True | | 2023-09-12T23:04:24.598439 |
| WorkerManager | service | 172.21.0.5 | 8004 | True | True | | 2023-09-12T23:04:24.598439 |
| text2vec | text2vec | 172.17.0.2 | 8003 | True | True | | 2023-09-12T23:04:29.844796 |
| vicuna-13b-v1.5 | llm | 172.17.0.2 | 8002 | True | True | | 2023-09-12T23:04:24.597775 |
| bge-reranker-base | text2vec | 172.21.0.5 | 8004 | True | True | | 2024-05-15T11:36:12.935012 |
+-------------------+------------+------------+------+---------+---------+-----------------+----------------------------+
Use model serving
The model service deployed as above can be used through dbgpt_server. First modify the .env
configuration file to change the connection model address
dbgpt start webserver --light