Chat
Given a list of messages comprising a conversation, the model will return a response.
Create Chat Completion
POST /api/v2/chat/completions
Examples
Stream Chat Completion
- Curl
- Python
DBGPT_API_KEY="dbgpt"
curl -X POST "http://localhost:5670/api/v2/chat/completions" \
-H "Authorization: Bearer $DBGPT_API_KEY" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-d "{\"messages\":\"Hello\",\"model\":\"chatgpt_proxyllm\", \"stream\": true}"
from dbgpt.client import Client
DBGPT_API_KEY = "dbgpt"
client = Client(api_key=DBGPT_API_KEY)
async for data in client.chat_stream(
model="chatgpt_proxyllm",
messages="hello",
):
print(data)
Chat Completion Stream Response
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "Hello"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "!"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " How"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " can"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " I"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " assist"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " you"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " today"}}]}
data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "?"}}]}
data: [DONE]
Chat Completion
- Curl
- Python
DBGPT_API_KEY="dbgpt"
curl -X POST "http://localhost:5670/api/v2/chat/completions" \
-H "Authorization: Bearer $DBGPT_API_KEY" \
-H "accept: application/json" \
-H "Content-Type: application/json" \
-d "{\"messages\":\"Hello\",\"model\":\"chatgpt_proxyllm\", \"stream\": false}"
from dbgpt.client import Client
DBGPT_API_KEY = "dbgpt"
client = Client(api_key=DBGPT_API_KEY)
response = await client.chat(model="chatgpt_proxyllm" ,messages="hello")
Chat Completion Response
{
"id": "a8321543-52e9-47a5-a0b6-3d997463f6a3",
"object": "chat.completion",
"created": 1710826792,
"model": "chatgpt_proxyllm",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I assist you today?"
},
"finish_reason": null
}
],
"usage": {
"prompt_tokens": 0,
"total_tokens": 0,
"completion_tokens": 0
}
}
Request body
messages string Required
A list of messages comprising the conversation so far. Example Python code.
model string Required
ID of the model to use. See the model endpoint compatibility table for details on which models work with the Chat API.
chat_mode string Optional
The DB-GPT chat mode, which can be one of the following: chat_normal
, chat_app
, chat_knowledge
, chat_flow
, default is chat_normal
.
chat_param string Optional
The DB-GPT The chat param value of chat mode: {app_id}
, {space_id}
, {flow_id}
, default is None
.
max_new_tokens integer Optional
The maximum number of tokens that can be generated in the chat completion.
The total length of input tokens and generated tokens is limited by the model's context length.
stream integer Optional
If set, partial message deltas will be sent.
Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE]
temperature integer Optional
What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.
conv_uid string Optional
The conversation id of the model inference, default is None
span_id string Optional
The span id of the model inference, default is None
sys_code string Optional
The system code, default is None
user_name string Optional
The web server user name, default is None
Chat Stream Response Body
id string
conv_uid of the convsersation.
model string
The model used for the chat completion.
created string
The Unix timestamp (in seconds) of when the chat completion was created.
choices array
A list of chat completion choices. Can be more than one if n is greater than 1.
-
index
integer
The index of the choice in the list of choices.
-
delta
object
The chat completion delta.
-
role
string
The role of the speaker. Can be
user
orassistant
. -
content
string
The content of the message.
-
finish_reason
string
The reason the chat completion finished. Can be
max_tokens
orstop
.
-
role
string
Chat Response Body
id string
conv_uid of the convsersation.
model string
The model used for the chat completion.
created string
The Unix timestamp (in seconds) of when the chat completion was created.
object string
The object type of the chat completion.
choices array
A list of chat completion choices. Can be more than one if n is greater than 1.
-
index
integer
The index of the choice in the list of choices.
-
delta
object
The chat completion delta.
-
role
string
The role of the speaker. Can be
user
orassistant
. -
content
string
The content of the message.
-
finish_reason
string
The reason the chat completion finished. Can be
max_tokens
orstop
.
-
role
string
usage object
The usage statistics for the chat completion.
-
prompt_tokens
integer
The number of tokens in the prompt.
-
total_tokens
integer
The total number of tokens in the chat completion.
-
completion_tokens
integer
The number of tokens in the chat completion.