Skip to main content
Version: v0.6.0

Chat

Given a list of messages comprising a conversation, the model will return a response.

Create Chat Completion

POST /api/v2/chat/completions

Examples​

Stream Chat Completion​

from dbgpt.client import Client

DBGPT_API_KEY = "dbgpt"
client = Client(api_key=DBGPT_API_KEY)

async for data in client.chat_stream(
    model="chatgpt_proxyllm",
    messages="hello",
):
    print(data)

Chat Completion Stream Response​

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "Hello"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "!"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " How"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " can"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " I"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " assist"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " you"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": " today"}}]}

data: {"id": "chatcmpl-ba6fb52e-e5b2-11ee-b031-acde48001122", "model": "chatgpt_proxyllm", "choices": [{"index": 0, "delta": {"role": "assistant", "content": "?"}}]}

data: [DONE]

Chat Completion​

from dbgpt.client import Client

DBGPT_API_KEY = "dbgpt"
client = Client(api_key=DBGPT_API_KEY)
response = await client.chat(model="chatgpt_proxyllm" ,messages="hello")

Chat Completion Response​

{
    "id": "a8321543-52e9-47a5-a0b6-3d997463f6a3",
    "object": "chat.completion",
    "created": 1710826792,
    "model": "chatgpt_proxyllm",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "Hello! How can I assist you today?"
            },
            "finish_reason": null
        }
    ],
    "usage": {
        "prompt_tokens": 0,
        "total_tokens": 0,
        "completion_tokens": 0
    }
}

Request body​

messages string Required

A list of messages comprising the conversation so far. Example Python code.

model string Required

ID of the model to use. See the model endpoint compatibility table for details on which models work with the Chat API.

chat_mode string Optional

The DB-GPT chat mode, which can be one of the following: chat_normal, chat_app, chat_knowledge, chat_flow, default is chat_normal.

chat_param string Optional

The DB-GPT The chat param value of chat mode: {app_id}, {space_id}, {flow_id}, default is None.

max_new_tokens integer Optional

The maximum number of tokens that can be generated in the chat completion.

The total length of input tokens and generated tokens is limited by the model's context length.

stream integer Optional

If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE]

temperature integer Optional

What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic.

conv_uid string Optional

The conversation id of the model inference, default is None

span_id string Optional

The span id of the model inference, default is None

sys_code string Optional

The system code, default is None

user_name string Optional

The web server user name, default is None

Chat Stream Response Body​

id string

conv_uid of the convsersation.

model string

The model used for the chat completion.

created string

The Unix timestamp (in seconds) of when the chat completion was created.

choices array

A list of chat completion choices. Can be more than one if n is greater than 1.

  • index integer

    The index of the choice in the list of choices.

  • delta object

    The chat completion delta.

    • role string

      The role of the speaker. Can be user or assistant.

    • content string

      The content of the message.

    • finish_reason string

      The reason the chat completion finished. Can be max_tokens or stop.

Chat Response Body​

id string

conv_uid of the convsersation.

model string

The model used for the chat completion.

created string

The Unix timestamp (in seconds) of when the chat completion was created.

object string

The object type of the chat completion.

choices array

A list of chat completion choices. Can be more than one if n is greater than 1.

  • index integer

    The index of the choice in the list of choices.

  • delta object

    The chat completion delta.

    • role string

      The role of the speaker. Can be user or assistant.

    • content string

      The content of the message.

    • finish_reason string

      The reason the chat completion finished. Can be max_tokens or stop.

usage object

The usage statistics for the chat completion.

  • prompt_tokens integer

    The number of tokens in the prompt.

  • total_tokens integer

    The total number of tokens in the chat completion.

  • completion_tokens integer

    The number of tokens in the chat completion.