3.4 Handling Streaming Requests

In this section, we will create a new HTTP trigger that returns a streaming response based on the request body of the POST request.

Stream The Numbers

Create a new file named http_trigger_stream_numbers.py in the awel_tutorial directory

from dbgpt._private.pydantic import BaseModel, Field
from dbgpt.core.awel import DAG, HttpTrigger, StreamifyAbsOperator, setup_dev_environment
from typing import AsyncIterator

class TriggerReqBody(BaseModel):
    n: int = Field(..., description="The number of integers to be streamed")

class NumberProducerOperator(StreamifyAbsOperator[TriggerReqBody, int]):
    """Create a stream of numbers from 0 to `n-1`"""
    async def streamify(self, req: TriggerReqBody) -> AsyncIterator[int]:
        for i in range(req.n):
            yield str(i) + "\n"

with DAG("awel_stream_numbers") as dag:
    trigger_task = HttpTrigger(
        endpoint="/awel_tutorial/stream_numbers", 
        methods="POST", 
        request_body=TriggerReqBody,
        status_code=200,
        streaming_predict_func=lambda x: True
    )
    task = NumberProducerOperator()
    trigger_task >> task

setup_dev_environment([dag], port=5555)

And run the following command to execute the code:

poetry run python awel_tutorial/http_trigger_stream_numbers.py

Now, open a new terminal and run the following command to send a POST request to the server:

curl -X POST \
"http://127.0.0.1:5555/api/v1/awel/trigger/awel_tutorial/stream_numbers" \
-H "Content-Type: application/json" \
-d '{"n": 5}'

The output should look like this:

0
1
2
3
4

Then you can stop the server by pressing Ctrl+C.

In this example, we created a HttpTrigger operator with a streaming predict function which is used to determine whether to stream the response(it always returns True in this example).