AI Search with GPT4o omits 'tool'

[harpalshergill]

Hello,

in app.py, there is the following set of code which omits the role=tool. I am trying to understand why?
I have a setup with AI Search index to bring the dataset back. my model_args correctly load the extra settings for ai search index.

when i ask question, there is often a case when i do not get the answer. but if i ask same question I do get it.

any insight on this would be appreciated. thank you.

async def send_chat_request(request_body, request_headers):
    filtered_messages = []
    messages = request_body.get("messages", [])
    for message in messages:
        if message.get("role") != 'tool':
            filtered_messages.append(message)
            
    request_body['messages'] = filtered_messages

[borgerta]

@harpalshergill I believe the != 'tool' code is removing previous tool calls/responses from the messages json object, presumably to avoid confusing the gpt model as your conversation grows to include multiple back and forth messages.

The next line of code in the send_chat_request function calls the prepare_model_args function:

for message in messages:
        if message.get("role") != 'tool':
            filtered_messages.append(message)
            
    request_body['messages'] = filtered_messages
    model_args = prepare_model_args(request_body, request_headers)`

The prepare_model_args function should include your tool definitions. So basically, the code in question scrubs previous tool calls/responses from the messages json, immediately rewrites the tool definitions into your messages json, and then submits a chat request to the openai client.

I may be wrong, but some debugging statements that print out the messages json content could verify this.