GPT-4o-Realtime-Preview Azure Support?

[nthings]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import { AzureChatOpenAI } from "@langchain/openai";
const llm: any = new AzureChatOpenAI({
    modelName: "gpt-4o-realtime-preview",
    deploymentName: config.azureOpenAIApiDeploymentName,
    openAIApiVersion: "2024-10-01",
    azureOpenAIApiInstanceName: config.azureOpenAIInstanceName,
    maxTokens: config.maxToken,
    temperature: config.temperature,
    audio: {"voice": "alloy", "format": "wav"},
    modalities: ["text", "audio"],
});

Description

Similar configuration as seen on the direct OpenAI audio model: https://www.datacamp.com/tutorial/gpt-4o-audio-preview

But when using gpt-4o-realtime-preview that is the only audio model on azure. This error rises:

Result: Failure Exception: 404 Resource not found Troubleshooting URL: https://js.langchain.com/docs/troubleshooting/errors/MODEL_NOT_FOUND

System Info

"@langchain/openai": 0.3.14
"langchain": 0.3.6
platform Azure function

[dosubot[bot]]

Hello @nthings! 👋 I'm here to assist you with bug fixes, answer your questions, and guide you on how to contribute.

The error you're encountering, "404 Resource not found," suggests that the model name "gpt-4o-realtime-preview" might not be supported or correctly configured in the LangChain library version you're using. According to the information available, the supported model names for AzureChatOpenAI in LangChain version 0.3.14 include:

• "gpt-3.5-turbo"
• "gpt-3.5-turbo-0301"
• "gpt-4-0314"
• "gpt-3.5-turbo-instruct"
• "davinci-002" ^[1].

The model "gpt-4o-realtime-preview" is not listed among the supported models, which could be the reason for the "404 Resource not found" error. You might want to verify if this model is available in your Azure deployment or check for any updates or documentation from LangChain that might have introduced support for this model after version 0.3.14.

To continue talking to Dosu, mention @dosu.

[nthings]

Is there a plan to support it? As Azure does not have gpt-4o-audio-preview like the official OpenAI implementation