Adding support for standardized API contracts for AI apps

[danielbentes]

The Azure Chat Protocol for AI applications featuring chat-like solutions and tools, is an effort to standarize API contracts across solutions and languages, to make them compatible and interoperable.
https://github.com/Azure-Samples/ai-chat-app-protocol

Would it be an idea to support this in Langserve?

Or would it be any other way to support this?

[PlebeiusGaragicus]

I'm all for this. Especially:

The recommended path is chat for a non-streaming request and chat/stream for a streaming request.

Reference issue for feature request to add OpenAI compatible API endpoints to LangServe. I'm surprised with the reason that there is no "demand" for this feature. It would easily allow LangServe models be used with the many open source web frontends popping up these days like Open WebUI at the very least.

[eyurtsev]

LangServe is based on the runnable interface and exposes runnable endpoints. LangServe doesn't have any specific advantage for exposing openai compatible API. This shouldn't be hard to do using LangChain since LangChain standardizes across a lot of different models already so one could write an adapter that does the appropriate translation.

[danielbentes]

I disagree with the proposition that LangServe/Langchain doesn't have any specific advantage for exposing chat protocol compatible API like https://github.com/microsoft/ai-chat-protocol or OpenAI-like protocol. I would definitely find it helpful and productive to expose my runnables to any chat interface that already supports these protocols "out of the box"