Exception Handling and HTTP Responses in the case of Exception

[mesutgurlek]

Hi, I am trying to build a langchain api using the langserve and I have followed the tutorial: https://github.com/langchain-ai/langserve/blob/main/examples/chat_with_persistence/server.py

I have modified the example, and reorganized the project structure with langchain project structure using langchain-cli.
Below is the project structure:

In this case when I have an exception in the llm_chain.py, than I want to return an HTTP Response with an Error code. However, in the initial tutorial, it is a single file that contains both the chain object and FastAPI endpoint. In my case, modules are separated, and I could not find any example about how to return a HTTP Response when I got an exception in the custom package which provides the chain object.

Could you provide any kind of use case / tutorial / example ?

[eyurtsev]

Hi @mesutgurlek,

It's not important whether the code is in different files or the same.

What information do you need to return? If it's just a more detailed error message, take a look at Fast API documentation here:
https://fastapi.tiangolo.com/tutorial/handling-errors/#fastapis-httpexception-vs-starlettes-httpexception

You can raise that exception to provide an error message to the client.

[mesutgurlek]

Hi @eyurtsev,

Thanks for the reference docs. I have read the detailed explanation and still the problem continues. Let me summarize my case with the example (Taken from LangServe repo) :

# This is the server.py
@app.get("/")
async def redirect_root_to_docs():
    return RedirectResponse("/docs")


# Edit this to add the chain you want to add
add_routes(app, chain_with_history, path="/chat_with_llm")

# This is from the package 
def create_session_factory(
        base_dir: Union[str, Path],
) -> Callable[[str], BaseChatMessageHistory]:
    """Create a session ID factory that creates session IDs from a base dir.

    Args:
        base_dir: Base directory to use for storing the chat histories.

    Returns:
        A session ID factory that creates session IDs from a base path.
    """
    base_dir_ = Path(base_dir) if isinstance(base_dir, str) else base_dir
    if not base_dir_.exists():
        base_dir_.mkdir(parents=True)

    def get_chat_history(session_id: str) -> FileChatMessageHistory:
        """Get a chat history from a session ID."""
        if not _is_valid_identifier(session_id):
            raise HTTPException(
                status_code=400,
                detail=f"Session ID `{session_id}` is not in a valid format. "
                       "Session ID must only contain alphanumeric characters, "
                       "hyphens, and underscores.",
            )
        file_path = base_dir_ / f"{session_id}.json"
        return FileChatMessageHistory(str(file_path))

    return get_chat_history

When I send request using swagger ui through the invoke endpoint:

However, my use case is using stream endpoint and it give undefined error:

Is there a difference for Exception handling depending on the endpoint ? Or just stream endpoint does not support such http responses?

Initially I thought that following case from the FastAPI docs might be happening (LangServe might using pydantic model in response_model so that FastAPI does not return an error):

https://fastapi.tiangolo.com/tutorial/handling-errors/#requestvalidationerror-vs-validationerror
''' If you use a Pydantic model in response_model, and your data has an error, you will see the error in your log.

But the client/user will not see it. Instead, the client will receive an "Internal Server Error" with a HTTP status code 500.

It should be this way because if you have a Pydantic ValidationError in your response or anywhere in your code (not in the client's request), it's actually a bug in your code. '''

Do you have any suggestion, including any kind of custom implementation?

[eyurtsev]

Ah that's an issue with LangServe. I have an issue here: #414

Exceptions encountered while streaming are sent as part of the streaming response, which is fine if it occurs in the middle of the stream, but should not be the case if it's before the streaming started as shown in your example.

[mesutgurlek]

Thanks in advance, I am following the issue 👍

[ola172]

Thanks in advance, I am following the issue 👍

hello, do you find any way to raise detailed http exception in case of streaming response?