How to get the session_id information from the config in chain callbacks?

[oldwinter]

add_route code

 add_routes(
        app,
        tubao_ai_chat_chain,
        path="/tubao-ai-chat",
        enable_feedback_endpoint=False,
        include_callback_events=True,  # todo 上线前去掉，这个是用于调试的
        per_req_config_modifier=_per_request_config_modifier,
        config_keys=["configurable", "session_id", "is_rewrite"],
    )

chain code

chain = _prompt | _model | StrOutputParser()
chain = chain. with_config( 
        config=RunnableConfig(
            callbacks=[
                CustomALLCallbackHandler(
                    session_id="123",  # todo .  how to get configurable.session_id in my chain code？
                    model_name=OPENAI_DEFAULT_MODEL,
                ),
            ],
        )
    )

my question

how to get configurable.session_id in my chain code？

I spent a long time trying to understand langserve, but still don't know what to do. Please advise.

[eyurtsev]

Could you provide a bit more context about what session ID is? Who generates it (client or server) and which part of the chain needs to use it?

[oldwinter]

Could you provide a bit more context about what session ID is? Who generates it (client or server) and which part of the chain needs to use it?

I use the demo here where session_id from :
https://python.langchain.com/docs/expression_language/how_to/message_history

chain_with_history.invoke(
    {"ability": "math", "question": "What does cosine mean?"},
    config={"configurable": {"session_id": "foobar"}},
)

request body just like this(copy from langserve swagger):

{
  "input": {
    "latest_question": "string"
  },
  "config": {
    "configurable": {
      "session_id": "string"
    }
  },
  "kwargs": {}
}

The session_id is generated by the client and passed to me through above URL call. In order to perform subsequent operations on MySQL database or other actions, I need to retrieve this session_id within the chain of callbacks. However, I am currently unsure how to obtain this session_id or the entire configuration at the time of the request. Could you please provide guidance on how to access it?

[eyurtsev]

@oldwinter

Could you provide more context about the goal of the code? Why is session_id need to be accessed from a callback handler? Callbacks do not accept config right now in their methods, so you can't do it with standard callbacks, but you can create custom code (sharing a snippet below).

Here's an example for chat with persistence -- I hope maybe it'll help and remove the need for using a callback handler?

https://github.com/langchain-ai/langserve/blob/main/examples/chat_with_persistence_and_user/server.py#L100

In this example, there's a configurable runnable that allows configuring value. The value is then accessed later by another runnable, which can run arbitrary logic on it. You can also create a custom callback handler and use it within that custom runnable if needed. (But I hope that none of this is actually necessary for your use case).

---

# Server code
from typing import Optional, Any

from fastapi import FastAPI
from langchain_core.runnables import RunnablePassthrough, ConfigurableField
from langchain_core.runnables import RunnableSerializable, RunnableConfig
from langchain_core.runnables.utils import Output, Input

from langserve import add_routes

app = FastAPI()


class ConstantRunnable(RunnableSerializable):
    """Runnable that always returns a constant value"""

    value: float

    def invoke(self, input: Any, config: Optional[RunnableConfig] = None) -> float:
        return self.value


customizable_runnable = ConstantRunnable(value=1.0).configurable_fields(
    value=ConfigurableField(
        id="value",
        name="Value",
        description="The value to return",
    )
)


class RunnableWithCustomLogic(RunnablePassthrough):
    def ainvoke(
        self, input: Input, config: Optional[RunnableConfig] = None, **kwargs: Any
    ) -> Output:
        from langchain_core.runnables.config import ensure_config

        config = ensure_config(config)
        raise ValueError(config["configurable"])
        return super().invoke(input, config, **kwargs)


server_runnable = customizable_runnable | RunnableWithCustomLogic()

add_routes(
    app,
    server_runnable,
    path="/add_one",
    config_keys=["configurable"],
)

# Client code
response = await client.post(
    "/add_one/invoke",
    json={"input": 1, "config": {"configurable": {"value": 8}}},
)

[ildiopantofola]

is there a way to send the same post request but using curl? In a similar situation I always get the "configurable" param missing error:
KeyError: 'value'