Does RemoteRunnable support Server-Sent Event error events?

[philbo]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

const remoteChain = new RemoteRunnable({
  url: `/myremote`,
});

const response = await remoteChain.stream(inputs);

for await (const chunk of response) {
  //if chunk == "error" {log; continue;}
  console.log(chunk);
}

Description

Langserve provides a way for an "error" event to be sent when requesting the /stream endpoint, for example something like:

< HTTP/1.1 200 OK
< content-type: text/event-stream; charset=utf-8
...
event: error
data: {"status_code": 500, "message": "Internal Server Error"}

As the response code is 200 the existing error handling (correctly) does not raise an error.

The Python RemoteRunnable client appears to support a way to handle this type of event.

Is there any mechanism in the Javascript RemoteRunnable client to handle errors sent from streaming endpoints?

System Info

[email protected]
dependencies:
@anthropic-ai/sdk: ^0.9.1 @langchain/core: ~0.1.44 binary-extensions: ^2.2.0 js-tiktoken: ^1.0.7 jsonpointer: ^5.0.1 langsmith: ~0.1.7 openapi-types: ^12.1.3 uuid: ^9.0.0 zod-to-json-schema: ^3.22.3
@langchain/community: ~0.0.36 @langchain/openai: ~0.0.19 expr-eval: ^2.0.2 js-yaml: ^4.1.0 langchainhub: ~0.0.8 ml-distance: ^4.0.0 p-retry: 4 yaml: ^2.2.1 zod: ^3.22.4

linux
Node v19.9.0

[dosubot[bot]]

Hey @philbo, great to see you diving into the depths of LangChain JS again! Hope this quest brings you some intriguing puzzles to solve. 🚀

Based on the information provided and the current state of the LangChain JS repository, it appears that the RemoteRunnable class does not have a specific mechanism to handle Server-Sent Event error events. The error handling in the stream method of the RemoteRunnable class is primarily focused on HTTP response status codes, not on the content of the Server-Sent Event stream.

The stream method checks if the HTTP response is not okay (i.e., !response.ok) and throws an error if the status code indicates a failure. However, this does not account for error events sent within the Server-Sent Event stream itself, as you've described.

Here is the relevant code snippet from the stream method:

if (!response.ok) {
  const json = await response.json();
  const error = new Error(
    `RemoteRunnable call failed with status code ${response.status}: ${json.message}`
  );
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  (error as any).response = response;
  throw error;
}

(Source: langchain-core/src/runnables/remote.ts)

As for customizing the error handling in the RemoteRunnable class, there doesn't appear to be a built-in way to do this based on the current repository code. The error handling logic is encapsulated within the stream method and is not exposed for customization.

It's worth noting that the RemoteRunnable class does parse the response from the streaming endpoint as a Server-Sent Event (SSE) stream, as confirmed by the test setup and test cases for streaming in the repository. However, the handling of specific SSE events, such as error events, is not explicitly addressed.

(Source: langchain-core/src/runnables/tests/runnable_remote.test.ts)

Given this, it seems that handling Server-Sent Event error events in the RemoteRunnable class would require modifications to the class itself. This could involve adding a new method or modifying the existing stream method to parse the SSE stream and handle error events appropriately.

I hope this information is helpful. If you have further questions or need more clarification, please let me know.

Sources

• langchain-core/src/runnables/remote.ts
• langchain-core/src/runnables/tests/runnable_remote.test.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[philbo]

I managed to extend the message parsing by reusing the langchain library functions

response = await fetch(/url, {...});

await getBytes(
    response.body!, 
    getLines(
      getMessages(
        async (message: EventSourceMessage) => {
          switch (message.event) {
            case 'error':
              console.error('error event:', message.data);
            case 'data':
              content += JSON.parse(message.data);
              break;
            case 'end':
              break;
            case 'metadata':
              break;
            default:
              console.log(`unhandled event type: '${message.event}' ${message.data}`);
              break;
          }
        },
      )
    )
  );