langchain agent -> Exiting LLM run with output: { "generations": [ [ null ] ] }

[bnn16]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

export const GPTmodel = (model, temp) => {
  return new ChatOpenAI({
        model,
        temperature,
        configuration: {
            baseURL: LLM_GATEWAY_BASE_URL,
            dangerouslyAllowBrowser: true,
            defaultHeaders: {
              blabla
            },
        },
        azureOpenAIBasePath: `ourpath`,
        azureOpenAIApiDeploymentName: model,
        azureOpenAIApiVersion: '2024-02-01',
        azureOpenAIApiKey: 'fake-key-we-use-auth',

}


export const ProcessMiningTutor = async () => {
    const llm = GPTmodel(GPTModel.GPT4, 0);

    const promptTemplate = SystemMessagePromptTemplate.fromTemplate(`
    answer the user question in a funny and rap like way.
    The user question is: {input}
    also what is the value of foo?
    `);

    const prompt = ChatPromptTemplate.fromMessages([
        [ 'system', 'You are a helpful assistant.' ],
        promptTemplate,
        new MessagesPlaceholder('agent_scratchpad'),
    ]);
    const tools = [
        new DynamicTool({
            name: 'FOO',
            description: 'call this to get the value of foo.',
            // eslint-disable-next-line @typescript-eslint/require-await
            func: async () => 'baz',
        }),
    ];

    const agent = await createOpenAIFunctionsAgent({
        llm,
        prompt,
        tools,
    });

    return new AgentExecutor({
        agent,
        tools,
        verbose: true,
    }).withConfig({ runName: 'Agent' });
};



then when I call it 


 const {
        isLoading, error,
    } = useQuery(
        {
            // eslint-disable-next-line @stylistic/max-len
            queryKey: [ 'agentResponse', currentQuestion?.id, currentQuestion, currentQuestion?.agent, currentQuestion?.question, askedQuestions ],
            queryFn: async () => {
                if (!currentQuestion) {
                    return;
                }
                console.log(currentQuestion);
                // I have multiple agents so just think I'm giving the agent from above
                const agentName = currentQuestion.agent;
                const agentCreator = agentRegistry[agentName];
                const executor = (await agentCreator()) as AgentExecutor;

               // this thing ALWAYS fails. 
                const result = executor.streamEvents({ input: currentQuestion.question }, { version: 'v1' });

                for await (const event of result) {
                    const eventType = event.event;
                    if (eventType === 'on_chain_start') {
                        // Was assigned when creating the agent with `.withConfig({"runName": "Agent"})` above
                        if (event.name === 'Agent') {
                            console.log('\n-----');
                            console.log(
                                `Starting agent: ${event.name} with input: ${JSON.stringify(
                                    event.data.input,
                                )}`,
                            );
                        }
                    } else if (eventType === 'on_chain_end') {
                        // Was assigned when creating the agent with `.withConfig({"runName": "Agent"})` above
                        if (event.name === 'Agent') {
                            console.log('\n-----');
                            console.log(`Finished agent: ${event.name}\n`);
                            console.log(`Agent output was: ${event.data.output}`);
                            console.log('\n-----');
                        }
                    } else if (eventType === 'on_llm_stream') {
                        const content = event.data?.chunk?.message?.content;
                        // Empty content in the context of OpenAI means
                        // that the model is asking for a tool to be invoked via function call.
                        // So we only print non-empty content
                        if (content !== undefined && content !== '') {
                            console.log(`| ${content}`);
                        }
                    } else if (eventType === 'on_tool_start') {
                        console.log('\n-----');
                        console.log(
                            `Starting tool: ${event.name} with inputs: ${event.data.input}`,
                        );
                    } else if (eventType === 'on_tool_end') {
                        console.log('\n-----');
                        console.log(`Finished tool: ${event.name}\n`);
                        console.log(`Tool output was: ${event.data.output}`);
                        console.log('\n-----');
                    }
                }

Description

I'm trying to use langchain to create an agent with openai functions/tools.
My application is living fully in the front-end. React.

In my network tab I can see the responce of the llm is SUCCESSFUL I CAN SEE THE CONTENT but the parser fails

@tanstack_react-query.js?v=77745d11:2942 Uncaught TypeError: Cannot read properties of undefined (reading 'content')
    at OpenAIFunctionsAgentOutputParser._baseMessageToString (chunk-NSISG4FD.js?v=77745d11:32:27)
    at OpenAIFunctionsAgentOutputParser._callWithConfig (chunk-NSISG4FD.js?v=77745d11:54:22)
    at OpenAIFunctionsAgentOutputParser._callWithConfig (chunk-2YXJZ3EI.js?v=77745d11:12447:28)
    at async OpenAIFunctionsAgentOutputParser._streamIterator (chunk-2YXJZ3EI.js?v=77745d11:12392:5)
    at async OpenAIFunctionsAgentOutputParser.transform (chunk-2YXJZ3EI.js?v=77745d11:12612:5)
    at async _RunnableSequence._streamIterator (chunk-2YXJZ3EI.js?v=77745d11:13336:24)

then after using verbose: true on the agent -> 

 Entering LLM run with input: {
  "messages": [
    [
      {
        "lc": 1,
        "type": "constructor",
        "id": [
          "langchain_core",
          "messages",
          "SystemMessage"
        ],
        "kwargs": {
          "content": "You are a helpful assistant.",
          "additional_kwargs": {},
          "response_metadata": {}
        }
      },
      {
        "lc": 1,
        "type": "constructor",
        "id": [
          "langchain_core",
          "messages",
          "SystemMessage"
        ],
        "kwargs": {
          "content": "\n    answer the user question in a funny and rap like way.
    The user question is: what the dawg doing?
    also what is the value of foo?  ",
          "additional_kwargs": {},
          "response_metadata": {}
        }
      }
    ]
  ]
}
chunk-2YXJZ3EI.js?v=77745d11:5262 [llm/end] [1:chain:Agent > 2:chain:OpenAIFunctionsAgent > 7:llm:ChatOpenAI] [1.28s] Exiting LLM run with output: {
  "generations": [
    [
      null
    ]
  ]
}
chunk-2YXJZ3EI.js?v=77745d11:5225 [chain/start] [1:chain:Agent > 2:chain:OpenAIFunctionsAgent > 8:parser:OpenAIFunctionsAgentOutputParser] Entering Chain run with input: {}
chunk-2YXJZ3EI.js?v=77745d11:5243 [chain/error] [1:chain:Agent > 2:chain:OpenAIFunctionsAgent > 8:parser:OpenAIFunctionsAgentOutputParser] [0ms] Chain run errored with error: "Cannot read properties of undefined (reading 'content')\n\nTypeError: Cannot read properties of undefined (reading 'content')\n    at OpenAIFunctionsAgentOutputParser._baseMessageToString (http://localhost:9000/node_modules/.vite/deps/chunk-NSISG4FD.js?v=77745d11:32:27)\n    at OpenAIFunctionsAgentOutputParser._callWithConfig (http://localhost:9000/node_modules/.vite/deps/chunk-NSISG4FD.js?v=77745d11:54:22)\n    at OpenAIFunctionsAgentOutputParser._callWithConfig (http://localhost:9000/node_modules/.vite/deps/chunk-2YXJZ3EI.js?v=77745d11:12447:28)\n    at async OpenAIFunctionsAgentOutputParser._streamIterator (http://localhost:9000/node_modules/.vite/deps/chunk-2YXJZ3EI.js?v=77745d11:12392:5)\n    at async OpenAIFunctionsAgentOutputParser.transform (http://localhost:9000/node_modules/.vite/deps/chunk-2YXJZ3EI.js?v=77745d11:12612:5)\n    at async _RunnableSequence._streamIterator (http://localhost:9000/node_modules/.vite/deps/chunk-2YXJZ3EI.js?v=77745d11:13336:24)"

But i can see the network tab

that the responce is there -_-

System Info

"@langchain/core": "^0.2.31",
"@langchain/openai": "^0.2.10",
"langchain": "^0.3.0",

platform macos
node version -> Node.js v20.12.2.
npm version 10.8.2

[dosubot[bot]]

@bnn16 Howdy! I'm here to help you with bugs, answer questions, and make you a contributor. Let's squash those bugs together!

I found a similar issue where the agent was not correctly incorporating the observations from the tool calls into its decision-making process, causing the agent to repeatedly call the same tool without using the observation for the final response ^[1]. Additionally, there was an issue where the response object from the invoke method was not structured as expected, leading to errors when accessing its properties ^[2].

To resolve the "Cannot read properties of undefined (reading 'content')" error, ensure that the response structure from the LLM is as expected and that the content property exists. Here are some steps to help you debug and fix the issue:

1. Verify the Response Structure:
   Ensure that the response from the LLM contains the expected structure. The content property should be properly nested within the choices array.

2. Add Detailed Logging:
   Add detailed logging to inspect the response before accessing its properties. This will help verify the structure of the response and identify any discrepancies.

3. Update the Code with Additional Checks:
   Modify your code to include additional checks and detailed logging:

const { ChatOpenAI } = require("@langchain/openai");

const model = new ChatOpenAI({
  model: "gpt-4",
  temperature: 0,
  configuration: {
    baseURL: LLM_GATEWAY_BASE_URL,
    dangerouslyAllowBrowser: true,
    defaultHeaders: {
      blabla
    },
  },
  azureOpenAIBasePath: `ourpath`,
  azureOpenAIApiDeploymentName: model,
  azureOpenAIApiVersion: '2024-02-01',
  azureOpenAIApiKey: 'fake-key-we-use-auth',
});

async function invokeModel() {
  try {
    const result = await model.invoke("Tell me a joke about cats");
    console.log("Model response:", result);

    // Check if the response contains the expected structure
    if (result && result.choices && result.choices[0] && result.choices[0].message && result.choices[0].message.content) {
      console.log("Message content:", result.choices[0].message.content);
    } else {
      console.error("Unexpected response structure:", result);
    }
  } catch (e) {
    console.error("Error invoking model:", e.message);
  }
}

invokeModel();

4. Ensure Correct API Key and Proxy Configuration:
   Make sure you are using a valid API key and that your proxy is correctly configured to return the expected response.

By following these steps, you can ensure that the response is correctly structured and handled, which should help resolve the error. If the issue persists, please provide the detailed logs from the invokeModel function to further diagnose the problem ^[2].

To continue talking to Dosu, mention @dosu.

[bnn16]

I don't even get past the const result = await model.invoke("Tell me a joke about cats");
line

since it fails under the hood of .invoke ._.
@dosu

[dosubot[bot]]

The model.invoke("Tell me a joke about cats") line fails because the invoke method expects a structured input, such as a list of chat messages or a formatted prompt, rather than a simple string.

To fix this, you should provide the input in the expected format. Here is an example of how you can structure the input correctly using the .withStructuredOutput() method to ensure the model returns structured data:

import { z } from "zod";

const joke = z.object({
  setup: z.string().describe("The setup of the joke"),
  punchline: z.string().describe("The punchline to the joke"),
  rating: z.number().optional().describe("How funny the joke is, from 1 to 10"),
});

const structuredLlm = model.withStructuredOutput(joke, { name: "joke" });

const jokeResult = await structuredLlm.invoke("Tell me a joke about cats");
console.log(jokeResult);

This should resolve the issue and allow the invoke method to process the request correctly, returning a structured JSON object with the joke's setup and punchline.

Additionally, you can pass runtime arguments to the invoke method to customize the behavior of the model. For example:

const input = `Tell me a joke about cats.`;
const options = {
  temperature: 0.7,
  maxOutputTokens: 50,
  stopSequences: ["\n\nHuman:"]
};

const result = await model.invoke(input, options);
console.log(result);

This ensures that the model's response is controlled and formatted as expected. For more details on runtime arguments and structured output, you can refer to the LangChain documentation on Google Generative AI chat model integration ^{[1][2][3][4][5]}.

To continue talking to Dosu, mention @dosu.