Why is returnDirect for Custom Tools of multi-action agents not supported?

[niklas-lohmann]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import { ChatPromptTemplate } from "@langchain/core/prompts";
import { DynamicTool } from "@langchain/core/tools";
import { ChatOpenAI } from "@langchain/openai";
import { AgentExecutor, createOpenAIFunctionsAgent } from "langchain/agents";
import { pull } from "langchain/hub";
import { customCallbackHandler } from "./callback-handler.js";

const questionOfLife = new DynamicTool({
  name: "Question_of_Life",
  description:
    "Rufe dies auf, um zu erfahren wie die endgültige Frage nach dem Leben, dem Universum und dem ganzen Rest beantwortet wird.",
  returnDirect: true,
  func: async () => "42"
});

const tools = [questionOfLife];

// Get the prompt to use - you can modify this!
// If you want to see the prompt in full, you can at:
// https://smith.langchain.com/hub/hwchase17/openai-functions-agent
const agentPrompt = await pull<ChatPromptTemplate>("hwchase17/openai-functions-agent");

const agentModel = new ChatOpenAI({
  modelName: "gpt-3.5-turbo-1106",
  temperature: 0
});

const agent = await createOpenAIFunctionsAgent({
  llm: agentModel,
  tools,
  prompt: agentPrompt
});

const agentExecutor = new AgentExecutor({
  agent,
  tools,
  verbose: false
});

class AgentService {
  async askLlm(msg: string) {
    return agentExecutor.invoke(
      { input: msg },
      {
        callbacks: [customCallbackHandler]
      }
    );
  }
}

export const agentService = new AgentService();

Description

I am trying to use OpenAI functions in combination with custom tools. For some use cases I want the output of a custom tool not to be parsed again by the LLM. I found the returnDirect option for custom tools in the documentation, but when I use the option, it returns the Error Tool with return direct ${tool.name} not supported for multi-action agent.. What is the reason why no direct return can be used for Open AI Function Agents?

System Info

[email protected]
Windows
[email protected]

[dosubot[bot]]

Hey @niklas-lohmann, nice to see you back here! Hope you're doing well.

The error you're encountering, "Tool with return direct ${tool.name} not supported for multi-action agent," is due to a specific check within the AgentExecutor constructor in the LangChain framework. This constructor checks if the agent being used is a "multi-action" type and if any of the tools specified for use with this agent have the returnDirect property set to true. If both conditions are met, the error is thrown.

Here's the relevant code snippet from the AgentExecutor constructor:

if (this.agent._agentActionType() === "multi") {
  for (const tool of this.tools) {
    if (tool.returnDirect) {
      throw new Error(
        `Tool with return direct ${tool.name} not supported for multi-action agent.`
      );
    }
  }
}

This design decision is likely made to ensure that the tools used with multi-action agents adhere to certain expectations regarding their output handling, which might not be compatible with the direct return mechanism.

The returnDirect property in the custom tools of the LangChain framework is used to determine whether the response from the chain should be the direct result of the graph query. When returnDirect is set to true, the chain directly returns the raw output of the graph query without any additional processing or formatting.

Unfortunately, this means that if you're using a multi-action agent, you won't be able to use the returnDirect property with your custom tools. You might need to find a different way to achieve your desired output. If you need further assistance, please provide more details about your use case and I'll do my best to help you find a solution.

You can find more details about this in the langchain/src/agents/executor.ts and examples/src/chains/graph_db_return_direct.ts files in the LangChain repository.

Sources

• langchain/src/agents/executor.ts
• examples/src/chains/graph_db_return_direct.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[jacoblee93]

This sounds like an oversight - could you open an issue or (if you're feeling brave!) take a stab at a PR?

[niklas-lohmann]

I opened a PR (#4643). Thanks!

[zAlweNy26]

Hi guys, sorry to reopening this discussion, but I'm still getting this error even after updating:

• [email protected]
• @langchain/[email protected]

Am I doing something wrong?
This is my snippet of code

const agent = RunnablePassthrough.assign({
	agent_scratchpad: x => generatedScratchpad((x.intermediateSteps ?? []) as AgentStep[]),
}).pipe(prompt).pipe(this.verboseRunnable).pipe(currentLLM).pipe(new ProceduresOutputParser())

const agentExecutor = AgentExecutor.fromAgentAndTools({
	agent,
	tools,
	returnIntermediateSteps: true,
	maxIterations: 5,
})

that throws the error:

langchainjs/langchain/src/agents/executor.ts

Line 433 in ed1f3fb

`Tool with return direct ${tool.name} not supported for multi-action agent.`

[varunmehra5]

Still getting this error on langchain==0.2.5

[zAlweNy26]

Me too :(

[benm5678]

Did anyone find a solution? Since functions seem to be deprecated trying to switch to the new tools param.

[zAlweNy26]

Which one are you talking about?