Example that combines message history / memory, tools, and the agent executor?

[theogravity]

All the conversational agent examples seem to only use a message followed by a single prompt.

• I have a message history that I generate with ChatPromptTemplate.fromPromptMessages()
• I use a BufferWindowMemory to trim large histories
• I have tools I'd like the agent to use
• I have an OpenAI agent where I stream out the response

[theogravity]

Here's my version:

  const STRING_SPLIT_REGEX = /( +)/;

  // key must be chat_history
  const memory = new BufferWindowMemory({ k: 2, memoryKey: 'chat_history', returnMessages: true });

  // Populate with past ai / user chat history
  // do not include the user's latest prompt as that's used separately
  const messageHistory = [];

  messageHistory.forEach((message: { role: string; content: string }) => {
    switch (message.role) {
      case 'system':
        // There is no system message for agent executors since they generate
        // their own for use with tools
        break;
      case 'ai':
        memory.chatHistory.addAIChatMessage(message.content);
        break;
      case 'user':
        memory.chatHistory.addUserMessage(message.content);
        break;
    }
  });

  const tools: Tool[] = [
    // Add your tools here
  ];

  const llm = new ChatOpenAI({
    openAIApiKey: OPEN_AI_API_KEY,
    // I've found this doesn't stream the final output properly, it instead streams the agent executor instruction
    // which is what we don't want
    streaming: false,
    n: 1,
    modelName: model,
  });

  const agent = await initializeAgentExecutorWithOptions(tools, llm, {
    agentType: 'structured-chat-zero-shot-react-description',
    verbose: false,
    memory,
      agentArgs: {
        inputVariables: ['input', 'agent_scratchpad', 'chat_history'],
        memoryPrompts: [new MessagesPlaceholder('chat_history')],
      },
  });

  const response = await agent.call({
    input: prompt,
  });

  // stream out the response by word at a time
  const strings = response.output.split(STRING_SPLIT_REGEX);

  strings.forEach((string: string) => {
    res.write(string)
  });

  // end the connection
  res.end()

Resources I read to figure this out:

• https://medium.com/@avra42/how-to-build-a-personalized-pdf-chat-bot-with-conversational-memory-965280c160f8
• Is there any way to combine chatbot and question answering over docs? langchain#2185
• Creating conversational bots with memory, agents, and tools langchain#2134
• https://www.phind.com/search?cache=e0b0515f-9598-4414-809f-88d5628cd0f6
• https://towardsdatascience.com/a-gentle-intro-to-chaining-llms-agents-and-utils-via-langchain-16cd385fca81
• using a Agent and wanted to stream just the final response langchain#2483
• https://debanjandhar.hashnode.dev/adding-memory-to-langchainjs-agent

[reyco1]

Did you ever find a way to stream the desired response?

[FrankDaze]

I think for stream output you should replace await agent.call() by await agent.stream()

And set here streaming: true

const llm = new ChatOpenAI({
    openAIApiKey: OPEN_AI_API_KEY,
    // I've found this doesn't stream the final output properly, it instead streams the agent executor instruction
    // which is what we don't want
    streaming: false,
    n: 1,
    modelName: model,
  });