Can't pass messages to my finalRetrievalChain

[latifs]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

const localModel = new ChatOpenAI({
    modelName: 'gpt-4o-2024-08-06',
    temperature: 0.0,
  });

  const REPHRASE_QUESTION_SYSTEM_TEMPLATE = `Given the following conversation and a follow up question,
  rephrase the follow up question to be a standalone question.`;

  const rephraseQuestionChainPrompt = ChatPromptTemplate.fromMessages([
    ['system', REPHRASE_QUESTION_SYSTEM_TEMPLATE],
    new MessagesPlaceholder('history'),
    [
      'human',
      'Rephrase the following question as a standalone question:\n{question}',
    ],
  ]);

  const rephraseQuestionChain = RunnableSequence.from([
    rephraseQuestionChainPrompt,
    localModel,
    new StringOutputParser(),
  ]);

  /////
  // Answer Question Chain
  /////

  const ANSWER_CHAIN_SYSTEM_TEMPLATE = `You are an experienced researcher, 
expert at interpreting and answering questions based on provided sources.
Using the below provided context and chat history, 
answer the user's question to the best of your ability using only the resources provided. Be verbose!
<context>
{context}
</context>`;

  const answerGenerationChainPrompt = ChatPromptTemplate.fromMessages([
    ['system', ANSWER_CHAIN_SYSTEM_TEMPLATE],
    new MessagesPlaceholder('history'),
    [
      'human',
      'Now, answer this question using the previous context and chat history:\n{standalone_question}',
    ],
  ]);

  const convertDocsToString = (documents: Document[]): string => {
    return documents
      .map((document) => {
        console.log('document:', document);
        return `<doc>\n${document.pageContent}\n</doc>`;
      })
      .join('\n');
  };

  const documentRetrievalChain = RunnableSequence.from([
    (input: { question: string }) => input.question,
    async (standaloneQ) => {
      return await retriever(standaloneQ, botId, model);
    },
    convertDocsToString,
  ]);

  const conversationalRetrievalChain = RunnableSequence.from([
    RunnablePassthrough.assign({ standalone_question: rephraseQuestionChain }),
    RunnablePassthrough.assign({ context: documentRetrievalChain }),
    answerGenerationChainPrompt,
    localModel,
    new StringOutputParser(),
  ]);

  const messageHistory = new ChatMessageHistory();

  const finalRetrievalChain = new RunnableWithMessageHistory({
    runnable: conversationalRetrievalChain,
    getMessageHistory: () => messageHistory,
    historyMessagesKey: 'history',
    inputMessagesKey: 'question',
  });

Description

So I've been trying to use a RunnableWithMessageHistory and am trying to pass an empty ChatMessageHistory for now (ideally it'd load previous message from a db) but even this is not working somehow.

somehow complaining about

Type 'InMemoryChatMessageHistory' is not assignable to type 'BaseChatMessageHistory | BaseListChatMessageHistory | Promise<BaseChatMessageHistory | BaseListChatMessageHistory>'.

System Info

none

[dosubot[bot]]

I found a similar issue that might help you resolve your problem. The error you're encountering is due to a type mismatch when passing the ChatMessageHistory to RunnableWithMessageHistory. You need to ensure that the getMessageHistory function returns an instance of a class that extends BaseChatMessageHistory or BaseListChatMessageHistory.

Here's an example of how you can do this using an in-memory chat message history:

import { RunnableWithMessageHistory } from '@langchain/core/runnables';
import { InMemoryChatMessageHistory } from '@langchain/core/chat_history';
import { RunnableSequence, RunnablePassthrough } from '@langchain/core/runnables';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { ChatOpenAI } from '@langchain/openai';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { MessagesPlaceholder } from '@langchain/core/prompts';

const localModel = new ChatOpenAI({
  modelName: 'gpt-4o-2024-08-06',
  temperature: 0.0,
});

const REPHRASE_QUESTION_SYSTEM_TEMPLATE = `Given the following conversation and a follow up question,
rephrase the follow up question to be a standalone question.`;

const rephraseQuestionChainPrompt = ChatPromptTemplate.fromMessages([
  ['system', REPHRASE_QUESTION_SYSTEM_TEMPLATE],
  new MessagesPlaceholder('history'),
  [
    'human',
    'Rephrase the following question as a standalone question:\n{question}',
  ],
]);

const rephraseQuestionChain = RunnableSequence.from([
  rephraseQuestionChainPrompt,
  localModel,
  new StringOutputParser(),
]);

const ANSWER_CHAIN_SYSTEM_TEMPLATE = `You are an experienced researcher, 
expert at interpreting and answering questions based on provided sources.
Using the below provided context and chat history, 
answer the user's question to the best of your ability using only the resources provided. Be verbose!
<context>
{context}
</context>`;

const answerGenerationChainPrompt = ChatPromptTemplate.fromMessages([
  ['system', ANSWER_CHAIN_SYSTEM_TEMPLATE],
  new MessagesPlaceholder('history'),
  [
    'human',
    'Now, answer this question using the previous context and chat history:\n{standalone_question}',
  ],
]);

const convertDocsToString = (documents: Document[]): string => {
  return documents
    .map((document) => {
      console.log('document:', document);
      return `<doc>\n${document.pageContent}\n</doc>`;
    })
    .join('\n');
};

const documentRetrievalChain = RunnableSequence.from([
  (input: { question: string }) => input.question,
  async (standaloneQ) => {
    return await retriever(standaloneQ, botId, model);
  },
  convertDocsToString,
]);

const conversationalRetrievalChain = RunnableSequence.from([
  RunnablePassthrough.assign({ standalone_question: rephraseQuestionChain }),
  RunnablePassthrough.assign({ context: documentRetrievalChain }),
  answerGenerationChainPrompt,
  localModel,
  new StringOutputParser(),
]);

const messageHistory = new InMemoryChatMessageHistory();

const finalRetrievalChain = new RunnableWithMessageHistory({
  runnable: conversationalRetrievalChain,
  getMessageHistory: () => messageHistory,
  historyMessagesKey: 'history',
  inputMessagesKey: 'question',
});

In this example, InMemoryChatMessageHistory is used to create an empty chat message history. This ensures that the getMessageHistory function returns a valid instance that is assignable to BaseChatMessageHistory ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[latifs]

@dosu,
How would I add messages retrieved from a database to the InMemoryChatMessageHistory?

[dosubot[bot]]

To add messages retrieved from a database to the InMemoryChatMessageHistory, you can follow this approach:

1. Retrieve the messages from your database.
2. Add each message to the InMemoryChatMessageHistory instance.

Here's a sample code snippet demonstrating this:

import { HumanMessage, AIMessage } from "@langchain/core/messages";
import { InMemoryChatMessageHistory } from "@langchain/core/chat_history";

// Assume you have a function to retrieve messages from the database
async function getMessagesFromDatabase() {
  // This function should return an array of messages with a type and content
  return [
    { type: 'human', content: 'hi' },
    { type: 'ai', content: 'what is up?' }
  ];
}

const history = new InMemoryChatMessageHistory();

const messages = await getMessagesFromDatabase();

for (const message of messages) {
  if (message.type === 'human') {
    await history.addMessage(new HumanMessage(message.content));
  } else if (message.type === 'ai') {
    await history.addMessage(new AIMessage(message.content));
  }
}

console.log(await history.getMessages());

/*
  [
    HumanMessage {
      content: 'hi',
      additional_kwargs: {}
    },
    AIMessage {
      content: 'what is up?',
      additional_kwargs: {}
    }
  ]
*/

This code assumes you have a function getMessagesFromDatabase that retrieves messages from your database and returns them in a format that includes the type and content of each message. The messages are then added to the InMemoryChatMessageHistory instance accordingly ^[1][2].

To continue talking to Dosu, mention @dosu.

[latifs]

@dosu,
how would I return the sources at the end of this chain?

[dosubot[bot]]

To return the sources at the end of the chain, you can use the following code:

import {
  RunnableMap,
  RunnablePassthrough,
  RunnableSequence
} from "@langchain/core/runnables";
import { formatDocumentsAsString } from "langchain/util/document";
import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { OpenAIEmbeddings, ChatOpenAI } from "@langchain/openai";
import { pull } from "langchain/hub";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { StringOutputParser } from "@langchain/core/output_parsers";

// Load documents
const loader = new CheerioWebBaseLoader(
  "https://lilianweng.github.io/posts/2023-06-23-agent/"
);
const docs = await loader.load();

// Split documents into chunks
const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200 });
const splits = await textSplitter.splitDocuments(docs);

// Create a vector store from the document chunks
const vectorStore = await MemoryVectorStore.fromDocuments(splits, new OpenAIEmbeddings());

// Retrieve and generate using the relevant snippets of the blog
const retriever = vectorStore.asRetriever();
const prompt = await pull<ChatPromptTemplate>("rlm/rag-prompt");
const llm = new ChatOpenAI({ model: "gpt-3.5-turbo", temperature: 0 });

const ragChainFromDocs = RunnableSequence.from([
  RunnablePassthrough.assign({ context: (input) => formatDocumentsAsString(input.context) }),
  prompt,
  llm,
  new StringOutputParser()
]);

let ragChainWithSource = new RunnableMap({ steps: { context: retriever, question: new RunnablePassthrough() }});
ragChainWithSource = ragChainWithSource.assign({ answer: ragChainFromDocs });

await ragChainWithSource.invoke("What is Task Decomposition");

This code sets up a chain that retrieves documents, formats them, and then uses a language model to generate an answer. The sources (documents) are returned at the end of the chain ^[1][2].

To continue talking to Dosu, mention @dosu.