Callback handleLLMEnd Called Twice – Is It Doubling Token Usage?

[juanmanuelarze]

Hi everyone,

I'm working on a project where I'm using both a history retriever and a source retriever. This is my code:

`const { model, modelOptions, dbSource, botPhone, metadata } = assistant;
const { from, text } = message;

    const chatModel = ChatModelFactory.getModel(model, {
        ...modelOptions, // streaming is false
        callbacks: [
            {
                handleLLMEnd: (output) => this.onLLMEnd(botPhone, output)
            }
        ]
    }); // This returns the ChatOpenAI class

    const embedding = EmbeddingsFactory.createEmbedding(model);
    const contextSource = AIContextSourceFactory.getSource(dbSource);
    contextSource.setEmbedding(embedding);

    const chatHistoryKey = QueueService.buildChatHistoryRedisKey(from, botPhone);

    const promptTemplate = QueueService.createPromptTemplate(assistant.prompt);
    
    const [chatHistory, retriever, documentChain] = await Promise.all([
        this.getChatHistory(chatHistoryKey),
        contextSource.getRetriever(metadata),
        createStuffDocumentsChain({
            llm: chatModel,
            prompt: promptTemplate,
        })
    ]);

    const historyAwareRetriever = await createHistoryAwareRetriever({
        llm: chatModel,
        retriever,
        rephrasePrompt: promptTemplate,
    });

    const retrievalChain = await createRetrievalChain({
        retriever: historyAwareRetriever,
        combineDocsChain: documentChain,
    });

    const [retrievedDocuments, response] = await Promise.all([
        retriever.invoke(text),
        retrievalChain.invoke({
            context: "",
            chat_history: chatHistory,
            input: text,
        })
    ]);`

I've noticed that the handleLLMEnd callback is being called twice.
I'm wondering if the second call is simply adding the tokens from the first call plus the response, or if the request itself is actually being made twice.

If it's the latter, that would be a problem because it would double my token usage, which directly impacts costs.

Could someone clarify this for me? Also, if I'm unintentionally causing duplicate requests and there's a way to fix it, I would really appreciate your guidance.

Thanks in advance!