New to LangChain TypeScript library (coming from Python SDK)

[hodgesz]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

The following is my RAG app Python code to use BGE embeddings from Hugging Face and Qdrant vector store.

model_name = os.getenv('EMBEDDINGS_MODEL_NAME')
model_kwargs = {'device': 'mps'}
encode_kwargs = {'normalize_embeddings': True}
embeddings = HuggingFaceBgeEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs,
    encode_kwargs=encode_kwargs
)
qdrant_base_url = os.getenv('QDRANT_BASE_URL')
qdrant_index = os.getenv('QDRANT_USERPILOT_INDEX')
client = qdrant_client.QdrantClient(url=qdrant_base_url, prefer_grpc=True)
vector_store = Qdrant(client=client, collection_name=qdrant_index, embeddings=embeddings,
                      content_payload_key='_node_content')


Here is what I think it equivalent in TypeScript library:

const llm = new OpenAI({
    openAIApiKey: import.meta.env.VITE_OPENAI_KEY,
    temperature: 0.0, // Can be between 0 and 1
    // modelName: "gpt-3.5-turbo-1106",
    modelName: "gpt-4-0125-preview",
});

const embeddingModel = new HuggingFaceTransformersEmbeddings({
    // modelName: "Xenova/all-MiniLM-L6-v2",
    modelName: "BAAI/bge-large-en",
  });

const vectorStore = await QdrantVectorStore.fromExistingCollection(
    embeddingModel,
    {
      // url: process.env.QDRANT_BASE_URL,
      url: "http://192.168.1.52:6333",
      collectionName: "atypical-dune-index",
      // contentPayLoadKey: '_node_content',
    },
  );

export async function generateAnswer(question: string) {
    let answer = ''
    const chain = new RetrievalQAChain({
        combineDocumentsChain: loadQARefineChain(llm),
        retriever: vectorStore.asRetriever(),
    });

    try {
        console.log(question)
        // answer = await llm.invoke(question);
        const result = await chain.call({
            query: question,
        });
        answer = result.output_text;
    } catch (e) {
        console.log(e)
        return 'Something went wrong'
    }

    return answer
}

Description

Does this look right to you?

I am currently getting the following error in the above try / catch

SyntaxError: Unexpected token '<', "<!doctype "... is not valid JSON
at JSON.parse ()
at getModelJSON (@langchain_community_embeddings_hf_transformers.js?v=eb99f90c:15678:15)
at async Promise.all (:5173/index 0)
at async loadTokenizer (@langchain_community_embeddings_hf_transformers.js?v=eb99f90c:19114:16)
at async AutoTokenizer.from_pretrained (@langchain_community_embeddings_hf_transformers.js?v=eb99f90c:22265:46)
at async Promise.all (:5173/index 0)
at async loadItems (@langchain_community_embeddings_hf_transformers.js?v=eb99f90c:30612:3)
at async pipeline (@langchain_community_embeddings_hf_transformers.js?v=eb99f90c:30572:19)
at async HuggingFaceTransformersEmbeddings.runEmbedding (@langchain_community_embeddings_hf_transformers.js?v=eb99f90c:30677:18)
at async HuggingFaceTransformersEmbeddings.embedQuery (@langchain_community_embeddings_hf_transformers.js?v=eb99f90c:30671:18)

Something went wrong

System Info

This is on Mac and Node version is 10.5.0

npm list
[email protected] /Users/hodgesz/VsCodeProjects/job_interviews/ai-frontend-workshop/my-gpt
├── @qdrant/[email protected]
├── @types/[email protected]
├── @types/[email protected]
├── @typescript-eslint/[email protected]
├── @typescript-eslint/[email protected]
├── @vitejs/[email protected]
├── @xenova/[email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└── [email protected]

[dosubot[bot]]

Hey there, @hodgesz! Great to see you diving into the TypeScript waters after your Python SDK adventures. 🚀

From the error message you provided, it seems like the issue is related to the loading of the Hugging Face model. The error message "Unexpected token '<', "<!doctype "... is not valid JSON" typically occurs when the response expected to be in JSON format is actually HTML or XML. This often happens when an API call is made, and the server responds with an error page in HTML instead of the expected JSON data.

In your case, it seems like the Hugging Face model is not being loaded correctly. The error is thrown when the library tries to parse the model's configuration file, which should be in JSON format. However, it seems like the library is receiving an HTML response instead, which is why the JSON parsing fails.

This could be due to several reasons:

1. The model name is incorrect or the model does not exist. Please double-check the model name "BAAI/bge-large-en" and ensure it is available on the Hugging Face model hub.

2. There might be a network issue that prevents the library from accessing the Hugging Face model hub. Please ensure that your network allows outgoing connections to the Hugging Face servers.

3. There might be an issue with the version of the @xenova/transformers library you are using. The library is responsible for loading the Hugging Face models. Please ensure you are using a compatible version of the library.

As for the rest of your code, it seems to be correct. You are correctly initializing the OpenAI model, the HuggingFaceTransformersEmbeddings, and the QdrantVectorStore. You are also correctly setting up the RetrievalQAChain and using it to generate an answer to a given question.

However, please note that the RetrievalQAChain expects the combineDocumentsChain to be an instance of BaseChain. In your code, you are passing the result of the loadQARefineChain(llm) function call, which should return an instance of BaseChain. Please ensure that this function is correctly implemented and returns the expected value.

I hope this helps! If you have any more questions or if the issue persists, please provide more details so I can assist you better.

Sources

• langchain-core/src/output_parsers/tests/json.test.ts
• examples/src/models/embeddings/hf_transformers.ts
• libs/langchain-community/src/vectorstores/qdrant.ts
• langchain/src/chains/retrieval_qa.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.