Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LLMGraphTransformer with tools and Ollama triggers Pydantic parsing errors #29515

Open
5 tasks done
pbrady opened this issue Jan 30, 2025 · 3 comments
Open
5 tasks done
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@pbrady
Copy link

pbrady commented Jan 30, 2025

Checked other resources

  • I added a very descriptive title to this issue.
  • I searched the LangChain documentation with the integrated search.
  • I used the GitHub search to find a similar question and didn't find it.
  • I am sure that this is a bug in LangChain rather than my code.
  • The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

I'm trying to use LLMGraphTransformer to construct a knowledge graph for RAG and am hitting an issue when using the ChatOllama client. In the example, I use 'llama3.1:7b' since its small enough to host locally. I've seen the same errors with larger models that I'm running on external clusters.

Model setup:

from langchain_ollama import ChatOllama
from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_community.graphs.graph_document import Document
from langchain.globals import set_debug

set_debug(True)

llm = ChatOllama(model='llama3.1',
                 base_url='http://localhost:11434',
                 num_ctx=8192,
                 verbose=True)
doc_transformer = LLMGraphTransformer(
    llm=llm,
    allowed_nodes=["Animal", "Object"],
    node_properties=["name", "description"],
)
doc = Document("The lazy dog slept while the ball rolled by.  Bob the clown scared the missle")

The test:

doc_transformer.convert_to_graph_documents([doc])

The output:

[GraphDocument(nodes=[], relationships=[], source=Document(metadata={}, page_content='The lazy dog slept while the ball rolled by.  Bob the clown scared the missle'))]

Note that the nodes and relationships are empty. If I simply do:

doc_transformer = LLMGraphTransformer(
    llm=llm,
    ignore_tool_usage=True
)

Then the doc_transformer does produce some nodes and relationships but I would like to constraint the types that are generated

Error Message and Stack Trace (if applicable)

Debug output containing parsing error

[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence] Entering Chain run with input:
�[0m{
"input": "The lazy dog slept while the ball rolled by. Bob the clown scared the missle"
}
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > prompt:ChatPromptTemplate] Entering Prompt run with input:
�[0m{
"input": "The lazy dog slept while the ball rolled by. Bob the clown scared the missle"
}
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > prompt:ChatPromptTemplate] [0ms] Exiting Prompt run with output:
�[0m[outputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableParallel] Entering Chain run with input:
�[0m[inputs]
�[32;1m�[1;3m[llm/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableParallel > llm:ChatOllama] Entering LLM run with input:
�[0m{
"prompts": [
"System: # Knowledge Graph Instructions for GPT-4\n## 1. Overview\nYou are a top-tier algorithm designed for extracting information in structured formats to build a knowledge graph.\nTry to capture as much information from the text as possible without sacrificing accuracy. Do not add any information that is not explicitly mentioned in the text.\n- Nodes represent entities and concepts.\n- The aim is to achieve simplicity and clarity in the knowledge graph, making it\naccessible for a vast audience.\n## 2. Labeling Nodes\n- Consistency: Ensure you use available types for node labels.\nEnsure you use basic or elementary types for node labels.\n- For example, when you identify an entity representing a person, always label it as 'person'. Avoid using more specific terms like 'mathematician' or 'scientist'.- Node IDs: Never utilize integers as node IDs. Node IDs should be names or human-readable identifiers found in the text.\n- Relationships represent connections between entities or concepts.\nEnsure consistency and generality in relationship types when constructing knowledge graphs. Instead of using specific and momentary types such as 'BECAME_PROFESSOR', use more general and timeless relationship types like 'PROFESSOR'. Make sure to use general and timeless relationship types!\n## 3. Coreference Resolution\n- Maintain Entity Consistency: When extracting entities, it's vital to ensure consistency.\nIf an entity, such as "John Doe", is mentioned multiple times in the text but is referred to by different names or pronouns (e.g., "Joe", "he"),always use the most complete identifier for that entity throughout the knowledge graph. In this example, use "John Doe" as the entity ID.\nRemember, the knowledge graph should be coherent and easily understandable, so maintaining consistency in entity references is crucial.\n## 4. Strict Compliance\nAdhere to the rules strictly. Non-compliance will result in termination.\nHuman: Tip: Make sure to answer in the correct format and do not include any explanations. Use the given format to extract information from the following input: The lazy dog slept while the ball rolled by. Bob the clown scared the missle"
]
}
�[36;1m�[1;3m[llm/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableParallel > llm:ChatOllama] [4.38s] Exiting LLM run with output:
�[0m{
"generations": [
[
{
"text": "",
"generation_info": {
"model": "llama3.1",
"created_at": "2025-01-30T22:12:55.86615Z",
"done": true,
"done_reason": "stop",
"total_duration": 4362782833,
"load_duration": 34890083,
"prompt_eval_count": 615,
"prompt_eval_duration": 1845000000,
"eval_count": 105,
"eval_duration": 2479000000,
"message": {
"role": "assistant",
"content": "",
"images": null,
"tool_calls": null
}
},
"type": "ChatGeneration",
"message": {
"lc": 1,
"type": "constructor",
"id": [
"langchain",
"schema",
"messages",
"AIMessage"
],
"kwargs": {
"content": "",
"response_metadata": {
"model": "llama3.1",
"created_at": "2025-01-30T22:12:55.86615Z",
"done": true,
"done_reason": "stop",
"total_duration": 4362782833,
"load_duration": 34890083,
"prompt_eval_count": 615,
"prompt_eval_duration": 1845000000,
"eval_count": 105,
"eval_duration": 2479000000,
"message": {
"lc": 1,
"type": "not_implemented",
"id": [
"ollama",
"_types",
"Message"
],
"repr": "Message(role='assistant', content='', images=None, tool_calls=None)"
}
},
"type": "ai",
"id": "run-1e29f523-02e1-4720-b4dd-314d7c6147a1-0",
"tool_calls": [
{
"name": "DynamicGraph",
"args": {
"nodes": [
{
"id": "dog",
"label": "animal"
},
{
"id": "ball",
"label": "object"
},
{
"id": "Bob",
"label": "person"
},
{
"id": "clown",
"label": "job"
},
{
"id": "missle",
"label": "object"
}
],
"relationships": [
{
"source": "dog",
"target": "ball",
"type": "NEARBY"
},
{
"source": "Bob",
"target": "missle",
"type": "SCARED"
}
]
},
"id": "0598d923-f25a-487f-9527-06d4169094a6",
"type": "tool_call"
}
],
"usage_metadata": {
"input_tokens": 615,
"output_tokens": 105,
"total_tokens": 720
},
"invalid_tool_calls": []
}
}
}
]
],
"llm_output": null,
"run": null,
"type": "LLMResult"
}
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableParallel] [4.39s] Exiting Chain run with output:
�[0m[outputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableWithFallbacks] Entering Chain run with input:
�[0m[inputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error>] Entering Chain run with input:
�[0m[inputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error>] Entering Chain run with input:
�[0m[inputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence] Entering Chain run with input:
�[0m[inputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence > chain:RunnableLambda] Entering Chain run with input:
�[0m[inputs]
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence > chain:RunnableLambda] [14ms] Exiting Chain run with output:
�[0m[outputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence > parser:PydanticToolsParser] Entering Parser run with input:
�[0m[inputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableLambda] Entering Chain run with input:
�[0m[inputs]
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableLambda] [0ms] Exiting Chain run with output:
�[0m{
"output": null
}
�[31;1m�[1;3m[chain/error]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence > parser:PydanticToolsParser] [2ms] Parser run errored with error:
�[0m"13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missingTraceback (most recent call last):\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1914, in _call_with_config\n context.run(\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 396, in call_func_with_variable_args\n return func(input, **kwargs) # type: ignore[call-arg]\n ^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 194, in \n lambda inner_input: self.parse_result(\n ^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/openai_tools.py", line 294, in parse_result\n pydantic_objects.append(name_dictres["type"])\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/pydantic/main.py", line 214, in init\n validated_self = self.pydantic_validator.validate_python(data, self_instance=self)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\npydantic_core._pydantic_core.ValidationError: 13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing"
�[31;1m�[1;3m[chain/error]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence] [17ms] Chain run errored with error:
�[0m"13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missingTraceback (most recent call last):\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3016, in invoke\n input = context.run(step.invoke, input, config)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 193, in invoke\n return self._call_with_config(\n ^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1914, in _call_with_config\n context.run(\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 396, in call_func_with_variable_args\n return func(input, **kwargs) # type: ignore[call-arg]\n ^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 194, in \n lambda inner_input: self.parse_result(\n ^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/openai_tools.py", line 294, in parse_result\n pydantic_objects.append(name_dictres["type"])\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/pydantic/main.py", line 214, in init\n validated_self = self.pydantic_validator.validate_python(data, self_instance=self)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\npydantic_core._pydantic_core.ValidationError: 13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing"
�[31;1m�[1;3m[chain/error]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error>] [19ms] Chain run errored with error:
�[0m"13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missingTraceback (most recent call last):\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3721, in invoke\n output = {key: future.result() for key, future in zip(steps, futures)}\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3721, in \n output = {key: future.result() for key, future in zip(steps, futures)}\n ^^^^^^^^^^^^^^^\n\n\n File "/opt/homebrew/Cellar/[email protected]/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 456, in result\n return self.__get_result()\n ^^^^^^^^^^^^^^^^^^^\n\n\n File "/opt/homebrew/Cellar/[email protected]/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result\n raise self._exception\n\n\n File "/opt/homebrew/Cellar/[email protected]/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run\n result = self.fn(*self.args, **self.kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3705, in _invoke_step\n return context.run(\n ^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3016, in invoke\n input = context.run(step.invoke, input, config)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 193, in invoke\n return self._call_with_config(\n ^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1914, in _call_with_config\n context.run(\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 396, in call_func_with_variable_args\n return func(input, **kwargs) # type: ignore[call-arg]\n ^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 194, in \n lambda inner_input: self.parse_result(\n ^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/openai_tools.py", line 294, in parse_result\n pydantic_objects.append(name_dictres["type"])\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/pydantic/main.py", line 214, in init\n validated_self = self.pydantic_validator.validate_python(data, self_instance=self)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\npydantic_core._pydantic_core.ValidationError: 13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing"
�[31;1m�[1;3m[chain/error]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error>] [21ms] Chain run errored with error:
�[0m"13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missingTraceback (most recent call last):\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1914, in _call_with_config\n context.run(\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 396, in call_func_with_variable_args\n return func(input, **kwargs) # type: ignore[call-arg]\n ^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/passthrough.py", line 481, in _invoke\n **self.mapper.invoke(\n ^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3721, in invoke\n output = {key: future.result() for key, future in zip(steps, futures)}\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3721, in \n output = {key: future.result() for key, future in zip(steps, futures)}\n ^^^^^^^^^^^^^^^\n\n\n File "/opt/homebrew/Cellar/[email protected]/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 456, in result\n return self.__get_result()\n ^^^^^^^^^^^^^^^^^^^\n\n\n File "/opt/homebrew/Cellar/[email protected]/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result\n raise self._exception\n\n\n File "/opt/homebrew/Cellar/[email protected]/3.11.11/Frameworks/Python.framework/Versions/3.11/lib/python3.11/concurrent/futures/thread.py", line 58, in run\n result = self.fn(*self.args, **self.kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3705, in _invoke_step\n return context.run(\n ^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 3016, in invoke\n input = context.run(step.invoke, input, config)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 193, in invoke\n return self._call_with_config(\n ^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1914, in _call_with_config\n context.run(\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 396, in call_func_with_variable_args\n return func(input, **kwargs) # type: ignore[call-arg]\n ^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/base.py", line 194, in \n lambda inner_input: self.parse_result(\n ^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/langchain_core/output_parsers/openai_tools.py", line 294, in parse_result\n pydantic_objects.append(name_dictres["type"])\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\n File "/Users/ptb/.virtualenvs/ml/lib/python3.11/site-packages/pydantic/main.py", line 214, in init\n validated_self = self.pydantic_validator.validate_python(data, self_instance=self)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\npydantic_core._pydantic_core.ValidationError: 13 validation errors for DynamicGraph\nnodes.0.type\n Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.1.type\n Field required [type=missing, input_value={'id': 'ball', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.2.type\n Field required [type=missing, input_value={'id': 'Bob', 'label': 'person'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.3.type\n Field required [type=missing, input_value={'id': 'clown', 'label': 'job'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nnodes.4.type\n Field required [type=missing, input_value={'id': 'missle', 'label': 'object'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.source_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_id\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.0.target_node_type\n Field required [type=missing, input_value={'source': 'dog', 'target...ball', 'type': 'NEARBY'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.source_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_id\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing\nrelationships.1.target_node_type\n Field required [type=missing, input_value={'source': 'Bob', 'target...ssle', 'type': 'SCARED'}, input_type=dict]\n For further information visit https://errors.pydantic.dev/2.10/v/missing"
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign] Entering Chain run with input:
�[0m[inputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign > chain:RunnableParallel] Entering Chain run with input:
�[0m[inputs]
�[32;1m�[1;3m[chain/start]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign > chain:RunnableParallel > chain:RunnableLambda] Entering Chain run with input:
�[0m[inputs]
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign > chain:RunnableParallel > chain:RunnableLambda] [0ms] Exiting Chain run with output:
�[0m{
"output": null
}
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign > chain:RunnableParallel] [1ms] Exiting Chain run with output:
�[0m{
"parsed": null
}
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableAssign] [1ms] Exiting Chain run with output:
�[0m[outputs]
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence > chain:RunnableWithFallbacks] [23ms] Exiting Chain run with output:
�[0m[outputs]
�[36;1m�[1;3m[chain/end]�[0m �[1m[chain:RunnableSequence] [4.41s] Exiting Chain run with output:
�[0m[outputs]

Description

  • I'm trying to generate a constrained knowledge graph with LLMGraphTransformer and Ollama but am hitting pydantic parsing errors or the type:
Field required [type=missing, input_value={'id': 'dog', 'label': 'animal'}, input_type=dict]

which looks like the dynamic graph is being invoked with the wrong arguments since the label is the type

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 24.2.0: Fri Dec 6 19:01:59 PST 2024; root:xnu-11215.61.5~2/RELEASE_ARM64_T6000
Python Version: 3.11.11 (main, Dec 3 2024, 17:20:40) [Clang 16.0.0 (clang-1600.0.26.4)]

Package Information

langchain_core: 0.3.32
langchain: 0.3.16
langchain_community: 0.3.15
langsmith: 0.2.11
langchain_experimental: 0.3.4
langchain_neo4j: 0.3.0
langchain_ollama: 0.2.2
langchain_openai: 0.3.2
langchain_text_splitters: 0.3.5
langchainhub: 0.1.21
langgraph_sdk: 0.1.51

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.10
async-timeout: Installed. No version info available.
dataclasses-json: 0.6.7
httpx: 0.28.1
httpx-sse: 0.4.0
jsonpatch: 1.33
langsmith-pyo3: Installed. No version info available.
neo4j: 5.27.0
numpy: 1.26.4
ollama: 0.4.7
openai: 1.60.0
orjson: 3.10.15
packaging: 23.2
pydantic: 2.10.5
pydantic-settings: 2.7.1
PyYAML: 6.0.1
requests: 2.32.3
requests-toolbelt: 1.0.0
SQLAlchemy: 2.0.37
tenacity: 9.0.0
tiktoken: 0.8.0
types-requests: 2.32.0.20241016
typing-extensions: 4.12.2
zstandard: Installed. No version info available.

@pbrady pbrady changed the title LLMGraphTransform with tools and Ollama triggers Pydantic parsing errors LLMGraphTransformer with tools and Ollama triggers Pydantic parsing errors Jan 30, 2025
@dosubot dosubot bot added the 🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature label Jan 30, 2025
@pbrady
Copy link
Author

pbrady commented Jan 31, 2025

I get the expected results with ChatOpenAI as the llm. The chain does hook into a different runnable.

Here's the ollama chain that fails:

[chain/start] [chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence > parser:PydanticToolsParser] Entering Parser run with input:
[inputs]

The openAI chain that succeeds:

chain/start] [chain:RunnableSequence > chain:RunnableAssign<parsed,parsing_error> > chain:RunnableParallel<parsed,parsing_error> > chain:RunnableSequence > chain:_oai_structured_outputs_parser] Entering Chain run with input:
[inputs]

@pbrady
Copy link
Author

pbrady commented Jan 31, 2025

I threw a IPython.embed() into the parser at

to get some more details on the ollama failure:

In [1]: json_results
Out[1]:
{'args': {'nodes': [{'id': 'red ball', 'label': 'object'},
   {'id': 'missle', 'label': 'object'},
   {'id': 'Bob the clown', 'label': 'person'}],
  'relationships': [{'source': 'red ball',
    'target': 'watched by',
    'type': 'WATCHED_BY'},
   {'source': 'Bob the clown', 'target': 'scared', 'type': 'SCARED'},
   {'source': 'missle', 'target': 'scared by', 'type': 'SCARED_BY'}]},
 'type': 'DynamicGraph'}
...
In [5]: name_dict = {tool.__name__: tool for tool in self.tools}
...
In [11]: name_dict[json_results['type']](**json_results['args'])
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[11], line 1
----> 1 name_dict[json_results['type']](**json_results['args'])

File ~/.virtualenvs/ml/lib/python3.11/site-packages/pydantic/main.py:214, in BaseModel.__init__(self, **data)
    212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    213 __tracebackhide__ = True
--> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    215 if self is not validated_self:
    216     warnings.warn(
    217         'A custom validator is returning a value other than `self`.\n'
    218         "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
    219         'See the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.',
    220         stacklevel=2,
    221     )

ValidationError: 15 validation errors for DynamicGraph
nodes.0.type
  Field required [type=missing, input_value={'id': 'red ball', 'label': 'object'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.10/v/missing
...

@pbrady
Copy link
Author

pbrady commented Jan 31, 2025

I noticed that call to llm.with_structured_output in the LLMGraphTransformer.__init__ method is done without specifying the method argument. This unspecified argument has different defaults for ChatOllama and ChatOpenAI.
For ChatOpenAI it defaults to json_schema. For ChatOllama, function_calling.

If I add the argument method="json_schema" then I get the expected results from ChatOllama as well! Not sure if this should be hardcoded or left as an argument to the constructor.

I also just realized that langchain_experimental is in a different repo...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Development

No branches or pull requests

1 participant