Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify method for with_structured_output #39

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

pbrady
Copy link

@pbrady pbrady commented Jan 31, 2025

The default option of "function_calling" for Ollama was leading to errors. Defaulting to "json_schema" (as ChatOpenAI already does) results in success for Ollama. Expose the method argument to the user for finer-grained control if needed.

Fixes #38.

I'm not sure if this is the right approach but figured it would be a good starting point for a discussion. The script below used to trigger the parsing error mentioned in the issue. With the fix here, Ollama is able to produce Nodes and Relationships:

from langchain_ollama import ChatOllama
from langchain_experimental.graph_transformers import LLMGraphTransformer
from langchain_core.documents import Document

llm = ChatOllama(model="llama3.1", num_ctx=4096)
doc = Document(
    "The lazy dog watched as the red ball rolled.  Bob the clown scared the missles."
)

allowed_nodes = ["Animal", "Person", "Object"]

doc_transformer = LLMGraphTransformer(
    llm=llm, allowed_nodes=allowed_nodes
)

graph_doc = doc_transformer.convert_to_graph_documents([doc])
print(graph_doc)

Output:

GraphDocument(nodes=[Node(id='Lazy Dog', type='Person', properties={}), Node(id='Red Ball', type='Object', properties={}), Node(id='Bob The Clown', type='Person', properties={})], relationships=[Relationship(source=Node(id='Lazy Dog', type='Person', properties={}), target=Node(id='Red Ball', type='Object', properties={}), type='WATCHED', properties={}), Relationship(source=Node(id='Bob The Clown', type='Person', properties={}), target=Node(id='Missles', type='Object', properties={}), type='SCARED', properties={})], source=Document(metadata={}, page_content='The lazy dog watched as the red ball rolled.  Bob the clown scared the missles.'))]

The default option of "function_calling" for Ollama was leading to
errors.  Defaulting to "json_schema" (as ChatOpenAI already does)
results in success for Ollama.  Expose the `method` argument to the
user for finer-grained control if needed.
Copy link
Member

@efriis efriis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ignore initial review - thought this was modifying ChatOllama not this)

@efriis efriis self-assigned this Feb 8, 2025
@@ -824,7 +825,9 @@ def __init__(
relationship_properties,
self._relationship_type,
)
structured_llm = llm.with_structured_output(schema, include_raw=True)
structured_llm = llm.with_structured_output(
schema, method=method, include_raw=True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I misunderstood - let's pass this in as a kwarg when it's defined, but let's remove the default of json_schema above.

Also open to something more general like structured_output_kwargs: Optional[dict] where you'd achieve this with structured_output_kwargs={"method": "json_schema"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In review
Development

Successfully merging this pull request may close these issues.

Leaving method unspecified in LLMGraphTransformer triggers Pydantic parsing errors
2 participants