-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Specify method
for with_structured_output
#39
base: main
Are you sure you want to change the base?
Conversation
The default option of "function_calling" for Ollama was leading to errors. Defaulting to "json_schema" (as ChatOpenAI already does) results in success for Ollama. Expose the `method` argument to the user for finer-grained control if needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(ignore initial review - thought this was modifying ChatOllama not this)
@@ -824,7 +825,9 @@ def __init__( | |||
relationship_properties, | |||
self._relationship_type, | |||
) | |||
structured_llm = llm.with_structured_output(schema, include_raw=True) | |||
structured_llm = llm.with_structured_output( | |||
schema, method=method, include_raw=True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I misunderstood - let's pass this in as a kwarg when it's defined, but let's remove the default of json_schema
above.
Also open to something more general like structured_output_kwargs: Optional[dict]
where you'd achieve this with structured_output_kwargs={"method": "json_schema"}
The default option of "function_calling" for Ollama was leading to errors. Defaulting to "json_schema" (as ChatOpenAI already does) results in success for Ollama. Expose the
method
argument to the user for finer-grained control if needed.Fixes #38.
I'm not sure if this is the right approach but figured it would be a good starting point for a discussion. The script below used to trigger the parsing error mentioned in the issue. With the fix here, Ollama is able to produce Nodes and Relationships:
Output:
GraphDocument(nodes=[Node(id='Lazy Dog', type='Person', properties={}), Node(id='Red Ball', type='Object', properties={}), Node(id='Bob The Clown', type='Person', properties={})], relationships=[Relationship(source=Node(id='Lazy Dog', type='Person', properties={}), target=Node(id='Red Ball', type='Object', properties={}), type='WATCHED', properties={}), Relationship(source=Node(id='Bob The Clown', type='Person', properties={}), target=Node(id='Missles', type='Object', properties={}), type='SCARED', properties={})], source=Document(metadata={}, page_content='The lazy dog watched as the red ball rolled. Bob the clown scared the missles.'))]