-
Notifications
You must be signed in to change notification settings - Fork 16.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: Standardize ChatGroq #22751
docs: Standardize ChatGroq #22751
Changes from 3 commits
e2a5f65
ad7d1c2
422e423
41bf8e2
2e34fea
c66590a
baadb69
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -85,15 +85,222 @@ | |
To use, you should have the | ||
environment variable ``GROQ_API_KEY`` set with your API key. | ||
|
||
Any parameters that are valid to be passed to the groq.create call can be passed | ||
in, even if not explicitly saved on this class. | ||
Any parameters that are valid to be passed to the groq.create call | ||
Check failure on line 88 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
can be passed in, even if not explicitly saved on this class. | ||
|
||
Example: | ||
.. code-block:: python | ||
|
||
from langchain_groq import ChatGroq | ||
|
||
model = ChatGroq(model_name="mixtral-8x7b-32768") | ||
|
||
Setup: | ||
Install ``langchain-groq`` and set environment variable | ||
Check failure on line 99 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
``GROQ_API_KEY``. | ||
|
||
.. code-block:: bash | ||
|
||
pip install -U langchain-groq | ||
export GROQ_API_KEY="your-api-key" | ||
|
||
Key init args — completion params: | ||
model_name: str | ||
Name of Groq model to use. E.g. "mixtral-8x7b-32768". | ||
temperature: float | ||
Sampling temperature. Ranges from 0.0 to 1.0. | ||
max_tokens: Optional[int] | ||
Max number of tokens to generate. | ||
|
||
Key init args — client params: | ||
cache: Union[BaseCache, bool, None] | ||
Whether to cache the response. Uses global cache if true, does not | ||
use a cache if false, uses global cache (if it is set) if None, | ||
and uses the provided cache if is instance of BaseCache. | ||
callbacks: Callbacks | ||
Callbacks to add to the run trace. | ||
custom_get_token_ids: Optional[Callable[[str], List[int]]] | ||
Optional encoder to use for counting tokens. | ||
groq_api_base: Optional[str] | ||
Base URL path for API requests, leave blank if not using a proxy | ||
Check failure on line 125 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
or service emulator. | ||
metadata: Optional[Dict[str, Any]] | ||
Metadata to add to the run trace. | ||
model_kwargs: Dict[str, Any] | ||
Holds any model parameters valid for create call not | ||
Check failure on line 130 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
explicitly specified. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: these are technically completion params |
||
streaming: bool = False | ||
Whether to stream the results or not. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we're generally moving off using this (recommended approach is just to call llm.stream directly) so don't think worth noting here |
||
|
||
See full list of supported init args and their descriptions in the params | ||
section. | ||
|
||
Instantiate: | ||
.. code-block:: python | ||
|
||
from langchain_groq import ChatGroq | ||
|
||
model = ChatGroq( | ||
model_name="mixtral-8x7b-32768", | ||
temperature=0.0, | ||
max_retries=2, | ||
# streaming=..., | ||
# other params... | ||
) | ||
|
||
Invoke: | ||
.. code-block:: python | ||
|
||
messages = [ | ||
("system", "You are a helpful translator. Translate the user | ||
Check failure on line 155 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
sentence to French."), | ||
("human", "I love programming."), | ||
] | ||
model.invoke(messages) | ||
|
||
.. code-block:: python | ||
|
||
AIMessage(content='The English sentence "I love programming" can | ||
Check failure on line 163 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
be translated to French as "J\'aime programmer". The word | ||
Check failure on line 164 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
"programming" is translated as "programmer" in French.', | ||
Check failure on line 165 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
response_metadata={'token_usage': {'completion_tokens': 38, | ||
Check failure on line 166 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
'prompt_tokens': 28, 'total_tokens': 66, 'completion_time': | ||
Check failure on line 167 in libs/partners/groq/langchain_groq/chat_models.py GitHub Actions / cd libs/partners/groq / make lint #3.8Ruff (W291)
|
||
0.057975474, 'prompt_time': 0.005366091, 'queue_time': None, | ||
'total_time': 0.063341565}, 'model_name': 'mixtral-8x7b-32768', | ||
'system_fingerprint': 'fp_c5f20b5bb1', 'finish_reason': 'stop', | ||
'logprobs': None}, id='run-ecc71d70-e10c-4b69-8b8c-b8027d95d4b8-0') | ||
|
||
Stream: | ||
.. code-block:: python | ||
|
||
for chunk in model.stream(messages): | ||
print(chunk) | ||
|
||
.. code-block:: python | ||
|
||
content='' id='run-4e9f926b-73f5-483b-8ef5-09533d925853' | ||
content='The' id='run-4e9f926b-73f5-483b-8ef5-09533d925853' | ||
content=' English' id='run-4e9f926b-73f5-483b-8ef5-09533d925853' | ||
content=' sentence' id='run-4e9f926b-73f5-483b-8ef5-09533d925853' | ||
... | ||
content=' program' id='run-4e9f926b-73f5-483b-8ef5-09533d925853' | ||
content='".' id='run-4e9f926b-73f5-483b-8ef5-09533d925853' | ||
content='' response_metadata={'finish_reason': 'stop'} | ||
id='run-4e9f926b-73f5-483b-8ef5-09533d925853 | ||
|
||
.. code-block:: python | ||
|
||
stream = model.stream(messages) | ||
full = next(stream) | ||
for chunk in stream: | ||
full += chunk | ||
full | ||
|
||
.. code-block:: python | ||
|
||
AIMessageChunk(content='The English sentence "I love programming" | ||
can be translated to French as "J\'aime programmer". | ||
Here\'s the breakdown of the sentence:\n\n* "J\'aime" is the | ||
French equivalent of "I love"\n* "programmer" is the French | ||
infinitive for "to program"\n\nSo, the literal translation | ||
is "I love to program". However, in English we often omit the | ||
"to" when talking about activities we love, and the same applies | ||
to French. Therefore, "J\'aime programmer" is the correct and | ||
natural way to express "I love programming" in French.', | ||
response_metadata={'finish_reason': 'stop'}, | ||
id='run-a3c35ac4-0750-4d08-ac55-bfc63805de76') | ||
|
||
Async: | ||
.. code-block:: python | ||
|
||
await model.ainvoke(messages) | ||
|
||
.. code-block:: python | ||
|
||
AIMessage(content='The English sentence "I love programming" can | ||
be translated to French as "J\'aime programmer". The word | ||
"programming" is translated as "programmer" in French. I hope | ||
this helps! Let me know if you have any other questions.', | ||
response_metadata={'token_usage': {'completion_tokens': 53, | ||
'prompt_tokens': 28, 'total_tokens': 81, 'completion_time': | ||
0.083623752, 'prompt_time': 0.007365126, 'queue_time': None, | ||
'total_time': 0.090988878}, 'model_name': 'mixtral-8x7b-32768', | ||
'system_fingerprint': 'fp_c5f20b5bb1', 'finish_reason': 'stop', | ||
'logprobs': None}, id='run-897f3391-1bea-42e2-82e0-686e2367bcf8-0') | ||
|
||
Tool calling: | ||
.. code-block:: python | ||
|
||
from langchain_core.pydantic_v1 import BaseModel, Field | ||
|
||
class GetWeather(BaseModel): | ||
'''Get the current weather in a given location''' | ||
|
||
location: str = Field(..., description="The city and state, | ||
e.g. San Francisco, CA") | ||
|
||
class GetPopulation(BaseModel): | ||
'''Get the current population in a given location''' | ||
|
||
location: str = Field(..., description="The city and state, | ||
e.g. San Francisco, CA") | ||
|
||
model_with_tools = model.bind_tools([GetWeather, GetPopulation]) | ||
ai_msg = model_with_tools.invoke("What is the population of NY?") | ||
ai_msg.tool_calls | ||
|
||
.. code-block:: python | ||
|
||
[{'name': 'GetPopulation', | ||
'args': {'location': 'NY'}, | ||
'id': 'call_bb8d'}] | ||
|
||
See ``ChatGroq.bind_tools()`` method for more. | ||
|
||
Structured output: | ||
.. code-block:: python | ||
|
||
from typing import Optional | ||
|
||
from langchain_core.pydantic_v1 import BaseModel, Field | ||
|
||
class Joke(BaseModel): | ||
'''Joke to tell user.''' | ||
|
||
setup: str = Field(description="The setup of the joke") | ||
punchline: str = Field(description="The punchline to the joke") | ||
rating: Optional[int] = Field(description="How funny the joke | ||
is, from 1 to 10") | ||
|
||
structured_model = model.with_structured_output(Joke) | ||
structured_model.invoke("Tell me a joke about cats") | ||
|
||
.. code-block:: python | ||
|
||
Joke(setup="Why don't cats play poker in the jungle?", | ||
punchline='Too many cheetahs!', rating=None) | ||
|
||
See ``ChatGroq.with_structured_output()`` for more. | ||
|
||
Response metadata | ||
.. code-block:: python | ||
|
||
ai_msg = model.invoke(messages) | ||
ai_msg.response_metadata | ||
|
||
.. code-block:: python | ||
|
||
{'token_usage': {'completion_tokens': 70, | ||
'prompt_tokens': 28, | ||
'total_tokens': 98, | ||
'completion_time': 0.111956391, | ||
'prompt_time': 0.007518279, | ||
'queue_time': None, | ||
'total_time': 0.11947467}, | ||
'model_name': 'mixtral-8x7b-32768', | ||
'system_fingerprint': 'fp_c5f20b5bb1', | ||
'finish_reason': 'stop', | ||
'logprobs': None} | ||
""" | ||
|
||
client: Any = Field(default=None, exclude=True) #: :meta private: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think we probably just want to stick to the groq client params here, don't need to document cache and callbacks. model cache is usually better specified globally https://python.langchain.com/v0.2/docs/how_to/chat_model_caching/ and callbacks are generally better passed in at runtime