Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Standardize ChatGroq #22751

Merged
merged 7 commits into from
Jun 14, 2024
211 changes: 209 additions & 2 deletions libs/partners/groq/langchain_groq/chat_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,15 +85,222 @@
To use, you should have the
environment variable ``GROQ_API_KEY`` set with your API key.

Any parameters that are valid to be passed to the groq.create call can be passed
in, even if not explicitly saved on this class.
Any parameters that are valid to be passed to the groq.create call

Check failure on line 88 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:88:71: W291 Trailing whitespace

Check failure on line 88 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:88:71: W291 Trailing whitespace
can be passed in, even if not explicitly saved on this class.

Example:
.. code-block:: python

from langchain_groq import ChatGroq

model = ChatGroq(model_name="mixtral-8x7b-32768")

Setup:
Install ``langchain-groq`` and set environment variable

Check failure on line 99 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:99:64: W291 Trailing whitespace

Check failure on line 99 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:99:64: W291 Trailing whitespace
``GROQ_API_KEY``.

.. code-block:: bash

pip install -U langchain-groq
export GROQ_API_KEY="your-api-key"

Key init args — completion params:
model_name: str
Name of Groq model to use. E.g. "mixtral-8x7b-32768".
temperature: float
Sampling temperature. Ranges from 0.0 to 1.0.
max_tokens: Optional[int]
Max number of tokens to generate.

Key init args — client params:
cache: Union[BaseCache, bool, None]
Whether to cache the response. Uses global cache if true, does not
use a cache if false, uses global cache (if it is set) if None,
and uses the provided cache if is instance of BaseCache.
callbacks: Callbacks
Callbacks to add to the run trace.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we probably just want to stick to the groq client params here, don't need to document cache and callbacks. model cache is usually better specified globally https://python.langchain.com/v0.2/docs/how_to/chat_model_caching/ and callbacks are generally better passed in at runtime

custom_get_token_ids: Optional[Callable[[str], List[int]]]
Optional encoder to use for counting tokens.
groq_api_base: Optional[str]
Base URL path for API requests, leave blank if not using a proxy

Check failure on line 125 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:125:77: W291 Trailing whitespace

Check failure on line 125 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:125:77: W291 Trailing whitespace
or service emulator.
metadata: Optional[Dict[str, Any]]
Metadata to add to the run trace.
model_kwargs: Dict[str, Any]
Holds any model parameters valid for create call not

Check failure on line 130 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:130:65: W291 Trailing whitespace

Check failure on line 130 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:130:65: W291 Trailing whitespace
explicitly specified.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: these are technically completion params

streaming: bool = False
Whether to stream the results or not.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're generally moving off using this (recommended approach is just to call llm.stream directly) so don't think worth noting here


See full list of supported init args and their descriptions in the params
section.

Instantiate:
.. code-block:: python

from langchain_groq import ChatGroq

model = ChatGroq(
model_name="mixtral-8x7b-32768",
temperature=0.0,
max_retries=2,
# streaming=...,
# other params...
)

Invoke:
.. code-block:: python

messages = [
("system", "You are a helpful translator. Translate the user

Check failure on line 155 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:155:77: W291 Trailing whitespace

Check failure on line 155 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:155:77: W291 Trailing whitespace
sentence to French."),
("human", "I love programming."),
]
model.invoke(messages)

.. code-block:: python

AIMessage(content='The English sentence "I love programming" can

Check failure on line 163 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:163:77: W291 Trailing whitespace

Check failure on line 163 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:163:77: W291 Trailing whitespace
be translated to French as "J\'aime programmer". The word

Check failure on line 164 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:164:70: W291 Trailing whitespace

Check failure on line 164 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:164:70: W291 Trailing whitespace
"programming" is translated as "programmer" in French.',

Check failure on line 165 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:165:69: W291 Trailing whitespace

Check failure on line 165 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:165:69: W291 Trailing whitespace
response_metadata={'token_usage': {'completion_tokens': 38,

Check failure on line 166 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:166:72: W291 Trailing whitespace

Check failure on line 166 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:166:72: W291 Trailing whitespace
'prompt_tokens': 28, 'total_tokens': 66, 'completion_time':

Check failure on line 167 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.8

Ruff (W291)

langchain_groq/chat_models.py:167:72: W291 Trailing whitespace

Check failure on line 167 in libs/partners/groq/langchain_groq/chat_models.py

View workflow job for this annotation

GitHub Actions / cd libs/partners/groq / make lint #3.11

Ruff (W291)

langchain_groq/chat_models.py:167:72: W291 Trailing whitespace
0.057975474, 'prompt_time': 0.005366091, 'queue_time': None,
'total_time': 0.063341565}, 'model_name': 'mixtral-8x7b-32768',
'system_fingerprint': 'fp_c5f20b5bb1', 'finish_reason': 'stop',
'logprobs': None}, id='run-ecc71d70-e10c-4b69-8b8c-b8027d95d4b8-0')

Stream:
.. code-block:: python

for chunk in model.stream(messages):
print(chunk)

.. code-block:: python

content='' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content='The' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content=' English' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content=' sentence' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
...
content=' program' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content='".' id='run-4e9f926b-73f5-483b-8ef5-09533d925853'
content='' response_metadata={'finish_reason': 'stop'}
id='run-4e9f926b-73f5-483b-8ef5-09533d925853

.. code-block:: python

stream = model.stream(messages)
full = next(stream)
for chunk in stream:
full += chunk
full

.. code-block:: python

AIMessageChunk(content='The English sentence "I love programming"
can be translated to French as "J\'aime programmer".
Here\'s the breakdown of the sentence:\n\n* "J\'aime" is the
French equivalent of "I love"\n* "programmer" is the French
infinitive for "to program"\n\nSo, the literal translation
is "I love to program". However, in English we often omit the
"to" when talking about activities we love, and the same applies
to French. Therefore, "J\'aime programmer" is the correct and
natural way to express "I love programming" in French.',
response_metadata={'finish_reason': 'stop'},
id='run-a3c35ac4-0750-4d08-ac55-bfc63805de76')

Async:
.. code-block:: python

await model.ainvoke(messages)

.. code-block:: python

AIMessage(content='The English sentence "I love programming" can
be translated to French as "J\'aime programmer". The word
"programming" is translated as "programmer" in French. I hope
this helps! Let me know if you have any other questions.',
response_metadata={'token_usage': {'completion_tokens': 53,
'prompt_tokens': 28, 'total_tokens': 81, 'completion_time':
0.083623752, 'prompt_time': 0.007365126, 'queue_time': None,
'total_time': 0.090988878}, 'model_name': 'mixtral-8x7b-32768',
'system_fingerprint': 'fp_c5f20b5bb1', 'finish_reason': 'stop',
'logprobs': None}, id='run-897f3391-1bea-42e2-82e0-686e2367bcf8-0')

Tool calling:
.. code-block:: python

from langchain_core.pydantic_v1 import BaseModel, Field

class GetWeather(BaseModel):
'''Get the current weather in a given location'''

location: str = Field(..., description="The city and state,
e.g. San Francisco, CA")

class GetPopulation(BaseModel):
'''Get the current population in a given location'''

location: str = Field(..., description="The city and state,
e.g. San Francisco, CA")

model_with_tools = model.bind_tools([GetWeather, GetPopulation])
ai_msg = model_with_tools.invoke("What is the population of NY?")
ai_msg.tool_calls

.. code-block:: python

[{'name': 'GetPopulation',
'args': {'location': 'NY'},
'id': 'call_bb8d'}]

See ``ChatGroq.bind_tools()`` method for more.

Structured output:
.. code-block:: python

from typing import Optional

from langchain_core.pydantic_v1 import BaseModel, Field

class Joke(BaseModel):
'''Joke to tell user.'''

setup: str = Field(description="The setup of the joke")
punchline: str = Field(description="The punchline to the joke")
rating: Optional[int] = Field(description="How funny the joke
is, from 1 to 10")

structured_model = model.with_structured_output(Joke)
structured_model.invoke("Tell me a joke about cats")

.. code-block:: python

Joke(setup="Why don't cats play poker in the jungle?",
punchline='Too many cheetahs!', rating=None)

See ``ChatGroq.with_structured_output()`` for more.

Response metadata
.. code-block:: python

ai_msg = model.invoke(messages)
ai_msg.response_metadata

.. code-block:: python

{'token_usage': {'completion_tokens': 70,
'prompt_tokens': 28,
'total_tokens': 98,
'completion_time': 0.111956391,
'prompt_time': 0.007518279,
'queue_time': None,
'total_time': 0.11947467},
'model_name': 'mixtral-8x7b-32768',
'system_fingerprint': 'fp_c5f20b5bb1',
'finish_reason': 'stop',
'logprobs': None}
"""

client: Any = Field(default=None, exclude=True) #: :meta private:
Expand Down
Loading