Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialize plugins only when their models are used #761

Open
polyrand opened this issue Feb 19, 2025 · 0 comments
Open

Initialize plugins only when their models are used #761

polyrand opened this issue Feb 19, 2025 · 0 comments

Comments

@polyrand
Copy link

I recently started passing API keys as CLI flags instead of storing them in the environment or the keys file. When I try to run llm with a plugin that requires an API key, there's an error even if I'm not using the model from the plugin.

For example, this works as expected:

uvx --with "llm-anthropic" "llm@latest" -m "claude-3.5-sonnet-latest" --key "${KEY:?}" "tell me a joke"

But this breaks:

uvx --with "llm-together" --with "llm-anthropic" "llm@latest" -m "claude-3.5-sonnet-latest" --key "${KEY:?}" "tell me a joke"
llm.errors.NeedsKeyException: No key found - add one using 'llm keys set together' or set the TOGETHER_API_KEY environment variable

I understand this is probably a bug on llm-together because there aren't any errors if I add --with "llm-gemini" or --with "llm-groq" to the command. For example, this works:

uvx --with "llm-gemini" --with "llm-anthropic" "llm@latest" -m "claude-3.5-sonnet-latest" --key "${KEY:?}" "tell me a joke"

But I'm wondering if it would be better to only initialize plugins if the model they are registering is actually used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant