Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set num_threads model parameter #360

Open
71f681c20d opened this issue Oct 24, 2024 · 2 comments
Open

Set num_threads model parameter #360

71f681c20d opened this issue Oct 24, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@71f681c20d
Copy link

Is your feature request related to a problem? Please describe.
When I run Alpaca, the Ollama instance is under-utilizing my resources, resulting in slower outputs.

Describe the solution you'd like
A menu option under Preferences>General for setting the number of threads as an Ollama model parameter

Describe alternatives you've considered
Editing config files for the ollama model

Additional context
This would add a lot of convenience for people who want to configure how many cores the ollama instance can use, and it would be nice easily change this value for each model. I currently have a change implemented already, and I'm testing it now.

@71f681c20d 71f681c20d added the enhancement New feature or request label Oct 24, 2024
@71f681c20d
Copy link
Author

I have a change implemented already, but still need to test it out. I'll do that today. Can I submit this issue?

@CodingKoalaGeneral
Copy link

I have a change implemented already, but still need to test it out. I'll do that today.

@71f681c20d any updates on that? i could see why this feature may be handy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants