Unable to set the max context limit

[kyllohd]

Bug Report

You should be able to override the max context window for the models. Especially because it seems to be enforcing a stricter limit based on the base model and not the model in use:

This should have a 128k context window, but GPT4All enforces 4096
https://huggingface.co/failspy/Phi-3-mini-128k-instruct-abliterated-v3-GGUF/tree/main

The same with https://huggingface.co/mradermacher/Llama-3-8B-source-lewd-context-GGUF which should allow for also an insenely high context window, but it is capped at 8k.

Steps to Reproduce

1. Download either of these models
2. Try to set the context window to something larger e.g 12288
3. The interface, sets the max window to a different value e.g 4096 or 8192

Expected Behavior

These models that were made to allow for an extended context should allow such contexts to be made.

Your Environment

• GPT4All version:
• Operating System:
• Chat model used (if applicable):

[ThiloteE]

Please, I want to know, how do you know the context window is set to max 4096? Screenshots? Source in the code?

By the way, I've added #2790

[cebtenzzre]

Cannot reproduce with the latest main branch—I can set the context limit as high as 131072. I'll have to assume this was fixed (since OP did not specify which version they tried, and this was opened a while ago).

[ningkaiyang]

I'm having this same problem right now. Going to try to install some other models with larger parameters, but I basically cannot set my context limit past exactly 4096 (for a 13B param model) or 8192 (llama 8B)

I type in anything larger (even by 1), and press enter to submit, and it reverts back to 4096 or 8192. I can set it down as low as 8 (by typing 0 and pressing enter it auto-goes to lowest), but when trying to set maximum it is limited to tiny amounts.

GPT4All version: 3.4.2 (installed today)
Operating System: MacOS sequoia 15.0.1
Chat model used (if applicable):
speechless-llama2-hermes-orca-platypus-wizardlm-13b.Q8_0.gguf (limited to 4096)
Meta-Llama-3-8B-Instruct.Q4_0.gguf (limited to 8192, strangely larger)

EDIT:
Installed
Qwen-2.5-Coder 32B, now see expected behavior for just this model, for up to 131072 context size.