json: cannot unmarshal object into Go struct field

[h-b]

Environment

codegpt version: 0.14.2 (commit: fd5758c)
Model: Gemma 2 (via local KoboldCpp 1.72 instance)

I’ve configured codegpt to use a local KoboldCpp instance:

codegpt config set openai.base_url http://localhost:5001/v1
codegpt config set openai.timeout 1m

Issue Description

The configured koboldcpp endpoint is designed to be compatible with the OpenAI API format, see https://github.com/LostRuins/koboldcpp#Notes. Given this API compatibility, codegpt review operates without throwing errors (although the output quality depends on the specific model used). However, codegpt commit encounters a persistent issue, resulting in the following error:

❯ codegpt commit
Summarize the commit message use gpt-3.5-turbo model
We are trying to summarize a git diff
PromptTokens: 100, CompletionTokens: 100, TotalTokens: 200
We are trying to summarize a title for pull request
PromptTokens: 100, CompletionTokens: 100, TotalTokens: 200
We are trying to get conventional commit prefix (Tools)
Error: json: cannot unmarshal object into Go struct field ChatCompletionChoice.choices.message of type string

Expected Behavior

The command should process the commit message successfully, even if the model outputs unusual characters.

[appleboy]

I will take it.

[appleboy]

I encounter the following problem.

***
Welcome to KoboldCpp - Version 1.74
For command line arguments, please refer to --help
***
Warning, GUI failed to start: Reason: no display name and no $DISPLAY environment variable
File selection GUI unsupported.
customtkinter python module required!
Please check command line: script.py --help

[h-b]

Hello,

Thank you for your update on this issue. While I'm not a maintainer of KoboldCpp, I can provide some general guidance based on the official documentation.

It appears that you're encountering unnecessary complications when trying to start KoboldCpp. The error message suggests that additional GUI-related features are being triggered, which isn't necessary for basic operation.

To run KoboldCpp as intended, you should only need to start the server component. Here's the simplest way to do this, as per the official documentation at https://github.com/LostRuins/koboldcpp?tab=readme-ov-file#readme

I start it like that, but this differs, depending on your OS, your hardware and the model.

python /usr/share/koboldcpp/koboldcpp.py --quiet --usecublas normal 0 --gpulayers 0 --threads 15 --blasthreads 15 /usr/share/koboldcpp/models/gemma-2-2b-it-Q6_K.gguf

No additional UI or software should be necessary. The customtkinter error you're seeing suggests that something is trying to start a GUI, which isn't required. The endpoint is available after starting the command line tool. In addition, there will be a web interface on http://localhost:5001

If you continue to face issues with this basic setup, I recommend reaching out to the KoboldCpp community or official support channels for more specific assistance.

Best regards,
Stefan