[Feature Request] Support for chain-of-thought models

[acolombier]

Is your feature request related to a problem? Please describe.

With the recent release of deepseek-r1, it would be great to add proper support to chain-of-thoughts model

Describe the solution you'd like

A valid solution would include

• A correct title, potentially generated by another model (optional): Alpaca uses the same model to generate title for the prompt, but chain of thought model aren't a great usecase for it. (See context)
• A better UI rendering allowing collapsing of the thoughts: the chain of thoughts can be very verbose, and while it is great to be able to see it, most of the time it is artifacts you don't need to as part of your prompt answer. This makes the "chat" hard to interact with (See context)
• A better responsiveness: currently, Alpaca is not responding well during the generation, beside using a dedicated server and thus not having the current machine under load

Describe alternatives you've considered

N/A

Additional context

[Jeffser]

Hi, thanks for the suggestion. I was actually looking into this to get Deepseek R1 working better for the next Alpaca release.

I didn't noticed the title issue. For now I'll make it so that Alpaca uses the default model for that, in the future I'll give it it's own option, something like title generation model.

I'm also working on a new widget for handling the <think> blocks.

[Jeffser]

First attempt:

[Jeffser]

I'll set this as finished

2a60dff
a360058

[Jeffser]

Reopening so people know this is done, I also changed the appearance to an attachment button, it opens a file previewer with the text in it.

[olumolu]

Make the thought part integrated in the chat it is not good looking and unintiative how deepseek do it in there app... Which is actually better.

[acolombier]

Looking very promising @Jeffser , thanks for that quick reaction. I'll give it a spin during the next few days.

[Aleksanaa]

Two problems:

1. Thoughts is also in markdown format, but isn't rendered as such in popup view.
2. <Think> isn't hidden in chat title.

[Jeffser]

The whole markdown processing thing is attached to the message widget (not attachments), I will make it more generic later so that it works in attachments too.

The default model set in preferences is now in charge of generating the titles, you should probably select one that isn't chain of thought, that's the solution for now.

I plan on rewriting all the instance manager code, and when I do, I'll separate the option to default model and title generating model.

[CodingKoalaGeneral]

please add an option to hide the thoughts like chatgpt does
since they often do not provide actual value to the user, who is just interested in the final result

(just show the thoughs bubble icon in ui then and user can click to expand it and see the generation )

deepseek has sometimes annoying emulated vocal expressions like hmmmmm let me see ...

off topic:
also the models (in general) seem to lose recent and origin context of conversations rather easily which can be quite annoying (as soon as previous and recent request are large in token input amount)

for example i provided the model with a large script and asked to fix one part and when i said combine the updated code regarding the fix with the original script, ai be like: what script? generating random wanna be placeholders around the fixed previous requested code

[acolombier]

Finally got the update over flatpak - sorry I didn't have time to build the test flatpak before! It's looking great, thanks for that! :)

One "bug" I found is, if you ask the model (R1 in my case) "forbidden" question (e.g historical facts), it will return an empty <thought> block, which will display a thought button but no action attached to it.

It would be great if the rendering of the thought section could be dynamic, rather than waiting for the whole response to generate, but this is probably a separated feature request. Is that something you are already working on?

Anyway, happy to close this issue if you would prefer to track follow up work in other ones