'str' object has no attribute 'parameters'

[playful1510]

To Reproduce
Steps to reproduce the behavior:

1. Use 'start_with_LLM.json' workflow.
2. Load local PIxtral-12B-2409 LLM and any PDF file.
3. Have input in LLM General Link system_prompt field.
4. Have ANY input in LLM General Link user_prompt field.

[heshengtao]

Which workflow does start_with_LLM json use, how is the local model loaded, direct loading or ollama loading? Is there a screenshot of the workflow?

[playful1510]

'start_with_LLM_local' workflow is one included with LLM Party. Model is loaded locally and directly. The only modification made between original workflow is adding a file_content input for a PDF and disabling memory.

[heshengtao]

I checked and found that the model you provided is only compatible with VLLM loading. I really want VLLM to become one of the loading methods for Party, but VLLM itself relies on too many things, and Party can't possibly add this library. Party mainly relies on transformer and llama cpp to load models, and only models that are compatible with these two methods can be directly loaded by Party.
If you want to use this model you've provided, you need to convert it with the VLLM library. You can easily deploy an OpenAI-style VLLM server using Docker, and then use Party's LLM API loader to use the model in VLLM, for example, using the following command line:

vllm serve mistralai/Pixtral-12B-2409 --dtype auto --api-key token-abc123