using in scripting

[simit22]

hello sorry to ask can i use my installed models in a python script as well
for example it do x and y and than ask a certain quesion from ai model and continue according to answer
if it is possible how can i use it inside python scripts
it doesnt just have to be python any other way to use it for code or scripting is also fine !!

[Jeffser]

Hi, Alpaca is developed in Python so of course you can also do some cool stuff with AI yourself using that language.

I suggest you look into this documentation page.

There's also this python library that might make it easier for you to start playing with Ollama.

Alpaca doesn't use that library because of some technical limitations but it will probably be enough for you to get started.

Also please note that Alpaca exposes the Ollama instance using the port 11435 instead of 11434.

[simit22]

Hi, Alpaca is developed in Python so of course you can also do some cool stuff with AI yourself using that language.

I suggest you look into this documentation page.

There's also this python library that might make it easier for you to start playing with Ollama.

Alpaca doesn't use that library because of some technical limitations but it will probably be enough for you to get started.

Also please note that Alpaca exposes the Ollama instance using the port 11435 instead of 11434.

after using alpaca for a while i think it use cpu only can i enable gpu for it as well ???

[Jeffser]

Nvidia should work by itself, Ollama only supports these AMD GPUs, to activate AMD support please read this wiki article

[CodingKoalaGeneral]

If you GPU does not have enough VRAM it uses the CPU (in case your PC has enough RAM)

use medium models for them to be able to fit in the VRAM.

What GPU do you have? Does your pc contain more than one GPU?

from ollama import Client

def main():
    client = Client(host='http://localhost:11434')

    print("Welcome to the Qwen2.5-Coder:0.5b CLI Chatbot!")
    print("Type 'exit' to end the chat.\n")

    conversation = []

    while True:
        user_input = input("You: ")
        if user_input.lower() == 'exit':
            print("Ending the chat. Goodbye!")
            break

        conversation.append({'role': 'user', 'content': user_input})

        response = client.chat(model='qwen2.5-coder:0.5b', messages=conversation)

        assistant_message = response['message']['content']
        print(f"Assistant: {assistant_message}\n")

        conversation.append({'role': 'assistant', 'content': assistant_message})

if __name__ == "__main__":
    main()

dep: pip install ollama