[QUESTION] <title> Qwen2-Audio loading issue - RuntimeError: Error during inference: Failed to load audio language model

[baibhavnfs2001]

Question or Issue

I tried running the qwen2audio model but it failed to load it. Can someone help me out.

`Model Qwen2-Audio-7.8B-Instruct:model-q4_K_M already exists at /home/innovation/.cache/nexa/hub/official/Qwen2-Audio-7.8B-Instruct/model-q4_K_M.gguf
Downloading Qwen2-Audio-7.8B-Instruct/projector-q4_K_M.gguf...
projector-q4_K_M.gguf: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.21G/1.21G [02:34<00:00, 8.39MB/s]
Verifying download: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.21G/1.21G [00:01<00:00, 1.06GB/s]
Successfully downloaded Qwen2-Audio-7.8B-Instruct/projector-q4_K_M.gguf to /home/innovation/.cache/nexa/hub/official/Qwen2-Audio-7.8B-Instruct/projector-q4_K_M.gguf
Successfully pulled model Qwen2-Audio-7.8B-Instruct:projector-q4_K_M to /home/innovation/.cache/nexa/hub/official/Qwen2-Audio-7.8B-Instruct/projector-q4_K_M.gguf, run_type: AudioLM

/home/innovation/Downloads/recordings/D0420-S1-T01.wav
What is the gender of the speaker
2025-01-16 18:27:46,508 - ERROR -
Error during audio generation: Error during inference: Failed to load audio language model
Traceback (most recent call last):
File "nexa/gguf/nexa_inference_audio_lm.py", line 215, in inference
RuntimeError: Failed to load audio language model

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "nexa/gguf/nexa_inference_audio_lm.py", line 170, in run
File "nexa/gguf/nexa_inference_audio_lm.py", line 223, in inference
RuntimeError: Error during inference: Failed to load audio language model

`

OS

Ubuntu 22.04

Python Version

3.12.2

Nexa SDK Version

0.0.9.7

GPU (if using one)

GeForce GTX 1050 Ti