Giving complete giberish results when i give the input to the model along with that it gives some warning with the tokenizer

[Akshay1-6180]

CODE:

import transformers
import torch
tokenizer = transformers.LlamaTokenizer.from_pretrained('chaoyi-wu/PMC_LLAMA_7B')
model = transformers.LlamaForCausalLM.from_pretrained('chaoyi-wu/PMC_LLAMA_7B',
                                                        torch_dtype=torch.bfloat16,
                                                        low_cpu_mem_usage=True,
                                                        device_map="auto",)
sentence = 'Hello, doctor , I have a huge pain in my chest , what could it be , can it be cancer' 
batch = tokenizer(
            sentence,
            return_tensors="pt", 
            add_special_tokens=False
        )
with torch.no_grad():
    generated = model.generate(inputs = batch["input_ids"], max_length=200, do_sample=True, top_k=50)
    print('model predict: ',tokenizer.decode(generated[0]))

it gave this
model predict: Hello, doctor , I have a huge pain in my chest , what could it be , can it be cancer and will they need to take a part of my heart or not ?" [P02].

Theme 2-Disease in relation to gender

In this theme, men are considered more rational and knowledgeable about their diseases while the women are seen as irrational. This is clearly seen from the answers:

'They don't know what happens to their heart, they just believe the patient's heart will go out of control and they will be dead, so they cannot say anything about their heart disease ' [P02].

'I am a mother, how can I think about something serious like my heart , I don't worry . ' [P05].

It seems for women, the disease is very severe since it kills many people in the family or close relatives, so

Along with that it gave some warnings
UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )

[chaoyi-wu]

Maybe, try our latest version https://huggingface.co/axiong/PMC_LLaMA_13B which has been instruction tuned and responses to chatting input much better.

The version MedLLaMA is a basic model without applying to various instruction. In zero-shot setting, you may consider it as always performing sentence completion instead of chatting or other things.

[Akshay1-6180]

its giving same issues with that too @chaoyi-wu @eltociear @xiaoman-zhang @WeixiongLin

import transformers
import torch

tokenizer = transformers.LlamaTokenizer.from_pretrained('axiong/PMC_LLaMA_13B')
model = transformers.LlamaForCausalLM.from_pretrained('axiong/PMC_LLaMA_13B')

sentence = 'Hello, doctor' 
batch = tokenizer(
    sentence,
    return_tensors="pt", 
    add_special_tokens=False
)

with torch.no_grad():
    generated = model.generate(
        inputs = batch["input_ids"],
        max_length=200,
        do_sample=True,
        top_k=50
    )
    print('model predict: ',tokenizer.decode(generated[0]))

You are using the legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This means that tokens that come after special tokens will not be properly handled. We recommend you to read the related pull request available at huggingface/transformers#24565
[2023-09-07 16:22:27,715] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████| 6/6 [02:08<00:00, 21.40s/it]/home/users/aks/miniconda3/envs/llm/lib/python3.11/site-packages/transformers/generation/utils.py:1270: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )
warnings.warn(
model predict: Hello, doctor can you hear voices as well Hello, Hello, doctor can you hear me now patient No, you are a doctor, you are a doctor Hello, can I go outside and buy a sandwich? patient No, not now, you have to take your medications first. Hello, doctor can you hear me now Hello, Hello, can I leave the room now patient No, you have to wait for the medications. Hello, doctor can you hear me now patient Hello, no, you are a patient. You are ill, why are you doing this Hello, can I go outside and buy a sandwich? patient No, you need to take your medications. Hello, can I leave the room now patient No, you have to wait for the medications. patient You are a doctor, why are you doing this Hello, can I go outside and buy a sandwich? patient No, not now, take your medications first. patient You are a doctor, why