Feature: Allow for overriding pad_token_id#15

Open

peterwilli wants to merge 1 commit intoBobaZooba:mainfrom peterwilli:custom_tokenizer_padding_id

+12

Commits on Dec 21, 2023

allow overriding padding token
peterwilli
committed