NeMo is not friendly to HF compatibility.

[dyang67]

Why isn’t NeMo compatible with HF’s checkpoints?

Each conversion causes many compatibility issues, significantly increasing Development costs and time.

Since HF models are updated daily, AI models' architectures undergo frequent changes, ensuring compatibility is quite challenging.

Give an example.

Mistral offers various models, such as NeMo-12B, Mistral-Small 22B, and the latest Mistral 24B. However, NeMo can only perfectly convert Mistral-7B and Mixtral-7*8B. With significant changes in Mistral 24B, converting HF checkpoints requires substantial effort and meticulous validation, creating a considerable challenge for developers.

Has NeMo considered being more compatible with HF? Directly load HF models without checkpoint conversion？

[akoumpa]

Hi @dyang67 thank you for the feedback,

we are working on making NeMo more HF compatible.

This work is under the term "nemo-automodel", you can find peft examples here and here, as well sft.

This is still under-development and we hope to have a first release out soon. Thank you.