[Feature Request] Support register customize quantization method out-of-tree

[ice-tong]

Feature request

Support register customize quantization method out-of-tree.

The usage would be as follows:

from transformers.quantizers import HfQuantizer
from transformers.quantizers import regsiter_quantization_config, register_quantizer
from transformers.utils.quantization_config import QuantizationConfigMixin


@regsiter_quantization_config("custom")
class CustomFakeQuantizationConfig(QuantizationConfigMixin):
    """The custom fake quantization config."""


@register_quantizer("custom")
class CustomFakeQuantizer(HfQuantizer):
    """The custom fake quantizer."""

Motivation

We would greatly appreciate it if HuggingFace could support registering custom quantization schemes externally. This would allow us to integrate the schemes of any LLM quantization tools and evaluate fake quantization models using the powerful combination of lm_eval + huggingface. Thank you for considering this!

Similar features have already been supported by vLLM, see:

• [Feature]: Support out-of-tree register of customize quantization method vllm-project/vllm#11926
• [Misc] Support register quantization method out-of-tree vllm-project/vllm#11969

Your contribution

If this feature request is considered, I'd happily submit a PR to implement it.

[Rocketknight1]

cc @SunMarc @MekkCyber

[SunMarc]

Looks like a nice feature ! Feel free to submit a PR for that ! cc @mht-sharma, I think you wanted to have something like that too