Add support for nvidia modelopt fp8 kv cache

[Edwardf0t1]

Motivation

Add support for modelopt fp8 kv cache, so SGLang can run modelopt's quantized models with fp8 kv cache enabled.

Modifications

• Add a ModelOptFp8KVCacheMethod class.
• Handle kv cache scalers name matching.

Test

nvidia/Llama-3.1-8B-Instruct-FP8 will be updated with FP8 kv cache enabled.

import sglang as sgl

def main():
    prompts = [
        "Hello, my name is",
        "The president of the United States is",
        "The capital of France is",
        "The future of AI is",
    ]
    sampling_params = {"temperature": 0.8, "top_p": 0.95}
    llm = sgl.Engine(model_path="nvidia/Llama-3.1-8B-Instruct-FP8", quantization="modelopt")

    outputs = llm.generate(prompts, sampling_params)
    for prompt, output in zip(prompts, outputs):
        print("===============================")
        print(f"Prompt: {prompt}\nGenerated text: {output['text']}")

if __name__ == "__main__":
    main()

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling.