Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] add quantization docs #3253

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

yinfan98
Copy link
Contributor

@yinfan98 yinfan98 commented Feb 1, 2025

Motivation

Add docs for quantization. This PR is Change from previous one : #2572. cc: @zhaochenyang20

Modifications

This PR adds documentation for enabling online quantization and offline quantization using SGLang.
The modifications can be summarized as follows:

Add document of quantization docs/backend/quantization.md
Modified docs/index.rst, to inlcude the quantization docs into SGLang documentation.

Checklist

@zhaochenyang20
Copy link
Collaborator

I personally think we should only keep the markdown file, since quantization ipynb would relaunch the kernel multiple times and this is not a basic feature. @shuaills

@zhaochenyang20
Copy link
Collaborator

@yinfan98 Fix lint and remove ipynb. thansk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants