Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc]: Invalid JSON examples in Engine Args Document #11965

Closed
1 task done
ardapekis opened this issue Jan 12, 2025 · 0 comments · Fixed by #12045
Closed
1 task done

[Doc]: Invalid JSON examples in Engine Args Document #11965

ardapekis opened this issue Jan 12, 2025 · 0 comments · Fixed by #12045
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers help wanted Extra attention is needed

Comments

@ardapekis
Copy link

📚 The doc issue

On page https://docs.vllm.ai/en/latest/serving/engine_args.html#engine-args

Regarding the flag --override-pooler-config. The documentation provides the following example:

Override or set the pooling method for pooling models. e.g. {“pooling_type”: “mean”, “normalize”: false}.’

However this example does not work if copy-pasted into a UTF-8 aware text editor as it is not a valid JSON document. (The quotation marks are not ascii quotation marks, they are left-quote and right-quote.) This is an insidious error as it is nearly invisible to the naked eye.

In addition to --override-pooler-config, this issue affects --override-neuron-config, --rope-scaling, and --mm-processor-kwargs.

Suggest a potential alternative/fix

Change

Override or set the pooling method for pooling models. e.g. {“pooling_type”: “mean”, “normalize”: false}.’

to

Override or set the pooling method for pooling models. e.g. {"pooling_type": "mean", "normalize": false}.’

(Replace non-ascii quotes with ascii quotes and surround with a code block to ensure verbatim inclusion.)

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
@ardapekis ardapekis added the documentation Improvements or additions to documentation label Jan 12, 2025
@DarkLight1337 DarkLight1337 added help wanted Extra attention is needed good first issue Good for newcomers labels Jan 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants