Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Logprobs #45

Open
wants to merge 1,231 commits into
base: afeldman-nm/logprobs
Choose a base branch
from
Open

Logprobs #45

wants to merge 1,231 commits into from

Conversation

robertgshaw2-redhat
Copy link
Collaborator

FILL IN THE PR DESCRIPTION HERE

FIX #xxxx (link existing issues this PR will resolve)

**BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing/overview.html **

patrickvonplaten and others added 30 commits November 30, 2024 07:45
@patrickvonplaten @youkaichao
[Interleaved ATTN] Support for Mistral-8B (vllm-project#10591)
e7cfc4e 
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@wangxiyuan
[doc] format fix (vllm-project#10789)
7e4bbda 
Signed-off-by: wangxiyuan <[email protected]>
@DarkLight1337
[Model] Replace embedding models with pooling adapter (vllm-project#1…
1337071 
…0769)

Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Misc] Improve type annotations for support_torch_compile (vllm-pro…
f877a7d 
…ject#10763)

Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Misc] Rename embedding classes to pooling (vllm-project#10801)
d2f058e 
Signed-off-by: DarkLight1337 <[email protected]>
@youkaichao
[doc] add warning about comparing hf and vllm outputs (vllm-project#1…
169a0ff 
…0805)

Signed-off-by: youkaichao <[email protected]>
@ywang96 @heheda12345 @Isotr0py
[Misc] Adding MMMU-Pro vision dataset to serving benchmark (vllm-pr…
c11f172 
…oject#10804)

Signed-off-by: Roger Wang <[email protected]>
Co-authored-by: Chen Zhang <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
f446a78
@afeldman-nm
removed fast tests from pipeline
fda0fcb
@KuntaiDu
[Core] Implement disagg prefill by StatelessProcessGroup (vllm-projec…
0590ec3 
…t#10502)

This PR provides initial support for single-node disaggregated prefill in 1P1D scenario.
Signed-off-by: KuntaiDu <[email protected]>
Co-authored-by: ApostaC <[email protected]>
Co-authored-by: YaoJiayi <[email protected]>
@Isotr0py
[Model] Add BNB support to Llava and Pixtral-HF (vllm-project#10795)
b18c9bb 
Signed-off-by: Isotr0py <[email protected]>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
0bcf5f4
@cduk
[core] Avoid metrics log noise when idle - include speculative decodi… (
b795477 
vllm-project#10809)
@WoosukKwon
[Kernel] Use out arg in flash_attn_varlen_func (vllm-project#10811)
073a4bd 
Signed-off-by: Woosuk Kwon <[email protected]>
@maxdebayser
Fill TorchSDPAAttentionMetadata seq_lens_field for prefill (vllm-proj…
e25810a 
…ect#10799)

Signed-off-by: Max de Bayser <[email protected]>
@youkaichao
[misc] remove xverse modeling file (vllm-project#10814)
63a1641 
Signed-off-by: youkaichao <[email protected]>
@wangxiyuan
[doc]Update config docstring (vllm-project#10732)
995a148 
Signed-off-by: wangxiyuan <[email protected]>
@xffxff @Isotr0py
[Model]: add some tests for aria model (vllm-project#10770)
ef31eab 
Signed-off-by: xffxff <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
ee1b910
@afeldman-nm @WoosukKwon
Update vllm/outputs.py
1fa0b71 
Co-authored-by: Woosuk Kwon <[email protected]>
@afeldman-nm
Merge branch 'afeldman-nm/v1_logprobs' of https://github.com/neuralma…
6ba743f 
…gic/vllm into v1_logprobs
@afeldman-nm
small fixes
bc1c004
@DarkLight1337
[CI/Build] Update mistral_common version for tests and docs (vllm-p…
e95f275 
…roject#10825)

Signed-off-by: DarkLight1337 <[email protected]>
@afeldman-nm
moved output processing commands into processor
bec886b 
Signed-off-by: Andrew Feldman <[email protected]>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
fef9f30
@youkaichao
[misc] use out argument for flash attention (vllm-project#10822)
a4c4daf 
Signed-off-by: youkaichao <[email protected]>
@afeldman-nm
added explanatory comment to EngineCore.update_from_output()
554f431
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
5022307
@youkaichao @afeldman-nm
[misc] move functions to config.py (vllm-project#10624)
5dea1d5 
Signed-off-by: youkaichao <[email protected]>
Signed-off-by: Andrew Feldman <[email protected]>
@DarkLight1337 @afeldman-nm
[Model] Support is_causal HF config field for Qwen2 model (vllm-pro…
930f2cc 
…ject#10621)

Signed-off-by: DarkLight1337 <[email protected]>
Signed-off-by: Andrew Feldman <[email protected]>
DarkLight1337 and others added 29 commits December 7, 2024 07:22
@DarkLight1337
[Model] Composite weight loading for multimodal Qwen2 (vllm-project#1…
bf0e382 
…0944)

Signed-off-by: DarkLight1337 <[email protected]>
@afeldman-nm
Merge branch 'main' into v1_logprobs
cef5ddb
@DarkLight1337
[Doc] Explicitly state that InternVL 2.5 is supported (vllm-project#1…
1c768fe 
…0978)

Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Model] Update multi-modal processor to support Mantis(LLaVA) model (v…
39e227c 
…llm-project#10711)

Signed-off-by: DarkLight1337 <[email protected]>
@DarkLight1337
[Doc] Explicitly state that PP isn't compatible with speculative deco…
c889d58 
…ding yet (vllm-project#10975)

Signed-off-by: DarkLight1337 <[email protected]>
@xffxff
[BugFix][Kernel]: fix illegal memory access in causal_conv1d when con…
78029b3 
…v_states is None (vllm-project#10928)

Signed-off-by: xffxff <[email protected]>
@afeldman-nm
Merge branch 'main' into v1_logprobs_merge
bed24db
@afeldman-nm
move
5ce8128
@youkaichao
[core][executor] simplify instance id (vllm-project#10976)
1b62745 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[core][misc] remove use_dummy driver for _run_workers (vllm-project#1…
7be15d9 
…0920)

Signed-off-by: youkaichao <[email protected]>
@youkaichao
[torch.compile] allow candidate compile sizes (vllm-project#10984)
fd57d2b 
Signed-off-by: youkaichao <[email protected]>
@ywang96
[V1] Initial support of multimodal models for V1 re-arch (vllm-projec…
a11f326 
…t#10699)

Signed-off-by: Roger Wang <[email protected]>
@youkaichao
[torch.compile][misc] fix comments (vllm-project#10993)
43b05fa 
Signed-off-by: youkaichao <[email protected]>
@youkaichao
[misc] clean up and unify logging (vllm-project#10999)
46004e8 
Signed-off-by: youkaichao <[email protected]>
@ywang96
[Doc][V1] Add V1 support column for multimodal models (vllm-project#1…
af7c4a9 
…0998)

Signed-off-by: Roger Wang <[email protected]>
@youkaichao
[torch.compile] add dynamo time tracking (vllm-project#11005)
d1c2e15 
Signed-off-by: youkaichao <[email protected]>
@ywang96
[V1] Fix Detokenizer loading in AsyncLLM (vllm-project#10997)
c690357 
Signed-off-by: Roger Wang <[email protected]>
@russellb
[Core] Require xgrammar >= 0.1.6 (vllm-project#11021)
e691b26 
Signed-off-by: Russell Bryant <[email protected]>
@wangxiyuan
[Platform] Move async output check to platform (vllm-project#10768)
aea2fc3 
Signed-off-by: wangxiyuan <[email protected]>
@varun-sundar-rabindranath
[V1] Input Batch Relocation (vllm-project#10962)
25b79d9 
Signed-off-by: Varun Sundar Rabindranath <[email protected]>
Co-authored-by: Varun Sundar Rabindranath <[email protected]>
@afeldman-nm
merge
14c7e56 
Signed-off-by: Andrew Feldman <[email protected]>
@afeldman-nm
removed VLLM_USE_V1 checks
bdd0abf 
Signed-off-by: Andrew Feldman <[email protected]>
@afeldman-nm
revert logprobs name changes
1fc981e 
Signed-off-by: Andrew Feldman <[email protected]>
@afeldman-nm
removing some unnecessary changes'
dc63ac1 
Signed-off-by: Andrew Feldman <[email protected]>
@afeldman-nm
removed fast checks
4f30408 
Signed-off-by: Andrew Feldman <[email protected]>
@afeldman-nm
wip test_completion
77488cb
@afeldman-nm
toward completion tests
f1a689c 
Signed-off-by: Andrew Feldman <[email protected]>
@afeldman-nm
serialization fix
e962aa7 
Signed-off-by: Andrew Feldman <[email protected]>
@robertgshaw2-redhat
merged
0f6790d
@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

  • Add ready label to the PR
  • Enable auto-merge.

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.