-
|
The difference between langserve and TGI, vLLM, or why do I need langserve when I already have vLLM? |
Beta Was this translation helpful? Give feedback.
Answered by
Vinno97
Jan 2, 2024
Replies: 2 comments
-
|
LangServe hosts LangChain apps built on top of LLMs. vLLM hosts a raw LLM |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
eyurtsev
-
|
Don't use LangServe for serving raw LLMs -- it's not designed for that use case and will not manage the hardware resources appropriately. It's specifically for serving LangChain apps built on tops of LLMs. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
LangServe hosts LangChain apps built on top of LLMs. vLLM hosts a raw LLM