Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 37.8k 5.7k

2. IBM/text-generation-inference IBM/text-generation-inference Public

   IBM development fork of https://github.com/huggingface/text-generation-inference

   Python 59 31

3. IBM/vllm IBM/vllm Public

   vLLM with support for IBM Spyre

   Python 10 15

4. foundation-model-stack/fms-guardrails-orchestrator foundation-model-stack/fms-guardrails-orchestrator Public

   🚀 Guardrails orchestration server for application of various detections on text generation input and output.

   Rust 6 23

5. IBM/mocktail IBM/mocktail Public

   HTTP & gRPC server mocking for Rust

   Rust 3 1

6. IBM/text-generation-router IBM/text-generation-router Public

   Routing proxy for TGIS

   Rust 2 5