Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, Ramachandran Ramjee
Who it is from
Georgia Institute of Technologyaffiliation declared by the authors, according to OpenAlex
Other papers from the same organizations
- Evaluating large language models for accuracy incentivizes hallucinationsGeorgia Institute of Technology · 801 citations
- Foundation Models Defining a New Era in Vision: A Survey and OutlookGeorgia Institute of Technology · 733 citations
- Small Language Models are the Future of Agentic AIGeorgia Institute of Technology · 628 citations
- VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic NavigationGeorgia Institute of Technology · 502 citations
- Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-FreeGeorgia Institute of Technology · 436 citations
- SWE-smith: Scaling Data for Software Engineering AgentsGeorgia Institute of Technology · 435 citations