Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, Ramachandran Ramjee
De quem é
Georgia Institute of Technologyafiliação declarada pelos autores, segundo o OpenAlex
Outros papers das mesmas organizações
- Evaluating large language models for accuracy incentivizes hallucinationsGeorgia Institute of Technology · 801 citações
- Foundation Models Defining a New Era in Vision: A Survey and OutlookGeorgia Institute of Technology · 733 citações
- Small Language Models are the Future of Agentic AIGeorgia Institute of Technology · 628 citações
- VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic NavigationGeorgia Institute of Technology · 502 citações
- Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-FreeGeorgia Institute of Technology · 436 citações
- SWE-smith: Scaling Data for Software Engineering AgentsGeorgia Institute of Technology · 435 citações