SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification
Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu and 7 more
Who it is from
- Carnegie Mellon Universityaffiliation declared by the authors, according to OpenAlex
- Shanghai Jiao Tong Universityaffiliation declared by the authors, according to OpenAlex
- University of California San Diegoaffiliation declared by the authors, according to OpenAlex
Other papers from the same organizations
- VM-UNet: Vision Mamba UNet for Medical Image SegmentationShanghai Jiao Tong University · 1,481 citations
- Explainability for Large Language Models: A SurveyBaidu · 1,443 citations
- Medical SAM adapter: Adapting segment anything model for medical image segmentationAgency for Science, Technology and Research · 1,361 citations
- A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPTBeihang University · 1,211 citations
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?Shanghai Artificial Intelligence Laboratory · 1,186 citations
- Multimodal Chain-of-Thought Reasoning in Language ModelsShanghai Jiao Tong University · 1,170 citations