Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
Zihan Qiu, Zekun Wang, Bo Zheng, Zeyu Huang, Kaiyue Wen, Songlin Yang, Rui Men, Le Yu e mais 5
De quem é
Georgia Institute of Technologyafiliação declarada pelos autores, segundo o OpenAlex- Sichuan Universityafiliação declarada pelos autores, segundo o OpenAlex
Outros papers das mesmas organizações
- Evaluating large language models for accuracy incentivizes hallucinationsGeorgia Institute of Technology · 801 citações
- Foundation Models Defining a New Era in Vision: A Survey and OutlookGeorgia Institute of Technology · 733 citações
- Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM ReasoningAlibaba Group · 722 citações
- Small Language Models are the Future of Agentic AIGeorgia Institute of Technology · 628 citações
- Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-ServeGeorgia Institute of Technology · 537 citações
- VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic NavigationGeorgia Institute of Technology · 502 citações