Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Yuxing Wei and 7 more
Who it is from
- University of Washingtonaffiliation declared by the authors, according to OpenAlex
Other papers from the same organizations
- s1: Simple test-time scalingStanford University · 1,957 citations
- Instruction Tuning for Large Language Models: A SurveyAlibaba Group · 1,416 citations
- REPLUG: Retrieval-Augmented Black-Box Language ModelsKorea Advanced Institute of Science and Technology · 1,206 citations
- Splitwise: Efficient Generative LLM Inference Using Phase SplittingMicrosoft · 1,096 citations
- Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A ReviewUniversity of Washington · 949 citations
- BLINK: Multimodal Large Language Models Can See but Not PerceiveColumbia University · 848 citations