Sign in
Rare find
MInference.
[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
github.com/microsoft/MInference
by
@microsoft
Problem
Audience
Keep exploring
Updates
Founder
0
Product
0
No founder updates yet. Check back soon.
๐งช
staging