Rare find

long-context-attention. USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

github.com/feifeibear/long-context-attention
by @feifeibear

Problem

Audience

Keep exploring

Updates

No founder updates yet. Check back soon.