Kye Gomez

swarms. The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework. Website: https://swarms.ai

tree-of-thoughts. Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%

BitNet. Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

awesome-multi-agent-papers. A compilation of the best multi-agent papers

Open-AF3. Implementation of Alpha Fold 3 from the paper: "Accurate structure prediction of biomolecular interactions with AlphaFold3" in PyTorch

LongNet. Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

zeta. Build high-performance AI models with modular building blocks

RT-2. Democratization of RT-2 "RT-2: New model translates vision and language into action"

VisionMamba. Implementation of Vision Mamba from the paper: "Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model" It's 2.8x faster than DeiT and saves 86.8% GPU memory when performing batch inference to extract features on high-res images

MultiModalMamba. A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.

Gemini. The open source implementation of Gemini, the model that will "eclipse ChatGPT" by Google

Med-PaLM. Towards Generalist Biomedical AI

ScreenAI. Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"

Sophia. Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

CM3Leon. An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning", an all-new multi modal AI that uses just a decoder to generate both text and images

PALM-E. Implementation of "PaLM-E: An Embodied Multimodal Language Model"

NaViT. My implementation of "Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution"

RT-X. Pytorch implementation of the models RT-1-X and RT-2-X from the paper: "Open X-Embodiment: Robotic Learning Datasets and RT-X Models"

MambaTransformer. Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling

LFM. An open source implementation of LFMs from Liquid AI: Liquid Foundation Models

Jamba. PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"

Vit-RGTS. Open source implementation of "Vision Transformers Need Registers"

Python-Package-Template. A easy, reliable, fluid template for python packages complete with docs, testing suites, readme's, github workflows, linting and much much more

AttentionIsOFFByOne. Implementation of "Attention Is Off By One" by Evan Miller

Andromeda. An all-new Language Model That Processes Ultra-Long Sequences of 100,000+ Ultra-Fast

swarms-pytorch. Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊

PALI3. Implementation of PALI3 from the paper PALI-3 VISION LANGUAGE MODELS: SMALLER, FASTER, STRONGER"

the-compiler. Seed, Code, Harvest: Grow Your Own App with Tree of Thoughts!

SwitchTransformers. Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

MORPHEUS-1. Implementation of "MORPHEUS-1" from Prophetic AI and "The world’s first multi-modal generative ultrasonic transformer designed to induce and stabilize lucid dreams. "

MoE-Mamba. Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Zeta

MambaByte. Implementation of MambaByte in "MambaByte: Token-free Selective State Space Model" in Pytorch and Zeta

Mixture-of-Depths. Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"

xLSTM. Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"

Lets-Verify-Step-by-Step. "Improving Mathematical Reasoning with Process Supervision" by OPENAI

FlashAttention20. Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels

Algorithm-Of-Thoughts. My implementation of "Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models"

SparseAttention. Pytorch Implementation of the sparse attention from the paper: "Generating Long Sequences with Sparse Transformers"

PALI. Democratization of "PaLI: A Jointly-Scaled Multilingual Language-Image Model"

RoboCAT. Implementation of Deepmind's RoboCat: "Self-Improving Foundation Agent for Robotic Manipulation" An next generation robot LLM

Kosmos2.5. My implementation of Kosmos2.5 from the paper: "KOSMOS-2.5: A Multimodal Literate Model"

phi-1. Plug in and play implementation of " Textbooks Are All You Need", ready for training, inference, and dataset generation

LiqudNet. Implementation of Liquid Nets in Pytorch

Kosmos-X. The Next Generation Multi-Modality Superintelligence

Gamba. Implementation of PyTorch: "GAMBA: MARRY GAUSSIAN SPLATTING WITH MAMBA FOR SINGLE-VIEW 3D RECONSTRUCTION"

HLT. Implementation of the transformer from the paper: "Real-World Humanoid Locomotion with Reinforcement Learning"

StarlightVision. A multi-modal AI Model that can generate high quality novel videos with text, images, or video clips.

movie-gen. An open source community implementation of the model from the paper: "Movie Gen: A Cast of Media Foundation Models". Join our community to help implement this model!

Infini-attention. Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTORCH

Griffin. Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"

Finetuning-Suite. Finetune any model on HF in less than 30 seconds

Sora. Implementation of the premier Text to Video model from OpenAI

LUMIERE. Implementation of the text to video model LUMIERE from the paper: "A Space-Time Diffusion Model for Video Generation" by Google Research

qformer. Implementation of Qformer from BLIP2 in Zeta Lego blocks.

Blockwise-Parallel-Transformer. 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.

AutoRT. Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents"

LM-Infinite. Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"

AudioFlamingo. Implementation of the model "AudioFlamingo" from the paper: "Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities"