New York

Shida Wang

Advanced
@radarFudan

Machine Learning and Math

radarfudan.github.io

Awesome-state-space-models. Collection of papers on state-space models

620

mamba-minimal-jax. Python

36

Curse-of-memory. Curse-of-memory phenomenon of RNNs in sequence modelling

19

mamba. Python

18

snippets. Jupyter Notebook

4

S6. Figure out what's next for S6

3

profiling-cuda-in-torch. Python

3

flash-fft-conv. FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

2

safari. Convolutions for Sequence Modeling

2

new_repo. Shell

2

benchmark_sequence_modeling. Python

2

annotated-mamba. Annotated version of the Mamba paper

2

EasyLM. Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax.

1

triton. Development repository for the Triton language and compiler

1

SSM_examples. Jupyter Notebook

1

attention_with_linear_biases. Code for the ALiBi method for transformer language models (ICLR 2022)

1

causal-conv1d. Causal depthwise conv1d in CUDA, with a PyTorch interface

1

radarFudan.

1

TinyLlama. The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

1

fairseq. Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

1

pythia. The hub for EleutherAI's work on interpretability and learning dynamics

1

EffHDC. Python

1

LHJ. Can we use lightning as data loader and jax as the models?

1

radarFudan.github.io. HTML

1

flash-attention. Fast and memory-efficient exact attention

1

awd-lstm-lm. LSTM and QRNN Language Model Toolkit for PyTorch

1

LM-RMT_bala. Recurrent Memory Transformer

1

lightning-hydra-template. PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡

1

awesome-neural-ode. A collection of resources regarding the interplay between differential equations, deep learning, dynamical systems, control and numerical methods.

1

Lp_solver. project for Linear programming

1

t5-pegasus-pytorch. Python

1

RWKV-CUDA. The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )

1

seq2seq-data-generator. Jupyter Notebook

1

google-research. Google Research

1

TCN. Sequence modeling benchmarks and temporal convolutional networks

1

18.408.

1

dockers. Shell

1

cheatsheets. Official Matplotlib cheat sheets

1

transformers. 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

1

sweep_trial. Python

1

INTEREST. Temporal re-weighting improve the long-term memory learning.

1
41
Apply