Ofir Press

attention_with_linear_biases. Code for the ALiBi method for transformer language models (ICLR 2022)

self-ask. Code and data for "Measuring and Narrowing the Compositionality Gap in Language Models"

YouMayNotNeedAttention. Code for the Eager Translation Model from the paper You May Not Need Attention

shortformer. Code for the Shortformer model, from the ACL 2021 paper by Ofir Press, Noah A. Smith and Mike Lewis.

sandwich_transformer. This repository contains the code for running the character-level Sandwich Transformers from our ACL 2020 paper on Improving Transformer Models by Reordering their Sublayers.

UsingTheOutputEmbedding. Code for the EACL paper "Using the Output Embedding to Improve Language Models" by Ofir Press and Lior Wolf

0plot. Use 0plot to automatically build matplotlib plots using ChatGPT.

PartialShuffle. Python

tstl_t5_bias. This is our implementation of the T5 bias for fairseq.