John Hewitt

structural-probes. Codebase for testing whether hidden states of neural networks encode discrete structures.

backpacks-flash-attn. The original Backpack Language Model implementation, a fork of FlashAttention

implicit-ins. Codebase for Instruction Following without Instruction Tuning

control-tasks. Repository describing example random control tasks for designing and interpreting neural probes

conditional-probing. Codebase for running (conditional) probing experiments

embed-init. Rough codebase for exploring initialization strategies for new word embeddings in pretrained LMs

dyckkm-learning. Codebase implementing LMs for learning the Dyck-(k,m) bounded hierarchical language

model-editing-canonical-examples. Python

truncation-sampling. Codebase describing experiments in Truncation Sampling as Language Model Desmoothing

dyckkm-constructions. Codebase implementing RNN constructions that generate Dyck-(k,m)

ts-mauve-experiments. Python

cs229-head-tracking. Code for analyzing mental health outcomes with VR head tracking data

transformers. 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

tinker-chess-example. Playing around with the Tinker toolkit on reinforcement learning for chess thinking traces