Stratified_Mixture_of_Experts. This is the repository for our EMNLP 2023 paper: Towards Being Parameter-Efficient: A Stratified Sparsely Activated Transformer with Dynamic Capacity.