Synthesizer: Rethinking Self-Attention for Transformer Models
Paper-reading notes: Synthesizer
Attention
Learning Transformer Programs
Paper-reading notes: Learning Transformer Programs
Reformer: The Efficient Transformer
Paper-reading notes: Reformer
ALTA: Compiler-Based Analysis of Transformers
Paper-reading notes: ALTA
Tracr: Compiled Transformers as a Laboratory for Interpretability
Paper-reading notes: Tracr
Thinking Like Transformers
Paper-reading notes: RASP
FNet: Mixing Tokens with Fourier Transforms
Paper-reading notes: FNet
Linformer: Self-Attention with Linear Complexity
Paper-reading notes: Linformer
Rethinking Attention with Performers
Paper-reading notes: Performers
What Formal Languages Can Transformers Express? A Survey
Paper-reading notes: What Formal Languages Can Transformers Express? A Survey