Efficiently Modeling Long Sequences with Structured State Spaces
Paper-reading notes: S4
Efficient-Llms
Retentive Network: A Successor to Transformer for Large Language Models
Paper-reading notes: RetNet
Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
Paper-reading notes: Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
xLSTM: Extended Long Short-Term Memory
Paper-reading notes: xLSTM Extended Long Short-Term Memory
RWKV: Reinventing RNNs for the Transformer Era
Paper-reading notes: RWKV: Reinventing RNNs for the Transformer Era