Introduction to Neural Networks and Seq2Seq Learning

During the summer semester of 2026, I took Introduction to Neural Networks and Sequence-to-Sequence Learning course. As you see, the main content is from perceptrons to modern sequence models. I started learning machine learning through Li Mu’s courses. I spent several weeks systemically studying his materials and organized the corresponding code in the repo. Also, I had already used neural networks in multiple projects before taking this course. I had trained language models, vision models, vision-language models, reinforcement-learning agents and even vision-language-action models… ...

July 15, 2026 | 3419 words | Author: Tan Ke

π Series (π₀, π₀.₅)

Physical Intelligence is a fast-rising company focused on bringing general-purpose AI into the physical world. In under two years since introducing their first VLA prototype model π₀, thet’ve made a huge impact in the embodied intelligence community. In this post, I’ll walk through the three main VLA models they’ve released so far, based on my reading of their blogs and papers. π₀ π₀ is a vision-language-action (VLA) model built on top of a pre-trained vision–language model (VLM) backbone. It is then robot-pretrained on a large mixture of open-source and in-house manipulation datasets to learn broad, general skills, and can be further post-trained on smaller, task-specific data to specialize for downstream applications. ...

March 1, 2026 | 2611 words | Author: Tan Ke

Large Concept Models

Paper Reading Notes: “Large Concept Models: Language Modeling in a Sentence Representation Space”
January 15, 2026 | 3217 words | Author: Tan Ke

Synthesizer

Paper Reading Notes: “Synthesizer: Rethinking Self-Attention for Transformer Models”
December 16, 2025 | 244 words | Author: Tan Ke

Learning Transformer Programs

Paper Reading Notes: “Learning Transformer Programs”
December 15, 2025 | 339 words | Author: Tan Ke

Reformer

Paper Reading Notes: “Reformer: The Efficient Transformer”
December 14, 2025 | 287 words | Author: Tan Ke

ALTA: Compiler-Based Analysis of Transformers

Paper-reading notes: ALTA
December 9, 2025 | 720 words | Author: Tan Ke

Tracr: Compiled Transformers as a Laboratory for Interpretability

Paper-reading notes: Tracr
December 8, 2025 | 59 words | Author: Tan Ke

Thinking Like Transformers

Paper-reading notes: RASP
December 7, 2025 | 273 words | Author: Tan Ke

FNet: Mixing Tokens with Fourier Transforms

Paper-reading notes: FNet
December 5, 2025 | 470 words | Author: Tan Ke