π Series (π₀, π₀.₅)

Physical Intelligence is a fast-rising company focused on bringing general-purpose AI into the physical world. In under two years since introducing their first VLA prototype model π₀ , thet’ve made a huge impact in the embodied intelligence community. In this post, I’ll walk through the three main VLA models they’ve released so far, based on my reading of their blogs and papers. π₀ π₀ is a vision-language-action (VLA) model built on top of a pre-trained vision–language model (VLM) backbone. It is then robot-pretrained on a large mixture of open-source and in-house manipulation datasets to learn broad, general skills, and can be further post-trained on smaller, task-specific data to specialize for downstream applications. ...

March 1, 2026 | 2621 words | Author: Tan Ke

Large Concept Models: Language Modeling in a Sentence Representation Space

Paper-reading notes: Large Concept Models: Language Modeling in a Sentence Representation Space
January 15, 2026 | 3217 words | Author: Tan Ke

Synthesizer: Rethinking Self-Attention for Transformer Models

Paper-reading notes: Synthesizer
December 16, 2025 | 244 words | Author: Tan Ke

Learning Transformer Programs

Paper-reading notes: Learning Transformer Programs
December 15, 2025 | 339 words | Author: Tan Ke

Reformer: The Efficient Transformer

Paper-reading notes: Reformer
December 14, 2025 | 287 words | Author: Tan Ke

ALTA: Compiler-Based Analysis of Transformers

Paper-reading notes: ALTA
December 9, 2025 | 720 words | Author: Tan Ke

Tracr: Compiled Transformers as a Laboratory for Interpretability

Paper-reading notes: Tracr
December 8, 2025 | 59 words | Author: Tan Ke

Thinking Like Transformers

Paper-reading notes: RASP
December 7, 2025 | 273 words | Author: Tan Ke

FNet: Mixing Tokens with Fourier Transforms

Paper-reading notes: FNet
December 5, 2025 | 470 words | Author: Tan Ke

Linformer: Self-Attention with Linear Complexity

Paper-reading notes: Linformer
December 4, 2025 | 236 words | Author: Tan Ke