Optimization in Machine Learning
The summary of the seminar “Optimization in Machine Learning”, covering Bayesian Optimization, multi-fidelity methods, handling discrete search spaces, and the BANANAS method for NAS.
Optimization
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search
Paper-reading notes: BANANAS
SINDy Implementation Notes
Github repo: https://github.com/mrtanke/SINDy This blog is basically my hands-on notes while implementing SINDy (Sparse Identification of Nonlinear Dynamics) as a small, understandable pipeline: generate data → build a candidate library → solve a sparse regression problem → sanity-check the discovered equation → then push it into harder settings like autoencoder and video-like data. The whole notebook is organized into three parts: (1) SINDy on ground-truth coordinates, (2) SINDy-Autoencoder, and (3) a bonus on high-dimensional “video” inputs. ...
Multiobjective Tree-Structured Parzen Estimator
Paper-reading notes: MOTPE
Bayesian Optimization is Superior to Random Search for Machine Learning Hyperparameter Tuning
Paper-reading notes: Bayesian Optimization
Random Search for Hyper-Parameter Optimization
Paper-reading notes: Random Search for Hyper-Parameter Optimization
A Tutorial on Bayesian Optimization
Paper-reading notes: A Tutorial on Bayesian Optimization