The Living Thing / Notebooks :

Overparameterization

a.k.a. improper learning

Learning with more parameters than you need. Can compressive sensing be put into this framework? Convex relaxations often hinge upon this.

The combination of overparameterization and SGD is argued to be the secret to how deep learning works, by Zeyuan Allen-Zhu, Yuanzhi Li and Zhao Song.

TBD.

Refs