As used in, e.g. lasso regression.

Nonlinear least squares with ceres-solver:

Ceres Solve is an open source C++ library for modeling and solving large, complicated optimization problems. It can be used to solve Non-linear Least Squares problems with bounds constraints and general unconstrained optimization problems. It is a mature, feature rich, and performant library that has been used in production at Google since 2010.

Minimal python Iteratively reweighted least squares by A.E. Haynes

Ricardo Carvalho, Adaptive Lasso: What it is and how to implement in R

## Refs

- BeLT17
- Bellec, P. C., Lecué, G., & Tsybakov, A. B.(2017) Towards the study of least squares estimators with convex penalty.
*arXiv:1701.09120 [Math, Stat]*. - ChYi08
- Chartrand, R., & Yin, W. (2008) Iteratively reweighted algorithms for compressive sensing. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008 (pp. 3869–3872). DOI.
- ChSh16
- Chatla, S. B., & Shmueli, G. (2016) Modeling Big Count Data: An IRLS Framework for CMP Regression and GAM.
*arXiv:1610.08244 [Stat]*. - CGWY12
- Chen, X., Ge, D., Wang, Z., & Ye, Y. (2012) Complexity of unconstrained L_2-L_p.
*Mathematical Programming*, 143(1–2), 371–383. DOI. - Frie02
- Friedman, J. H.(2002) Stochastic gradient boosting.
*Computational Statistics & Data Analysis*, 38(4), 367–378. DOI. - FHHT07
- Friedman, J., Hastie, T., Höfling, H., & Tibshirani, R. (2007) Pathwise coordinate optimization.
*The Annals of Applied Statistics*, 1(2), 302–332. DOI. - FrHT10
- Friedman, J., Hastie, T., & Tibshirani, R. (2010) Regularization Paths for Generalized Linear Models via Coordinate Descent.
*Journal of Statistical Software*, 33(1), 1–22. DOI. - GaRC09
- Gasso, G., Rakotomamonjy, A., & Canu, S. (2009) Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming.
*IEEE Transactions on Signal Processing*, 57(12), 4686–4698. DOI. - KaLa10
- Karampatziakis, N., & Langford, J. (2010) Online Importance Weight Aware Updates.
*arXiv:1011.1576 [Cs]*. - MaNT04
- Madsen, K., Nielsen, H. B., & Tingleff, O. (2004) Methods for non-linear least squares problems.
- PoKo97
- Portnoy, S., & Koenker, R. (1997) The Gaussian hare and the Laplacian tortoise: computability of squared-error versus absolute-error estimators.
*Statistical Science*, 12(4), 279–300. DOI. - RhGl15
- Rhee, C.-H., & Glynn, P. W.(2015) Unbiased Estimation with Square Root Convergence for SDE Models.
*Operations Research*, 63(5), 1026–1043. DOI. - RoZh07
- Rosset, S., & Zhu, J. (2007) Piecewise linear regularized solution paths.
*The Annals of Statistics*, 35(3), 1012–1030. DOI. - YuTo09
- Yun, S., & Toh, K.-C. (2009) A coordinate gradient descent method for ℓ 1-regularized convex minimization.
*Computational Optimization and Applications*, 48(2), 273–307. DOI.