The Living Thing / Notebooks : Distributed optimization for regression

See practical cloud, message passing cloud.

Implementations

Spark.

CoCOA.

Refs

BFLL15
Battey, H., Fan, J., Liu, H., Lu, J., & Zhu, Z. (2015) Distributed Estimation and Inference with Statistical Guarantees. arXiv:1509.05457 [math, Stat].
BoCN16
Bottou, L., Curtis, F. E., & Nocedal, J. (2016) Optimization Methods for Large-Scale Machine Learning. arXiv:1606.04838 [cs, Math, Stat].
BPCP11
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Found. Trends Mach. Learn., 3(1), 1–122. DOI.
JSTT14
Jaggi, M., Smith, V., Takac, M., Terhorst, J., Krishnan, S., Hofmann, T., & Jordan, M. I.(2014) Communication-Efficient Distributed Dual Coordinate Ascent. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, & K. Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 27 (pp. 3068–3076). Curran Associates, Inc.
MKJS15
Ma, C., Konečnỳ, J., Jaggi, M., Smith, V., Jordan, M. I., Richtárik, P., & Takáč, M. (2015) Distributed Optimization with Arbitrary Local Solvers. arXiv Preprint arXiv:1512.04039.
MSJJ15
Ma, C., Smith, V., Jaggi, M., Jordan, M. I., Richtárik, P., & Takáč, M. (2015) Adding vs Averaging in Distributed Primal-Dual Optimization. arXiv:1502.03508 [cs].
MaBG10
Mateos, G., Bazerque, J. A., & Giannakis, G. B.(2010) Distributed Sparse Linear Regression. IEEE Transactions on Signal Processing, 58(10), 5262–5276. DOI.
ShZh13
Shalev-Shwartz, S., & Zhang, T. (2013) Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research, 14(Feb), 567–599.
ShSZ14
Shamir, O., Srebro, N., & Zhang, T. (2014) Communication-Efficient Distributed Optimization using an Approximate Newton-type Method. In ICML (Vol. 32, pp. 1000–1008).
SFJJ15
Smith, V., Forte, S., Jordan, M. I., & Jaggi, M. (2015) L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework. arXiv:1512.04011 [cs].
Yang13
Yang, T. (2013) Trading computation for communication: Distributed stochastic dual coordinate ascent. In Advances in Neural Information Processing Systems (pp. 629–637).