References

Alghunaim, Sulaiman A., Ernest K. Ryu, Kun Yuan, and Ali H. Sayed. 2019. “Decentralized Proximal Gradient Algorithms with Linear Convergence Rates.”

Assran, Mahmoud, Nicolas Loizou, Nicolas Ballas, and Michael Rabbat. 2018. “Stochastic Gradient Push for Distributed Deep Learning.” arXiv Preprint arXiv:1811.10792.

Koloskova, Anastasia, Sebastian U Stich, and Martin Jaggi. 2019. “Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication.” arXiv Preprint arXiv:1902.00340.

Li, Zhi, Wei Shi, and Ming Yan. 2019. “A Decentralized Proximal-Gradient Method with Network Independent Step-Sizes and Separated Convergence Rates.” IEEE Transactions on Signal Processing 67 (17). IEEE: 4494–4506.

Lian, Xiangru, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. 2017. “Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent.” In Advances in Neural Information Processing Systems, 5330–40.

Nedić, Angelia, Alex Olshevsky, and Michael G Rabbat. 2018. “Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization.” Proceedings of the IEEE 106 (5). IEEE: 953–76.

Pu, Shi, and Angelia Nedić. 2018. “Distributed Stochastic Gradient Tracking Methods.”

Pu, Shi, Alex Olshevsky, and Ioannis Ch. Paschalidis. 2019a. “A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent.”

———. 2019b. “Asymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning.”

Pu, Shi, Wei Shi, Jinming Xu, and Angelia Nedić. 2018. “A Push-Pull Gradient Method for Distributed Optimization in Networks.” In 2018 Ieee Conference on Decision and Control (Cdc), 3385–90. IEEE.

Shi, Wei, Qing Ling, Gang Wu, and Wotao Yin. 2015. “Extra: An Exact First-Order Algorithm for Decentralized Consensus Optimization.” SIAM Journal on Optimization 25 (2). SIAM: 944–66.

Stich, Sebastian U, Jean-Baptiste Cordonnier, and Martin Jaggi. 2018. “Sparsified Sgd with Memory.” In Advances in Neural Information Processing Systems, 4447–58.

Tang, Hanlin, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu. 2018. “D \(\^{} 2\): Decentralized Training over Decentralized Data.” arXiv Preprint arXiv:1803.07068.

Yuan, Kun, Sulaiman A Alghunaim, Bicheng Ying, and Ali H Sayed. 2019. “On the Performance of Exact Diffusion over Adaptive Networks.” In 2019 Ieee 58th Conference on Decision and Control (Cdc), 4898–4903. IEEE.

Yuan, Kun, Bicheng Ying, Xiaochuan Zhao, and Ali H Sayed. 2018. “Exact Diffusion for Distributed Optimization and Learning—Part Ii: Convergence Analysis.” IEEE Transactions on Signal Processing 67 (3). IEEE: 724–39.