Abstract
The convergence of an error-feedback algorithm is studied for decentralized stochastic gradient descent (DSGD) algorithm with compressed information sharing over time-varying graphs. It is shown that for both strongly-convex and convex cost functions, despite of imperfect information sharing, the convergence rates match those with perfect information sharing. To do so, we show that for strongly-convex loss functions, with a proper choice of a step-size, the state of each node converges to the global optimizer at the rate of mathcal{O}left( {T{-1}} right). Similarly, for general convex cost functions, with a proper choice of step-size, we show that the value of loss function at a temporal average of each node's estimates converges to the optimal value at the rate of mathcal{O}left( {T{-1/2 + varepsilon }} right) for any ϵ > 0.
| Original language | English (US) |
|---|---|
| Title of host publication | 2022 American Control Conference, ACC 2022 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| Pages | 2791-2796 |
| Number of pages | 6 |
| ISBN (Electronic) | 9781665451963 |
| DOIs | |
| State | Published - 2022 |
| Event | 2022 American Control Conference, ACC 2022 - Atlanta, United States Duration: Jun 8 2022 → Jun 10 2022 |
Publication series
| Name | Proceedings of the American Control Conference |
|---|---|
| Volume | 2022-June |
| ISSN (Print) | 0743-1619 |
Conference
| Conference | 2022 American Control Conference, ACC 2022 |
|---|---|
| Country/Territory | United States |
| City | Atlanta |
| Period | 6/8/22 → 6/10/22 |
Bibliographical note
Funding Information:H. Reisizadeh (email: hadir@umn.edu) and S. Mohajer (email: soheil@umn.edu) are with the University of Minnesota, and B. Touri (email: btouri@ucsd.edu) is with the University of California San Diego. The work of H. Reisizadeh and S. Mohajer is supported in part by the National Science Foundation under Grants CCF-1749981.
Publisher Copyright:
© 2022 American Automatic Control Council.