Abstract
The goal of many machine learning tasks is to learn a model that has small population risk. While mini-batch stochastic gradient descent (SGD) and variants are popular approaches for achieving this goal, it is hard to prescribe a clear stopping criterion and to establish high probability convergence bounds to the population risk. In this paper, we introduce Stable Gradient Descent which validates stochastic gradient computations by splitting data into training and validation sets and reuses samples using a differential private mechanism. StGD comes with a natural upper bound on the number of iterations and has high-probability convergence to the population risk. Experimental results illustrate that StGD is empirically competitive and often better than SGD and GD.
| Original language | English (US) |
|---|---|
| Title of host publication | 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018 |
| Editors | Ricardo Silva, Amir Globerson, Amir Globerson |
| Publisher | Association For Uncertainty in Artificial Intelligence (AUAI) |
| Pages | 766-775 |
| Number of pages | 10 |
| ISBN (Electronic) | 9781510871601 |
| State | Published - 2018 |
| Event | 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018 - Monterey, United States Duration: Aug 6 2018 → Aug 10 2018 |
Publication series
| Name | 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018 |
|---|---|
| Volume | 2 |
Other
| Other | 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018 |
|---|---|
| Country/Territory | United States |
| City | Monterey |
| Period | 8/6/18 → 8/10/18 |
Bibliographical note
Funding Information:ers for their valuable comments. The re search was supported by NSF grants IIS- 1563950, IIS-1447566, IIS-1447574, IIS-1422557, CCF-1451986, CNS-1314560, IIS-0953274, IIS-1029711, and NASA grant NNX12AQ39A.
Funding Information:
We thank the reviewers for their valuable comments. The research was supported by NSF grants IIS-1563950, IIS-1447566, IIS-1447574, IIS-1422557, CCF-1451986, CNS-1314560, IIS-0953274, IIS-1029711, and NASA grant NNX12AQ39A.
Publisher Copyright:
© 34th Conference on Uncertainty in Artificial Intelligence 2018. All rights reserved.