TY - JOUR
T1 - Strong oracle optimality of folded concave penalized estimation
AU - Fan, Jianqing
AU - Xue, Lingzhou
AU - Zou, Hui
PY - 2014/6
Y1 - 2014/6
N2 - Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamental issue still remains that it is not clear whether the local optimum computed by a given optimization algorithm possesses those nice theoretical properties. To close this important theoretical gap in over a decade, we provide a unified theory to show explicitly how to obtain the oracle solution via the local linear approximation algorithm. For a folded concave penalized estimation problem, we show that as long as the problem is localizable and the oracle estimator is well behaved, we can obtain the oracle estimator by using the one-step local linear approximation. In addition, once the oracle estimator is obtained, the local linear approximation algorithm converges, namely it produces the same estimator in the next iteration. The general theory is demonstrated by using four classical sparse estimation problems, that is, sparse linear regression, sparse logistic regression, sparse precision matrix estimation and sparse quantile regression.
AB - Folded concave penalization methods have been shown to enjoy the strong oracle property for high-dimensional sparse estimation. However, a folded concave penalization problem usually has multiple local solutions and the oracle property is established only for one of the unknown local solutions. A challenging fundamental issue still remains that it is not clear whether the local optimum computed by a given optimization algorithm possesses those nice theoretical properties. To close this important theoretical gap in over a decade, we provide a unified theory to show explicitly how to obtain the oracle solution via the local linear approximation algorithm. For a folded concave penalized estimation problem, we show that as long as the problem is localizable and the oracle estimator is well behaved, we can obtain the oracle estimator by using the one-step local linear approximation. In addition, once the oracle estimator is obtained, the local linear approximation algorithm converges, namely it produces the same estimator in the next iteration. The general theory is demonstrated by using four classical sparse estimation problems, that is, sparse linear regression, sparse logistic regression, sparse precision matrix estimation and sparse quantile regression.
KW - Folded concave penalty
KW - Local linear approximation
KW - Nonconvex optimization
KW - Oracle estimator
KW - Sparse estimation
KW - Strong oracle property
UR - http://www.scopus.com/inward/record.url?scp=84902449177&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84902449177&partnerID=8YFLogxK
U2 - 10.1214/13-AOS1198
DO - 10.1214/13-AOS1198
M3 - Article
AN - SCOPUS:84902449177
SN - 0090-5364
VL - 42
SP - 819
EP - 849
JO - Annals of Statistics
JF - Annals of Statistics
IS - 3
ER -