CIF: Small: Exploiting Sparsity for Dimensionality Reduction

Existing approaches to exploit sparsity present in statistical or deterministic signal descriptors have mostly relied on linear models, and available compressive sampling (CS) implementations are analog. On the other hand, major application domains, such as signal compression and reduced-rank approximation of complex systems, entail bilinear models. In addition, the pertinent technology used in everyday life, for e.g., speech and image compression, is digital. Accordingly, the need arises for fundamental research to account for sparsity in bilinear models, as well as put CS on equal footing with digital compression modules, which rely on the `workhorses' for dimensionality reduction, namely principal component analysis (PCA) or canonical correlation analysis (CCA), and have their performance benchmarked by rate-distortion limits.

This research aims at developing algorithms, and corresponding performance limits for sparsity-cognizant dimensionality reduction, compression, and reconstruction tasks. Key to achieving these goals are optimal formulations for sparse PCA, sparse CCA, and associated quantization schemes, along with sparsity-aware rate-distortion metrics for comparison with sparse overcomplete basis expansions (SOBE). The vision is to have available tools and figures of merit to exploit the `right' form of sparsity for the `right' application domain. Optimization of sparse PCA, sparse CCA, and SOBE approaches draws from contemporary advances in sparsity-aware regression, basis pursuit, subspace tracking, and coordinate-descent solvers of minimization problems regularized with the l_1 norm of the unknowns. The research agenda leverages these tools to investigate a number of challenging directions. These include: (d1) sparsity-exploiting and power-aware compression over non-ideal links; (d2) sparsity-cognizant, adaptive compression for (non-)stationary processes with memory; (d3) sparsity-aware reconstruction of hidden sources, and reduced-rank identification of sparse systems; and (d4) comparisons between deterministic and statistical descriptors of sparsity.