DC: Small: One Thousand Points of Light: Accelerating Data-Intensive Applications By Proxy

A large class of distributed data-rich applications, including distributed data mining, distributed workflows, and Web 2.0 Mashups, are increasingly relying on cloud services to meet their data storage and computing demands. However, today, these applications are responsible for combining data and results from different specialized cloud services individually, which can lead to significant performance and reliability bottlenecks, due to the lack of appropriate resources connecting the applications to multiple clouds, resulting in a significant impediment to their successful deployment. This project proposes a cloud proxy network that allows optimized and reliable data-centric operations to be performed at strategic network locations. In this model, proxies may take on several data-centric roles: interacting with cloud services, routing data to each other, caching data for later use, and invoking compute-intensive data operators for intermediate processing. The proposed solution will enable an efficient coupling of cloud services to yield improved end-to-end performance and reliability for newly emerging data-intensive applications. This project will explore the potential of the proxy network architecture by evaluating its merits using a volunteer approach, focusing on four main research challenges: Proxy Performance, Proxy Reliability, Proxy Information Sparsity, and Proxy Selection.

The broader impact of this project is to amplify the effectiveness and productivity of diverse scientific, social, and engineering communities for enabling data-driven scientific inquiry in a performance-efficient and reliable manner. Proxy middleware will be released to the wider community towards this end. Educating a new generation of students in data-centric computing through major curriculum innovation is also proposed.