TY - GEN
T1 - Accelerating distributed workflows with edge resources
AU - Ramakrishnan, Siddharth
AU - Reutiman, Robert
AU - Chandra, Abhishek
AU - Weissman, Jon B
PY - 2013
Y1 - 2013
N2 - Distributed data-intensive workflow applications are increasingly relying on and integrating remote resources including community data sources, services, and computational platforms. Increasingly, these are made available as data, SAAS, and IAAS clouds. The execution of distributed data-intensive workflow applications can exposé network bottlenecks between clouds that compromise performance. In this paper, we focus on alleviating network bottlenecks by using a proxy network. In particular, we show how proxies can eliminate network bottlenecks by smart routing and perform in-network computations to boost workflow application performance. A novel aspect of our work is the inclusion of multiple proxies to accelerate different workflow stages optimizing different performance metrics. We show that the approach is effective for workflow applications and broadly applicable. Using Montage1 as an exemplar workflow application, results obtained through experiments on Planet Lab showed how different proxies acting in a variety of roles can accelerate distinct stages of Montage. Our microbenchmarksalso show that routing data through select proxies can accelerate network transfer for TCP/UDP bandwidth, delay, and jitter, in general.
AB - Distributed data-intensive workflow applications are increasingly relying on and integrating remote resources including community data sources, services, and computational platforms. Increasingly, these are made available as data, SAAS, and IAAS clouds. The execution of distributed data-intensive workflow applications can exposé network bottlenecks between clouds that compromise performance. In this paper, we focus on alleviating network bottlenecks by using a proxy network. In particular, we show how proxies can eliminate network bottlenecks by smart routing and perform in-network computations to boost workflow application performance. A novel aspect of our work is the inclusion of multiple proxies to accelerate different workflow stages optimizing different performance metrics. We show that the approach is effective for workflow applications and broadly applicable. Using Montage1 as an exemplar workflow application, results obtained through experiments on Planet Lab showed how different proxies acting in a variety of roles can accelerate distinct stages of Montage. Our microbenchmarksalso show that routing data through select proxies can accelerate network transfer for TCP/UDP bandwidth, delay, and jitter, in general.
KW - Distributed computing
KW - data-intensive computing
KW - network systems
KW - workflows
UR - http://www.scopus.com/inward/record.url?scp=84899766654&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84899766654&partnerID=8YFLogxK
U2 - 10.1109/IPDPSW.2013.240
DO - 10.1109/IPDPSW.2013.240
M3 - Conference contribution
AN - SCOPUS:84899766654
SN - 9780769549798
T3 - Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium Workshops and PhD Forum, IPDPSW 2013
SP - 2129
EP - 2138
BT - Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium Workshops and PhD Forum, IPDPSW 2013
PB - IEEE Computer Society
T2 - 2013 IEEE 37th Annual Computer Software and Applications Conference, COMPSAC 2013
Y2 - 22 July 2013 through 26 July 2013
ER -