This National Science Foundation (NSF) project
investigates optimizing network transfers using introspective
modeling of end-systems. The
bottleneck for the transferring data at very high speeds often turns
out to be the end-system performance. In the absence of definitive
knowledge about the workload at the receiving end-system, the
sender's transmission rate often overshoots the critical bottleneck
rate of the receiver. This typically results in oscillations between
the extremes and poor performance. To optimize the performance of
the transport protocols and achieve the important flow control
functionality, it is important to estimate the receiving end-system
effective bottleneck rate. In this project we will use modeling and
active analysis of the end-system to estimate this rate.
We will develop queueing
network models for representing the different multi-core and
multiprocessor end-systems running different types of workloads. We
will develop a software tool to be integrated with existing
transport protocols. We will carry out experimental analysis for
different types of end-systems with different configurations and
workloads. We will apply and extend methods that have been proposed
to address the limitations of queueing network models for
performance analysis of computer systems with bursty workloads and
correlated service times. The
software tool will be made available to the research community to
analyze and optimize distributed applications and systems. The
research project will provide a framework to train graduate and
undergraduate students in both analytical and experimental methods,
and develop knowledge and intuition about next generation computer
systems and distributed applications.
Project Title and Duration: [0917315/08-004661]
Estimating the End-system Network I/O Bottleneck Rate to Optimize
Transport Layer Performance. September 2009 - August 2012. Funded
under the American Recovery and Reinvestment Act of 2009 (Public Law
- Dipak Ghosal,Optimizing Transport of Big Data over Dedicated Networks," Invited Talk, (NDM 2012 @ SC'12) The 2nd International Workshop on Network-aware Data Management, November 11th, 2012, Salt Lake City, Utah.
Ahuja, Matthew Farrens, Dipak Ghosal, “Cache-aware affinitization on commodity
multicores for high-speed network flows,” ANCS '12: Proceedings of the eighth
ACM/IEEE symposium on Architectures for networking and communications systems, October
- Vishal Ahuja, Dipak
Ghosal, Matthew Farrens, Minimizing the Data Transfer Time Using Multicore
End-System Aware Flow Bifurcation,” CCGRID '12: Proceedings of the 2012 12th
IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid
2012), May 2102.
- V. Ahuja, A. Banerjee, M. Farrens, G. Serazzi, D. Ghosal, "Introspective End-system Modeling to Optimize the
Transfer Time of Rate Based Protocols", In Proceedings of the 20th International ACM Symposium on High Performance Parallel and Distributed Computing, San Jose, CA, June 8-11, 2011.
- V. Ahuja, R. Archibald, A. Banerjee, M. Farrens, and D.
Ghosal, "Estimating the Network I/O BottleneActive
End-System Analysis to Estimate the Network I/O
Bottleneck Rate," Workshop on The Influence
of I/O on Microprocessor Architecture (IOM-2009)
Raleigh, North Carolina, February 15, 2009
- A. Banerjee, D.
Ghosal, B. Mukherjee, and W. Feng, Algorithms for
Integrated Routing and Scheduling for Aggregating Data
from Distributed Resources on a Lambda Grid" IEEE
Transaction on Parallel and Distributed Systems, 2008,
vol. 19, Issue 1, pp. 24-34.
- A. Banerjee, D.
Ghosal, and B. Mukherjee,
Modeling and Analysis to Estimate the End-System
Performance Bottleneck Rate for High-Speed Data Transfer,
Fifth International Workshop on Protocols for Fast
Long-Distance Networks (PFLDNet) 2007, Los Angeles.
Banerjee, W. Feng, B. Mukherjee, and D. Ghosal, RAPID:
An End-System Aware Protocol for Intelligent
Data-Transfer over Lambda-Grids, in the Proceedings of
the IEEE/ACM International Parallel and Distributed
Processing Symposium (IPDPS 2006), Rhode Island, Greece,
- A. Banerjee, W. Feng,
B. Mukherjee, and D. Ghosal, "End-system Performance
Aware Transport over Optical Circuit-Switched
Connections," IEEE INFOCOM High-Speed Networking
Workshop: The Terabits Challenge, April 2006.
Rao, Q. Wu, S. Carter, W. Wing, A. Banerjee, D. Ghosal,
and B. Mukherjee, "Control Plane for Advance Bandwidth
Scheduling in Ultra High-Speed Networks," IEEE INFOCOM
High-Speed Networking Workshop: The Terabits Challenge,
- A. Banerjee,
W.-C. Feng, B. Mukherjee, and D. Ghosal, Routing and
Scheduling Large File Transfers over Lambda Grids, Third
International Workshop on Protocols for Fast
Long-Distance Networks (PFLDNet) 2005, February 3,4
To request reprints and/or information about this project, please contact