Technological Advances and Problems of High Performance Communications
An ecosystem of solutions along a stack of technology layers
Cohesively collecting state-of-the-art contributions from leading researchers in industry, national laboratories, and academia, Attaining High Performance Communications: A Vertical Approach discusses various issues pertaining to high performance communications in a particular layer of a vertical stack. It explores efficient interconnection hardware, the architectural aspects of network adapters and their integration with processor cores, the design of scalable and robust high performance end-to-end communications services and protocols, and system services and tools for new multi-core environments.
No single solution applied at one particular layer can help applications solve all performance-related issues with communication services. Instead, this book shows that a coordinated effort is needed among the layers. It covers many different types of technologies and layers across the stack, from the architectural features of the hardware, through the protocols and their implementation in operating system kernels, to the manner in which application services and middleware are using underlying platforms. The book also describes key developments in high-end platforms, high performance interconnection fabrics and communication libraries, and multi- and many-core systems.
This volume addresses the challenges involved in emerging types of communications applications, platforms, and services. Examining each layer in the vertical stack, it illustrates how to eliminate bottlenecks and provide optimization opportunities.
Ada Gavrilovska is a Research Scientist in the College of Computing and the Center for Experimental Research in Computer Systems (CERCS) at Georgia Tech.
High Performance Interconnects for Massively Parallel Systems Introduction Performance Network Topology Network Features Future Directions Summary Commodity High Performance Interconnects Introduction Overview of Past Commodity Interconnects, Features, and Trends InfiniBand Architecture Existing InfiniBand Adapters and Switches Existing InfiniBand Software Stacks Designing High-End Systems with InfiniBand: Case Studies Current and Future Trends of InfiniBand Ethernet vs. EtherNOT Overview Introduction Background Ethernet vs. EtherNOT? Commercial Perspective Concluding Remarks System Impact of Integrated Interconnects Introduction Technology Trends Integrated Interconnects Case Study: Implementation of Global Address Spaces Future Trends and Expectations Network Interfaces for High Performance Computing Introduction Network Interface Design Issues Current Approaches to Network Interface Design Issues Research Directions Summary Network Programming Interfaces for High Performance Computing Introduction The Evolution of High Performance Computing (HPC) Network Programming Interfaces Low-Level Network Programming Interfaces Distinguishing Characteristics Supporting Message Passing Interface (MPI) Supporting Shared Memory (SHMEM) and Partitioned Global Address Space (PGAS) Portals 4.0 High Performance IP-Based Transports Introduction Transmission Control Protocol (TCP) TCP Performance Tuning User Datagram Protocol (UDP)-Based Transport Protocols Stream Control Transmission Protocol (SCTP) Summary Remote Direct Memory Access and iWARP Introduction Remote Direct Memory Access (RDMA) iWARP Summary Accelerating Communication Services on Multi-Core Platforms Introduction The "Simple" Onload Approach Partitioned Communication Stacks Specialized Network Multi-Cores Toward Heterogeneous Multi-Cores Summary Virtualized I/O Introduction Split Device Driver Model Direct Device Access Model Opportunities and Tradeoffs Summary The Message Passing Interface Introduction MPI's Layer in the Network Stack Threading and MPI Point-to-Point Communications Collective Operations Implementation Strategies Summary High Performance Event Communication Introduction Design Points The EVPath Architecture Performance Microbenchmarks Usage Scenarios Summary The Case of the Fast Financial Feed Introduction Market Data Processing Systems Performance Requirements The OPRA Case Study Summary Data-Movement Approaches for High Performance Computing Storage Systems Introduction Lustre Panasas Parallel Virtual File System 2 (PVFS2) Lightweight File Systems Other Microsoft Project Plan (MPP) File Systems Summary Network Simulation Introduction Discrete Event Simulation Maintaining the Event List Modeling Routers, Links, and End-Systems Modeling Network Packets Modeling the Network Applications Visualizing the Simulation Distributed Simulation Summary References