Table of Contents
PPT Slide
Why High Performance Computing?
Grand Challenges
Grand Challenges
Grand Challenges
Commercial Applications
Powerful Computers
Speed and Storage
Solution?
This Talk
This Talk
PPT Slide
Distributed Memory Machine Architecture
Centralized Shared Memory Multiprocessor
Centralized Shared Memory Multiprocessor
Distributed Shared Memory (Clusters of SMPs)
This Talk
What’s Next?
Beowulf
“The Dark Side of Clusters”
Non-Shared Address Space
Large and Expensive Clusters
This Talk
Programming Model Requirements
Programming Model Requirements
Parallel Framework
Message Passing Model
Shared Address Model Summary
Shared Address/Memory Multiprocessor Model
Advantages of shared-memory communication model
Advantages of message-passing communication model
Programming Models
OpenMP (1)
OpenMP (2) - C
OpenMP (3) - Fortran
Vector Processing (1)
Vector Processing (2)
Multi-threading
Other Issues: Debugging
Other Issues: Performance Tuning
Other Issues: Fault Tolerance
This Talk
What Does Coherency Mean?
Potential HW Coherency Solutions
Basic Snoopy Protocols
Basic Snoopy Protocols
PPT Slide
PPT Slide
PPT Slide
PPT Slide
PPT Slide
PPT Slide
Programmer’s Abstraction for a Sequential Consistency Model
This Talk
Top 500 (November 10, 2001)
Top500 Statistics
This Talk
Intel Architecture 64
Alpha is gone...
IBM’s POWER4 is the 2001/2002 winner
Implicit X Explicit Instruction Level Parallelism
Instruction Level Parallelism
Instruction Level Parallelism
IA-64 Innovations
This Talk
Below Above the line
Some Common Loop Optimizations
Unswitching
Loop Peeling
Index Set Splitting
Scalar Expansion
Loop Fusion
Loop Fusion (Another Example)
Loop Fission
Loop Reversal
Loop Interchanging
This Talk
Speedup
Absolute?Relative Speedup
Absolute ?Relative Speedup
Speedup
Which is Better?
It Depends!
Superlinear Speedups
Amdahl’s Law (1)
Amdahl’s Law (2)
Scalability
Choose Right Algorithm
This Talk
The Reality Is ...
Do You Need Parallelism?
Resistance to Parallelism
Starting Out...
Starting Out...
Granularity (1)
Granularity (2)
Granularity (3)
Architectures
Program Design
Vector Processing (3)
Distributed Memory (1)
Distributed Memory (2)
Communication
Synchronization
Message Passing (1)
Message Passing (2)
Master/Slave (1)
Master/Slave (2)
Pitfall: Deadlock
Pitfall: Deadlock
Pitfall: Load Balancing
Shared Memory (1)
Shared Memory (2)
Communication
Synchronization
Pitfall: Shared Data Access
Multi-threading
Simultaneous Multi-threading
Top 500
Top 500
Top 500
Top 500
Top 500
Top 500
Top 500
Conclusions
Reminders
We Want You!
|