Introduction

Profiling Code for Performance

Performance Concepts

Figure 1

Dot product formula
Dot product between vectors X and Y

Figure 2

Dot product formula
Dot product between vectors X and Y

Figure 3

Dot product formula
Dot product between vectors X and Y

Figure 4

Dot product formula
Dot product between vectors X and Y

Figure 5

Dot product formula
Dot product between vectors X and Y

Figure 6

Diagram of the memory hierarchy in a typical computer
Memory Hierarchy in a Computer

Figure 7

Formula and diagram of a matrix multiply
Matrix multiplication C = A * B

Figure 8

Formula and diagram of a matrix multiply
Matrix multiplication C = A * B

Figure 9

Formula and diagram of a matrix multiply
Matrix multiplication C = A * B

Figure 10

Formula and diagram of a matrix multiply
Matrix multiplication C = A * B

Figure 11

Formula and diagram of a matrix multiply
Matrix multiplication C = A * B

Parallel Computing Concepts

Multi-Threaded Programs

Figure 1

Shared-memory multi-threaded dot product showing the memory layout of both vectors
Diagram of a shared-memory multi-threaded dot product

Message-Passing Programs

Figure 1

Distributed-memory dot product showing the layout of both vectors on both computers
Diagram of a distributed-memory multi-node dot product on two computers

Figure 2

Distributed-memory dot product showing the layout of both vectors on both computers
Diagram of a distributed-memory multi-node dot product on two computers

Figure 3

Distributed-memory dot product showing the layout of both vectors on both computers
Diagram of a distributed-memory multi-node dot product on two computers

Figure 4

Distributed-memory dot product showing the layout of both vectors on both computers
Diagram of a distributed-memory multi-node dot product on two computers

Figure 5

Distributed-memory dot product showing the layout of both vectors on both computers
Diagram of a distributed-memory multi-node dot product on two computers

Language Survey

C and C++ Languages

The Fortran Language

The Python Language

The R Language

The Matlab Language

Array Jobs

Accelerating Scientific Computing with GPUs

High-Throughput Computing

HPC Resources