CS240A Resource Page
Machine documentation
MPI language resources
MPI is the lowest common denominator of parallel programming, a standard
implementation of the message-passing programming model (which dates back
to the first commercial parallel machines in the 1980s) that has been
augmented with some newer constructs like collectives
(such as broadcast and global sum) and one-sided communication.
The great advantages of MPI are that it runs on practically any parallel
machine you can name, and it gives you very detailed control over all
the communication that happens in your program.
UPC language resources
UPC (universal parallel C) is a language that combines much of the
ease of use of the shared-memory programming model with the ability to
control data locality for performance on a distributed-memory machine.
You can download and install a UPC compiler on pretty much any parallel
machine; it will also be available on some of the SDSC machines.
Cilk language resources
Cilk is a version of C and C++ that has been extended for
shared-memory parallel programming, targeted especially at
multicore and manycore chips.
There are several versions of Cilk around, with very confusing names.
The one we'll use is called "Cilk Plus"
or sometimes "Intel Cilk Plus",
and is built into the Intel C compiler "icc".
(Older versions you might run into are called
"Cilk++", "Intel Cilk++", and just "Cilk".
They all have slightly different syntax and tools,
so please stick to Cilk Plus.
General programming and data structures resources
Books on parallel computing
- Lin, C., and Snyder, L. 2008. Principles of Parallel Programming, Addison Wesley.
- Pacheco, P. 2011. An Introduction to Parallel Programming, Morgan Kaufmann.
- Mattson, T. G., Sanders, B. A., and Massingill, B. L. 2004. Patterns for Parallel Programming, Addison Wesley.
- Grama, A., Gupta, A., Karypis, G., and Kumar, V. 2003. Introduction to Parallel Computing, Second Edition, Addison Wesley.
- Dongarra, J., et al. 2002. The Sourcebook of Parallel Computing, Morgan Kaufmann.
- Goedecker, S., and Hoisie, A. 2001. Performance Optimization of Numerically Intensive Codes, Society for Industrial Mathematics.
- Foster, I. 1995. Designing and Building Parallel Programs, Addison Wesley, also available online.
Numerical algorithms resources
- J. Shewchuk, An
introduction to the conjugate gradient method without the agonizing pain.
What it says. A good paper.
- J. Demmel,
Applied
Numerical Linear Algebra. SIAM, 1997.
This is a wonderful book and you should buy it if you plan
to do anything in computational science or numerical analysis.
- Y. Saad,
Iterative
Methods for Linear Systems. SIAM, second edition 2003.
The first edition is available
online
without charge.
- R. Barrett and 9 other authors,
Templates
for the Solution of Linear Systems:
Building Blocks for Iterative Methods. SIAM, 1994.
Available
online.
This contains very concise but very useful descriptions of many
iterative methods.
- W. Briggs, V. Henson, and S. McCormick.
A
Multigrid Tutorial. SIAM, second edition 2000.
A superb tutorial. The text isn't online but this home
page includes an excellent set of slides that follows the book.
- Stephen Boyd et al.,
Convex Optimization.
This is a link to the Stanford MOOC
that began on January 21, 2014 by the guru of convex optimization methods.
Interesting links
Similar courses elsewhere and elsewhen
- Applications of Parallel Computers, UC Berkeley
(link)
- Applied Parallel Computing, MIT
(link)
- The 2011 version of CS240A
(link)
- CS140, the undergraduate version of this class
(link)