DC stands for ``Introduction to High Performance Scientific Computing by David L. Chopp. SIAM 2019.
PP stands for "An Introduction to Parallel Programming" by Peter Pacheco, 2011.
PDF
Recording on 1/7 and 2/3 was unavaiable due to a classroom equipment failure.
| # | Topics | References | Slides | Assignments | Additional references |
| 1 |
Parallel Architectures Lect. 1/7 1/8 Dis. 1/9 . |
PP Ch. 2 |
Introduction to CS140 Parallel architectures Discussion 1 Slides |
Exercise 1 (Due on Wed 1/15 3pm) |
Background: Overview of C C memory layout |
| 2 |
Parallel Software Lect. 1/13, 1/15 Dis. 1/16 |
Lecture notes(Section 1) |
Parallelism and Parallel Software MPI Discussion 2 Slides |
Exercise 2 (Due on Wed 1/22 3pm) |
|
| 3 |
Distributed Memory Programming with MPI Program Parallelization Lect. 1/22 Dis. 1/23 |
MPI tutorial
Lecture notes(Section 3) |
Continue on MPI If time permits, Program Parallelization Discussion 3 MPI |
PA1(MPI) (Due on Wed. 1/29 11:59pm) |
PP (Ch. 3) or DC (Ch. 16, 17, 19) How to run MPI on CSIL How to run MPI on Expanse |
| 4 |
Program Parallelization Shared Memory Programming with OpenMP Lect. 1/27, 1/29, Dis. 1/30 |
PP (Ch. 5) Lecture notes (Section 4&6 ) |
Continue on Program Parallelization OpenMP Discussion 4 Discussion: parallel matrix multiplication |
Exercise 3 (Due on Wed 2/5 3pm) |
|
| 5 |
Scientific Computing Algorithms Lect. 2/3, 2/5 Dis. 2/6 |
Lecture notes (Sections 6&8) |
Scientific computing algorithms Midterm summary If time permits, Pthreads Discussion 5: OpenMP/EX3 |
PA2 (OpenMP/Pthreads) Part A due: Mon 2/17 Part B due: Wed 2/26 |
|
| 6 |
Shared memory programming with Pthreads Lec. 2/12 . Dis. 2/13. |
Pthreads tutorial |
Continue on Pthreads Discussion 6 Pthreads |
EX4 (Due on Wed 2/19 3pm) |
GDB debugging with pthreads |
| 7 |
Parallel Programming with GPU Lec. 2/19. Dis. 2/20 |
David's intro on CUDA |
GPU I GPU II Discussion: EX4 sol Midterm |
PA2 Part B due on Wed 2/26 |
CUDA Programming |
| 8 |
Iterative Methods for Machine Learning Applications Lec. 2/24 2/26 Dis. 2/27 |
Partial derivaties SGD |
SGD and its Parallelization Discussion 8 (GPU) |
PA3(GPU) (Due on Wed. 3/5) |
Using GPU machines on Expanse DC (Ch. 22, 23, 24.1) |
| 9 10 |
MapReduce and Spark Parallel Programming 3/3 3/5 Dis. 3/6 3/10 3/10 (no class) Dis 3/13 |
MapReduce short guide Spark RDD examples |
Parallel Programming with MapReduce MapReduce Programming with Spark Discussion (SGD/MapReduce) CS140 Summary More on Parallel Applications |
EX5 Due on Wed 3/12 @3pm |
Holidays : 1/ 20 (MLK Jr. Day) 2/17 (Presidents' Day)
Midterm : Feb 10 Mon (tentatively) 9:30am-10:45am @PHELP 1160
Final exam: Wednesday, March 19, 2025 8:00 AM - 11:00 AM @PHELP 1160