DC stands for ``Introduction to High Performance Scientific Computing by David L. Chopp. SIAM 2019.
PP stands for "An Introduction to Parallel Programming" by Peter Pacheco, 2011.
PDF
| # | Topics | References | Slides | Assignments | Additional references |
| 1 |
Parallel Architectures Parallel Software Lect. 1/5 1/7 Dis. 1/8 |
PP Ch. 2 |
Introduction to CS140 Parallel architectures Parallel Software Discussion 1 Slides |
Exercise 1 (Due on Wed 1/14 1pm) |
Background: C C memory layout |
| 2 |
Parallel Software and MPI Lect. 1/12 1/14 Dis. 1/15 |
Lecture notes(Sect. 1 ) MPI tutorial |
Continue on Parallel Software MPI Discussion 2 Slides |
Exercise 2 (Due on Wed 1/21 noon) |
PP (Ch. 3) or DC (Ch. 16, 17, 19) How to run MPI on CSIL How to run MPI on Expanse |
| 3 |
Parallel Programming with MPI Program Parallelization Lect. 1/21 Dis. 1/22 |
Lecture notes(Sect. 3) |
Continue on MPI Program Parallelization Discussion 3 MPI |
PA1(SIMD and MPI) (Due on Wed. 1/28 11:59pm) |
|
| 4 |
Program Parallelization Scientific Computing Algorithms OpenMP Shared Memory Programming Lect. 1/26 1/28 Dis. 1/29 |
PP (Ch. 5) Lecture notes (Sections 4, 6&8 ) |
Continue on Program Parallelization Scientific computing algorithms OpenMP Discussion 4 Discussion: parallel matrix multiplication |
Exercise 3 (Due on Wed 2/4 noon) |
|
| 5 |
OpenMP Cache-aware programming Pthreads Lect. 2/2 2/4 Dis. 2/5 |
PP (Ch. 5) |
Continue on OpenMP Cache-aware programming and BLAS Midterm summary Pthreads Discussion 5: OpenMP/EX3 |
PA2 (OpenMP/Pthreads/BLAS) Part A due: Tue 2/17 Parts B/C due: Wed 2/25 |
|
| 6 |
Pthreads programming CUDA GPU programming Lec. 2/11 . Dis. 2/12 |
Pthreads tutorial |
Pthreads GPU I Discussion 6: Pthreads and BLAS |
EX4 (Due on Wed 2/18 noon) |
GDB debugging with pthreads |
| 7 |
Parallel Programming with GPU Lec. 2/17 (Holiday) 2/18 Dis. 2/19 |
David's intro on CUDA |
GPU II Discussion: EX4 sol |
PA2 Part B due on Wed 2/25 |
CUDA Programming |
| 8 |
Iterative Methods for Machine Learning Applications Lec. 2/23 2/25 Dis. 2/26 |
Partial derivaties SGD |
SGD and its Parallelization Parallel Programming with MapReduce Discussion 8 (GPU) |
PA3 GPU(Due on Wed. 3/4) |
Using GPU machines on Expanse DC (Ch. 22, 23, 24.1) |
| 9 10 |
MapReduce and Spark Parallel Programming More on Parallel Applications 3/2 3/4 Dis. 3/5 3/11 |
MapReduce short guide Spark RDD examples Transformers CNN |
Continue on MapReduce MapReduce Programming with Spark More on Parallel Applications Discussion 9 (SGD/MapReduce) CS140 Summary EX5 |
EX5 Due on Tu 3/9 @8am Optional exercise Due on Th 3/19 Noon |
Holidays : 1/ 19 Mon (MLK Jr. Day) 2/16 Mon (Presidents' Day)
Midterm : Feb 9 Mon (tentatively) 9:30am-10:45am @lecture room
Final exam: Wednesday, March 18, 2026 8:00 AM - 11:00 AM @lecture room (LSB 1001)