CS240A: Applied Parallel Computing: Spring 2010

Interdisciplinary introduction to applied parallel computing on modern supercomputers. Topics include applications oriented architecture, software systems, MPI, UPC, Cilk, caches and vector processors, and parallel algorithms for algorithms such as linear algebra, N-body problems, multigrid, fast multipole, mesh partitioning, and graph algorithms. The course includes a project with hands-on emphasis on understanding the realities and myths of what is possible on the world's fastest machines.

Class Meeting

Mondays and Wednesdays 11:00-12:45, Trailer 932.

Instructor

• Office: 5109 Harold Frank Hall
• Office hours: see my web page.
• Email: gilbert@cs.ucsb.edu

Teaching Assistant

• Email: hnielsen@umail.ucsb.edu
• Office hours: Tuesday 1:00 - 2:00 and Wednesday 3:00 - 4:00 in Phelps 1413

Supercomputing Guru

• Engineering College Infrastructure
• Office: Harold Frank Hall 3116
• Phone: 805-893-8287

Discussion and Announcements

There is a Google discussion group for the class at http://groups.google.com/group/UCSB-Computer-Science-240A-Spring-2010. All students are expected to join this group and to watch it for course announcements. You may also use it to post questions (of general interest to the class) for the instructor and assistant. The mailing address for posting to the group is UCSB-Computer-Science-240A-Spring-2010@googlegroups.com.

Hans's TA Page

Resources, References, and Documentation

• To get access to Triton (which you'll need), you must generate an RSA public key file "id_rsa.pub" and email it to Stefan Boeriu, stefan@engineering.ucsb.edu, right away. On a Linux machine you can use "ssh -keygen -t rsa" to generate the file.

Course Requirements

• Research homework 1: 10%
• Programming homeworks 2-4 or 5: 45%
• Final project: 40%
• Class participation: 5%

Homework

• Homework 1 (due Mon Apr 5): Problem, completed assignments
• Homework 2 (due Mon Apr 12): Problem, sequential code, FAQ, hw2harness.c, hw2harness.h
• Homework 3 (due Mon Apr 26): Problem, sequential code, FAQ, hw3harness.c, hw3harness.h
• Homework 4 (due Mon May 3): Problem, harness and Makefile, FAQ

Final Project

• Due dates and requirements
• Last year's projects
• This year's projects
• Schedule for final presentations and demos

Class Schedule and Lecture Slides

• Mon Mar 29: Introduction [slides; bone density modeling; betweenness centrality]
  (Homework 1 assigned)

• Wed Mar 31: Parallel programming models and complexity; MPI introduction [MPI slides ]

• Mon Apr 5: MPI continued; finite differences and sparse linear systems [examples of MPI code, Matlab example files]
  (Homework 1 due, Homework 2 assigned)

• Tue Apr 6, 5:00-6:00pm, ESB 1003 (Cooper lab): SDSC orientation and performance tools (Stefan Boeriu, guest speaker)

• Wed Apr 7: Conjugate gradients and other Krylov iterations; parallel complexity measures [slides on complexity measures, cg and stencils]

• Mon Apr 12: CG implementation issues: dot product, matvec, stencils
  (Homework 2 due, Homework 3 assigned)

• Wed Apr 14: Models for the cost of moving data: stencil computations

• Mon Apr 19: Models for the cost of moving data: matrix multiplication [slides]

• Wed Apr 21: Sources of parallelism in physical simulation I: Types of problems [slides, Sharks and Fish, a model of real fish]

• Thu Apr 22, 9:00-10:25am, ESB 1003 (Cooper lab): Triton performance tools and numerical libraries (Mahidhar Tatieni, SDSC, guest speaker)

• Mon Apr 26: Multithreaded shared-memory model; Cilk++ [model and language pptx, pdf; examples pptx, pdf]
  (Homework 3 due, Homework 4 assigned)

• Wed Apr 28: Graphs and parallel graph algorithms [slides, papers on maximal independent sets, connected components, and strongly connected components; a good paper on architecture issues in graph computation; sequential and parallel MIS programs]

• Mon May 3: Betweenness centrality (Adam Lugowski, guest speaker) [references]
  (Homework 4 due)

• Wed May 5: (JRG travel, no class)

• Fri May 7: (Final project teams and proposals due)

• Mon May 10: (Class cancelled due to flight delays)

• Wed May 12: General sparse matrix computation [slides on dense and sparse linear systems]

• Mon May 17: Graph and hypergraph partitioning [slides]

• Wed May 19: Trees, scans, PRAMs, and log n [slides; a paper by Guy Blelloch on scans]
  (Final project progress reports due)

• Mon May 24: Grab bag [PGAS and UPC, Matlab*P, Intel multicores, Cray multithreads, GPUs, Challenges in combinatorial scientific computing]

• Wed May 26: Final project presentations [schedule]

• Mon May 31: (Memorial Day, no class)

• Wed Jun 2: Final project presentations [schedule]

• Thu Jun 3: Final project demos, 9:00 am to noon in GSL [schedule]

• Mon Jun 7: (Final project papers due, emailed to John and Hans)

Interesting Links

• Web site and white paper from Berkeley on the future of parallel computing in the multicore era.
• Top 500 SuperComputers
• Top 10 Algorithms
• Dead SuperComputer Society
• Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers
• A nice introduction to finite element methods in a biomedical application
• Horst Simon's Feb 2005 CSE panel talk

Similar Courses Elsewhere and Elsewhen

• Applied Parallel Computing, MIT (link)
• Applications of Parallel Computers, UC Berkeley (link)
• CS240A, Winter 2006, UCSB (link)