CS 219 References

CS 219, Spring 2018: References

Textbooks

I will assign reading in these three books, which will be on reserve at the library. They are also available online from the library via the UCSB VPN. The textbooks and several of the other references are published by SIAM, the Society for Industrial and Applied Mathematics. UCSB students can sign up for free student membership in SIAM, which gives you a big discount on their book prices (and is a good idea for lots of other reasons too).

T. Davis. Direct Methods for Sparse Linear Systems. SIAM, 2006. This is a concise but thorough introduction to most of the basic algorithms for sparse direct solvers. It includes C code in a spartan but readable style; all the code (complete with Matlab interfaces) is on the book's web site.
Y. Saad. Iterative Methods for Linear Systems. SIAM, second edition 2003. A comprehensive background on iterative methods, especially strong on incomplete factorization preconditioning for nonsymmetric problems.
W. Briggs, V. Henson, and S. McCormick. A Multigrid Tutorial. SIAM, second edition 2000. A really excellent tutorial. This home page includes a good set of slides that follows the book.

Other books

R. Barrett and 9 other authors. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. SIAM, 1994. The full text is also available online in postscript and html without charge. This contains very concise but very useful descriptions of many iterative methods, with a little on preconditioning.
A. Greenbaum. Iterative Methods for Solving Linear Systems. SIAM, 1997. This contains a wonderfully thorough and quite readable exposition of the theoretical background for the Krylov subspace iterative methods, plus a briefer overview of preconditioners, multigrid, and domain decomposition.
A. George and J. Liu. Computer Solution of Sparse Positive Definite Systems. Prentice-Hall, 1981. This is the classic work on sparse Cholesky factorization, introducing the graph-theoretic approach. It was written before elimination trees, supernodal and multifrontal methods, and the modern implementations of minimum degree and graph partitioning.
I. Duff, A. Erisman and J. Reid. Direct Methods for Sparse Matrices. Oxford University Press, second edition 2017. This is especially strong on sparse nonsymmetric methods.
A. George, J. Gilbert, and J. Liu. Graph Theory and Sparse Matrix Computation. Springer-Verlag, 1993. This is a broad collection of papers.
B. Smith, P. Bjorstad, and W. Gropp. Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations. Cambridge University Press, 1996. A comprehensive book on various domain decomposition methods and multigrid.
K. Sigmon and T. Davis. Matlab Primer. CRC Press, sixth edition 2002. A good introduction to Matlab, concise but thorough.
J. Kepner and J. Gilbert, eds. Graph Algorithms in the Language of Linear Algebra. SIAM, 2011. Using sparse matrices to compute with graphs, instead of the other way around.
J. Demmel. Applied Numerical Linear Algebra. SIAM, 1997. This is a wonderful book and you should buy it if you plan to do anything in computational science or numerical analysis. It has an excellent chapter on conjugate gradients, but no sparse direct methods and not much on preconditioning.
L. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, 1997. This is my favorite book on the numerical aspects of linear systems, eigenvalues, and their algorithms. Nothing on sparse direct methods, but every sparse matrix person should read it anyway.
G. Golub and C. Van Loan. Matrix Computations. Johns Hopkins University Press, third edition 1996. A very careful and thorough reference on numerical linear algebra.

Papers and other references (this list will grow during the quarter)

J. Shewchuk, An introduction to the conjugate gradient method without the agonizing pain. (What it says. A good paper.)
Michele Benzi's survey of preconditioning.

References on solving Laplacian linear systems

Dan Spielman, Algorithms, graph theory, and linear equations in Laplacian matrices. A very readable survey.
Nisheeth Vishnoi, Lx = b: Laplacian solvers and their algorithmic applications. A long, detailed, valuable survey paper from a theoretical point of view.
Sivan Toledo and Haim Avron's book chapter on combinatorial preconditioning.
Doron Chen and Sivan Toledo's experiments with the original Vaidya support graph preconditioners.
Erik Boman and Bruce Hendrickson, Support theory for preconditioning. A very clean linear algebraic framework for combinatorial preconditioning.
Erik Boman, Doron Chen, Bruce Hendrickson, and Sivan Toledo, Maximum-weight-basis preconditioners. Extends support theory to factor-width-2 (i.e. symmetric diagonally dominant) matrices.
Dan Spielman and Shang-Hua Teng, Nearly-linear time algorithms for preconditioning and solving symmetric, diagonally dominant linear systems. An elaborate and beautiful theoretical result; the work of teasing out a practical method is still going on.
Ioannis Koutis, Gary L. Miller and Richard Peng, Approaching optimality for solving SDD systems. A large step toward making the Spielman-Teng approach practical.
Koutis, Miller, and Tolliver, Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing. Implementation and experiments with a combinatorial multigrid method (which they call CMG) that uses some of the ideas in Spielman/Teng and Koutis/Miller/Peng.
Jon Kelner, Lorenzo Orecchia, Aaron Sidford, and Allen Zhu. A simple, combinatorial algorithm for solving SDD systems in nearly-linear time. A strikingly simple method (not preconditioned CG) that uses some ideas from support theory.
Rasmus Kyng and Sushant Sachdeva. Approximate Gaussian elimination for Laplacians -- fast, sparse, and simple. Probablistic incomplete factorization gives a theoretically linear times polylog Laplacian solver.
Oren Livne and Achi Brandt. Lean algebraic multigrid (LAMG): Fast graph Laplacian linear solver. Using multigrid ideas for Laplacian linear systems.
Seymour Parter. The use of linear graphs in Gauss elimination. The classic 1961 paper that first modeled Gaussian elimination as a process on graphs, and proved that trees can be factored by GE without fill.

References on graph algorithms and linear algebra:

The Graph BLAS forum, which develops standard API's for sparse matrix primitives for use in graph algorithms.
Tim Davis's SuiteSparse contains a sequential C implementation of Graph BLAS.
A. Buluç and J. R. Gilbert. The combinatorial BLAS: Design, implementation, and applications. The parallel library that led to Graph BLAS.
M. Wolf, M. Deveci, J. Berry, S. Hammond, and S. Rajamanickam. Fast linear algebra-based triangle counting with KokkosKernels. A matrix approach to a graph analytic.
A. Buluç and J. R. Gilbert. Parallel sparse matrix-matrix multiplication and indexing: Implementation and experiments. A highly parallel implementation of the central primitive in Graph BLAS.
A. Buluç et al. High-productivity and high-performance analysis of filtered semantic graphs. Using sparse matrices to do analytics on semantic graphs.

References on multigrid and domain decomposition:

MGNet home page. (A repository of all sorts of multigrid software, tutorials, papers, and information.)

Software

SuiteSparse, a comprehensive library of C sparse matrix routines by Tim Davis that can be used either standalone or with Matlab.
TAUCS, a library of software by Sivan Toledo's group at Tel-Aviv that contains direct solvers and many different preconditioners, including several support-graph preconditioners.
MatlabBGL, a package of Matlab graph routines by David Gleich that uses the Boost Graph Library.
Meshpart, a somewhat older package of Matlab graph routines that I tend to use for demos because I know it pretty well :).
SuperLU, a library of sparse direct solvers for both symmetric and nonsymmetric problems. In addition to sequential code, SuperLU includes a multithreaded code for shared-memory machines and a message-passing MPI code for distributed-memory machines.
BTER and Feastpack, a library by Kolda, Pinar, et al. that includes a good generator for random graphs with specified degree distribution and clustering behavior.