Lecture: TH 9:00am - 10:50am	Room: PHELP 2510
Instructor: Yuan-Fang Wang	Office: Harold Frank Hall, 3113
Office hours: TH 11:00am - 12:00pm	Phone: 893-3866
Enrollment code: 53322

Prerequisite:

Course Description:

This course will cover several topics on pattern recognition (PR), artificial neural networks (ANN), and machine learning (ML). Pattern recognition is a classical research area that deals with recognizing patterns (objects) based on their features (traits or appearance). It has seen wide applications in speech recognition, image analysis, target detection, optical character recognition, fingerprint identification, dating services, insurance fraud detection, DNA sequence alignment, protein structure matching, data mining, network intrusion detection, engine trouble shooting, among many others. Artificial neural networks provide a general computing framework which is purported to be highly parallel, distributed, and fault tolerant. Machine learning deals with algorithms and formulations that enable a machine to learn and improve its performance from experience, that is, to modify its behaviors and execution on the basis of acquired information and data analysis. These areas share a lot of commonality in addressing similar problems in classification and clustering.

Though active research activities are still being conducted in these areas, many useful results and practical algorithms have been developed which should prove useful. This course will discuss some of these results. You should consider concepts and algorithms presented here as general mathematic tools which hopefully will enrich your math "toolbox" and may become handy some day.

Grading:

• Class attendance and participation in the discussion = 10%
• Two - three assignments or miniprojects (written + programming) = 60%
• Class/research project presentations = 30%

There is no exam in the course. The final grade will be determined based on attendance and class participation, assignments, and research/project presentations. The difference between assignments (miniprojects) and class/research projects is if you get to choose the topics or work on assigned topics. Be warned that how much you get out of this course will in a very large degree be decided by your motivation to learn and your curiosity to explore and experiment with the many ideas and algorithms presented in the course.

Topics Covered:

We will discuss the following "traditional" topics (Topics may be deleted and additional topics added depending on the interests of the participants and the available time):

• Introduction to Pattern Recognition
• Parameter Estimation Techniques
• Non-parametric Estimation Techniques
• Linear Discriminant Functions
• Dimension Reduction Techniques
• Nonmetric Data
• Introduction to Artificial Neural Networks
• Simple Perceptrons
• Multi-layer Perceptrons
• Radial Basis Function Networks
• Mixture Density
• Unsupervised Clustering
• Unsupervised Hebbian and Competitive Learning
• Hopfield network
• Correlated and Time-Varying Events (Hidden Markov Models)

• Support Vector Machines
• Generalization Performance
• Boosting
• Kernel Methods
• Deep learning
• Deep learning applications to image and vision problems

Textbooks:

• T. Hastie, R. Tibshirani, and J. H. Friedman, "The Elements of Statistical Learning", 2nd edition, Springer, 2009
• Richard O. Duda, Peter E. Hart, and David G.Stork, "Pattern Classification", second edition, Wiley-Interscience, 2001
• Chritopher Bishop, "Pattern Recognition and Machine Learning,", Springer,2006.
• Tom M. Mitchell, "Machine Learning", McGraw-Hill, 1997
• Simon Haykin, "Neural Networks", second edition, Prentice Hall, 1999.
• John Shawe-Taylor and Nello Cristianini, "Kernel Methods for Pattern Analysis", Cambridge University Press, 2004

Other Useful References (oldies but goodies)

1. Duda and Heart, Pattern Classification and Scene Analysis, John-Wiley & sons, New York, NY, 1973.
2. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, New York, NY, 1972.

General class policies and announcements:

• This class has a Web site at http://www.cs.ucsb.edu/~cs291a. You will find there useful information such as announcements, deadlines, syllabus, assignments, and on-line copies of relevant research papers. It is your responsibility to check the Web site on a regular basis.
• It is never too early to start thinking about what you want to do for the class project. As mentioned, considerable freedom will be given here. An in-depth survey of a particular topic can be appropriate If some implementation, testing, and comparing study is a part (i.e., some programming exercise is included). A programming project should involve validation on real data sets (best if it is related to your own research so your effort is not wasted). As most classifiers are fairly straightforward to implement, and those complicated ones, such as neural network classifiers, trees, and SVM, are available publicly (with or without the source codes), e.g., Matlab Neural Network Toolbox, sklearn and Pandas. You can also find public data archives at many web sites, such as PR Archives and UCI Machine Learning web site. The complexity of the project most likely will be in data collection, feature selection, performance tuning, cross validation, comparison of performance using different classifiers, and validation and interpretation of results. For graduate students, particular emphasis will be on the creativity and originality of the project. You need to let me know as soon as you are done with the second assignment and no later than the 8th week what your project idea is. No elaborate project proposal is needed. Just drop by an email telling me the rough idea of your project (topics, data sets, tools, techniques, partners, etc.) This could be a single-person effort or a group project. You need to get my approval if you want to work as a team. More substantial efforts in research, data collection, implementation, validation, and creativity are expected for team projects.