Machine Learning and Pattern Recognition

Machine Learning and Pattern Recognition

Instructor: Jayadev Acharya, 382 Rhodes Hall
Office hours: Mo 10-11, 380 Rhodes Hall

Course Staff:
Sourbh Bhadane, OH: We 2-3, 380 Rhodes Hall
Yujia Ma, OH: Tu 4.30-5.30, 380 Rhodes Hall (starting 2/5)
Ziteng Sun, OH: Th 4-5.30, 380 Rhodes Hall

Lectures: MWF 11:15-12:05, 203 Phillips Hall
Discussion: Tu 9:05-9:55, 203 Phillips Hall


The course is devoted to the understanding the mathematical underpinnings of machine learning, and consider some applications within ECE. The first half of the course will provide basics of supervised learning, unsupervised learning, and learning theory. The second half will focus on some applications in some fields with ECE.
A break-up of the course coverage is the following:
Supervised Learning: 5 weeks
Unsupervised Learning: 2 weeks
Learning Theory: 1.5 weeks
Engineering applications: 4 weeks

Prerequisites: Linear Algebra (Math2940 or equivalent), Basic Probability and Statistics (ECE3100, STSCI3080, ECE3250 or equivalent). The discussion sessions will be devoted to reinforce various concepts that we will not cover but use in the lectures.


We will take materials from various sources. Some books are:
  • Pattern Recognition and Machine Learning, Christopher Bishop
  • A Course in Machine Learning, Hal Daume III (available here)
  • Machine Learning: a Probabilistic Perspective, Kevin Murphy (available online at Cornell Library)
  • The Elements of Statistical Learning: Trevor Hastie, Robert Tibshirani, Jerome Friedman (available here)
  • Machine Learning, Tom Mitchell


Assignments: 25%
Almost weekly assignments.

Miniproject: 22.5%.

Midterm: 20%

Final: 27.5%

We will use Piazza for announcements, discussions, and for posting materials. We will use CMS for uploading assignments.


We will have ten assignments.
  • The lowest score will be dropped.
  • You will have six late days that you can choose to use as you see fit without any penalty. Late days are counted in quantums of one day.

Assignment Assign date Due date (eod)
A1 1/30 2/8
A2 2/6 2/15
A3 2/13 2/22
A4 2/20 3/1
A5 2/28 3/8
A6 3/13 3/22
A7 3/20 3/29
A8 3/27 4/12
A9 4/17 4/26
A10 4/24 5/3


The midterm will be held on 3/14 at 7.30 PM

Tentative Syllabus (subject to change)

Supervised Learning: Decision Trees, Naive Bayes, Linear classifiers, Perceptron, Logistic regression, Linear regression, Regularization, SVM, Kernel trick, Bagging, Boosting, Bias variance trade-offs, Neural networks

Unsupervised Learning: Principal Components Analysis, Johnson Lindenstrauss Lemma, Clustering

Learning Theory: VC dimension, generalization bounds

ECE applications: Hidden Markov Models, Viterbi Algorithm, Speech recognition, Compressive Sensing and sparsity, Collaborative Filtering, Managing data centers/cloud computing


We will use python as the programming language. Familiarity with python will be helpful, but not necessary. Please install Python, and play around. How to install: We will use scipy and sci-kit-learn, which come with an anaconda update, but need manual installation for the second option (use pip).
Please install Jupyter. It is an interactive (and awesome) way to do ML using Python. It might be easier to install with Anaconda, but if you can installed Python the hard way, you should be able to install it too.