CS
678 – Topics and Reading Assignments
Winter
2019
Advanced Backpropagation Concepts (BP Example Homework)
Required: On-line
vs Batch
Optional: Classification
Based Objective Functions
Optional: Conjugate
Gradient
Slides: Brain
and Nervous System Overview
Slides: Backpropagation
Slides: Classification
Based
Hopfield Networks
Required: Hopfield Chapter
Optional: Hopfield
and Tank
Optional: Multcons Hopfield Extensions
Slides: Hopfield
Optional Slides: Multcons Slides
Boltzmann Machines
Required: Original
Boltzmann
Slides: Boltzmann
Recurrent Neural Networks (BPTT Example Homework)
Required: Recurrent
Neural Network Intro
Required: Recurrent
Neural In Depth Chapter
Optional: BPTT
(Backpropagation Through Time)
Slides: Recurrent
Deep Learning
Slides: Deep
Learning
Introduction
Required: Deep
Learning Overview (sections 1-4)
Required: Deep
Networks – This is a nice easy to follow tutorial on deep nets
overall with a good early focus on CNNs (may skip code section)
Suggested: Why
deep networks are hard to train
Convolutional Networks
Required: Convolutional
Neural Networks
Unsupervised Pre-Training: Deep Belief Networks and Stacked
Auto-Encoders
Required: Representation
Learning - Just intro and 15.1
Required: Deep
Learning Overview (sections 5-7, 9, 10)
(DBN Example Homework)
Suggested: UFLDL
(Unsupervised Feature Learning and Deep Learning) Tutorial
Optional: RBM
Training Notes
Supervised Deep Networks
Required: LSTM Blogs: Britz, Olah, and Arun
Further Readings/Information: Deep
Learning website and UFLDL
website
One-line Deep learning books: Goodfellow, Nielson
Midterm – Testing Center (M-W Feb 25-27) – No class on
Tu, Feb 26
Support Vector Machines
Required: Mitchell, T., Machine Learning, Ch. 8.4
Required: SVM
Brief Introduction
Required: SVM
Tutorial
Optional: An
Introduction to Kernel Based Learning Algorithms
Slides: SVM
Speech Recognition, Hidden Markov Models (HMM Baum-Welch Homework)
Required: Speech
Overview
Slides: Speech
Recognition
Required: HMM
and Speech
Rule Based Learning (Sequential Covering), Homework
Required: Mitchell, T., Machine Learning, Ch. 10.1-10.3
Required: CN2
Slides: Rule
Learning
Semi-Supervised Learning
Slides: Semi-Supervised
Ensembles and Bayesian Model Averaging
Slides: Ensembles,
Ensembles
and Bayesian Model Averaging