CS
678 – Topics and Reading Assignments

Winter
2019

__Advanced Backpropagation Concepts (BP Example Homework)__

Required: On-line
vs Batch

Optional: Classification
Based Objective Functions

Optional: Conjugate
Gradient

Slides: Brain
and Nervous System Overview

Slides: Backpropagation

Slides: Classification
Based

__Hopfield Networks__

Required: Hopfield Chapter

Optional: Hopfield
and Tank

Optional: Multcons Hopfield Extensions

Slides: Hopfield

Optional Slides: Multcons Slides

__Boltzmann Machines__

Required: Original
Boltzmann

Slides: Boltzmann

__Recurrent Neural Networks (BPTT Example Homework)__

Required: Recurrent
Neural Network Intro

Required: Recurrent
Neural In Depth Chapter

Optional: BPTT
(Backpropagation Through Time)

Slides: Recurrent

__Deep Learning__

Slides: Deep
Learning

__Introduction__

Required: Deep
Learning Overview (sections 1-4)

Required: Deep
Networks – This is a nice easy to follow tutorial on deep nets
overall with a good early focus on CNNs (may skip code section)

Suggested: Why
deep networks are hard to train

__Convolutional Networks__

Required: Convolutional
Neural Networks

__Unsupervised Pre-Training: Deep Belief Networks and Stacked
Auto-Encoders__

Required: Representation
Learning - Just intro and 15.1

Required: Deep
Learning Overview (sections 5-7, 9, 10)

(DBN Example Homework)

Suggested: UFLDL
(Unsupervised Feature Learning and Deep Learning) Tutorial

Optional: RBM
Training Notes

__Supervised Deep Networks__

Required: LSTM Blogs: Britz, Olah, and Arun

Further Readings/Information: Deep
Learning website and UFLDL
website

One-line Deep learning books: Goodfellow, Nielson

Midterm – Testing Center (M-W Feb 25-27) – No class on
Tu, Feb 26

__Support Vector Machines__

Required: Mitchell, T., Machine Learning, Ch. 8.4

Required: SVM
Brief Introduction

Required: SVM
Tutorial

Optional: An
Introduction to Kernel Based Learning Algorithms

Slides: SVM

__Speech Recognition, Hidden Markov Models (HMM Baum-Welch Homework)__

Required: Speech
Overview

Slides: Speech
Recognition

Required: HMM
and Speech

__Rule Based Learning (Sequential Covering), Homework__

Required: Mitchell, T., Machine Learning, Ch. 10.1-10.3

Required: CN2

Slides: Rule
Learning

__Semi-Supervised Learning__

Slides: Semi-Supervised

__Ensembles and Bayesian Model Averaging__

Slides: Ensembles,
Ensembles
and Bayesian Model Averaging