CS 678 – Topics and Reading Assignments

Winter 2019


Advanced Backpropagation Concepts (BP Example Homework)

Required: On-line vs Batch

Optional: Classification Based Objective Functions

Optional: Conjugate Gradient

Slides:  Brain and Nervous System Overview

Slides:  Backpropagation

Slides:  Classification Based


Hopfield Networks

Required: Hopfield Chapter

Optional: Hopfield and Tank

Optional: Hopfield in Wikipedia

Optional: Multcons Hopfield Extensions

Slides: Hopfield

Optional Slides: Multcons Slides


Boltzmann Machines

Required: Original Boltzmann

Slides: Boltzmann


Recurrent Neural Networks (BPTT Example Homework)

Required: Recurrent Neural Network Intro

Required: Recurrent Neural In Depth Chapter

Optional: BPTT (Backpropagation Through Time)

Slides:  Recurrent


Deep Learning

Slides: Deep Learning


Required: Deep Learning Overview (sections 1-4)

Required: Deep Networks – This is a nice easy to follow tutorial on deep nets overall with a good early focus on CNNs (may skip code section)

Suggested: Why deep networks are hard to train


Convolutional Networks

Required: Convolutional Neural Networks


Unsupervised Pre-Training: Deep Belief Networks and Stacked Auto-Encoders

Required: Representation Learning - Just intro and 15.1

Required: Deep Learning Overview (sections 5-7, 9, 10)

 (DBN Example Homework)

Suggested: UFLDL (Unsupervised Feature Learning and Deep Learning) Tutorial

Optional: RBM Training Notes


Supervised Deep Networks


Required: LSTM Blogs: Britz, Olah, and Arun

Further Readings/Information: Deep Learning website and UFLDL website

One-line Deep learning books: Goodfellow, Nielson


Midterm – Testing Center (M-W Feb 25-27) – No class on Tu, Feb 26


Support Vector Machines

Required: Mitchell, T., Machine Learning, Ch. 8.4

Required: SVM Brief Introduction

Required: SVM Tutorial

Optional: An Introduction to Kernel Based Learning Algorithms

Slides:  SVM


Speech Recognition, Hidden Markov Models (HMM Baum-Welch Homework)

Required: Speech Overview

Slides: Speech Recognition

Required: HMM and Speech


Rule Based Learning (Sequential Covering), Homework

Required: Mitchell, T., Machine Learning, Ch. 10.1-10.3

Required: CN2

Slides: Rule Learning


Semi-Supervised Learning

Slides: Semi-Supervised


Ensembles and Bayesian Model Averaging

Slides: Ensembles, Ensembles and Bayesian Model Averaging