CS 678 – Topics and Reading Assignments

Winter 2017


Advanced Backpropagation Concepts (BP Example Homework)

Required: On-line vs Batch

Optional: Classification Based Objective Functions

Optional: Conjugate Gradient

Slides:  Brain and Nervous System Overview

Slides:  Backpropagation

Slides:  Classification Based


Hopfield Networks

Required: Hopfield and Tank

Optional: Hopfield Chapter

Optional: Multcons Hopfield Extensions

Slides: Hopfield

Optional Slides: Multcons Slides


Boltzmann Machines

Required: Original Boltzmann

Slides: Boltzmann


Recurrent Neural Networks (BPTT Example Homework)

Required: Recurrent Neural Network Intro

Required: Recurrent Neural In Depth Chapter

Optional: BPTT (Backpropagation Through Time)

Slides:  Recurrent


Deep Learning

Slides: Deep Learning


Required: Deep Learning Overview (sections 1-4)

Required: Deep Networks – This is a nice easy to follow tutorial on deep nets overall with a good early focus on CNNs (may skip code section)

Suggested: Why deep networks are hard to train


Convolutional Networks

Required: Convolutional Neural Networks


Unsupervised Pre-Training: Deep Belief Networks and Stacked Auto-Encoders

Required: Representation Learning - Just intro and 15.1

Required: Deep Learning Overview (sections 5-7, 9, 10)

 (DBN Example Homework)

Suggested: UFLDL (Unsupervised Feature Learning and Deep Learning) Tutorial

Optional: RBM Training Notes


Supervised Deep Networks


Further Readings/Information: Deep Learning website and UFLDL website

One-line Deep learning books: Goodfellow, Nielson


Midterm – Testing Center (M-W Feb 27-Mar 1) – No class on Tuesday, Feb 28


Support Vector Machines

Required: Mitchell, T., Machine Learning, Ch. 8.4

Required: SVM Brief Introduction

Required: SVM Tutorial

Optional: An Introduction to Kernel Based Learning Algorithms

Slides:  SVM


Speech Recognition, Hidden Markov Models (HMM Baum-Welch Homework)

Required: Speech Overview

Slides: Speech Recognition

Required: HMM and Speech


Rule Based Learning (Sequential Covering), Homework

Required: Mitchell, T., Machine Learning, Ch. 10.1-10.3

Required: CN2

Slides: Rule Learning


Semi-Supervised Learning

Slides: Semi-Supervised


Ensembles and Bayesian Model Averaging

Slides: Ensembles, Ensembles and Bayesian Model Averaging


April 6 – Deep Learning Project Presentations (4 minutes each with 1 minute for questions) – Prepare carefully


Apr 11, 13, 18 – Model of your choice presentations – 10 minute presentation with 2-3 minutes question/discussion – Paper due beginning of class the day you present.


Final – In class, Monday, April 24 from 2:30pm-5:30pm