CS 678 – Topics and Reading Assignments

Winter 2017

 

Advanced Backpropagation Concepts (BP Example Homework)

Required: On-line vs Batch

Optional: Classification Based Objective Functions

Optional: Conjugate Gradient

Slides:  Brain and Nervous System Overview

Slides:  Backpropagation

Slides:  Classification Based

 

Hopfield Networks

Required: Hopfield and Tank

Optional: Hopfield Chapter

Optional: Multcons Hopfield Extensions

Slides: Hopfield

Optional Slides: Multcons Slides

 

Boltzmann Machines

Required: Original Boltzmann

Slides: Boltzmann

 

Recurrent Neural Networks (BPTT Example Homework)

Required: Recurrent Neural Network Intro

Required: Recurrent Neural In Depth Chapter

Optional: BPTT (Backpropagation Through Time)

Slides:  Recurrent

 

Deep Learning

Slides: Deep Learning

Introduction

Required: Deep Learning Overview (sections 1-4)

Required: Deep Networks – This is a nice easy to follow tutorial on deep nets overall with a good early focus on CNNs (may skip code section)

Suggested: Why deep networks are hard to train

 

Convolutional Networks

Required: Convolutional Neural Networks

 

Unsupervised Pre-Training: Deep Belief Networks and Stacked Auto-Encoders

Required: Representation Learning - Just intro and 15.1

Required: Deep Learning Overview (sections 5-7, 9, 10)

 (DBN Example Homework)

Suggested: UFLDL (Unsupervised Feature Learning and Deep Learning) Tutorial

Optional: RBM Training Notes

 

Supervised Deep Networks

 

Further Readings/Information: Deep Learning website and UFLDL website

One-line Deep learning books: Goodfellow, Nielson

 

Midterm – Testing Center (M-W Feb 27-Mar 1) – No class on Tuesday, Feb 28

 

Support Vector Machines

Required: Mitchell, T., Machine Learning, Ch. 8.4

Required: SVM Brief Introduction

Required: SVM Tutorial

Optional: An Introduction to Kernel Based Learning Algorithms

Slides:  SVM

 

Speech Recognition, Hidden Markov Models (HMM Baum-Welch Homework)

Required: Speech Overview

Slides: Speech Recognition

Required: HMM and Speech

 

Rule Based Learning (Sequential Covering), Homework

Required: Mitchell, T., Machine Learning, Ch. 10.1-10.3

Required: CN2

Slides: Rule Learning

 

Semi-Supervised Learning

Slides: Semi-Supervised

 

Ensembles and Bayesian Model Averaging

Slides: Ensembles, Ensembles and Bayesian Model Averaging

 

April 6 – Deep Learning Project Presentations (4 minutes each with 1 minute for questions) – Prepare carefully

 

Apr 11, 13, 18 – Model of your choice presentations – 10 minute presentation with 2-3 minutes question/discussion – Paper due beginning of class the day you present.

 

Final – In class, Monday, April 24 from 2:30pm-5:30pm