CS 678 - Advanced Machine Learning and Neural Networks

Syllabus

 

Winter 2013, TuTh 1:35-2:50pm, 134 TMCB

Professor: Tony Martinez, 3326 TMCB, x6464, http://axon.cs.byu.edu/~martinez

martinez@cs.byu.edu

Office Hours: by appointment

 

 Course Website: http://axon.cs.byu.edu/~martinez/classes/678/

 

Goals:  Continuation in the study of the philosophy, utility, and models of machine learning, such that students are able to propose original research with potential follow-up in a graduate research program.  Expand the creativity of the students in all aspects of computing.

 

Prerequisites:  CS 478 (Introduction to Neural Networks and Machine Learning)

 

Text:  Papers available from the class website.   Also, we will read a few chapters from Machine Learning by Tom Mitchell.  You are expected to read the assigned literature before and optionally after the scheduled lecture.

 

Assignments:  We will have three projects during the semester allowing you to build and experiment with some new machine learning models.  You will present a paper of your choice to the class.  There will also be a midterm and a final.

 

Grading (~):

Paper Presentation

10%

Recurrent Neural Network Project

13%

Deep Learning Project

13%

Model of your choice Project

14%

Midterm

25%

Final

25%

 

Grading is on a curve and some amount of subjectivity is allowed for attendance, participation, perceived effort, etc.  If you think, you'll be all right.  The mid-term exam will be in the testing center and final exam will be in class on Monday, April 22 from 11:00am-2:00pm.

 

Late assignments:  Assignments are expected on time (beginning of class on due date).  Late papers will be marked off at 5%/school day late.  However, if you have any unusual circumstances (sick, out of town, unique from what all the other students have, etc.), which you inform me of, then I will not take off any late points.  Nothing can be accepted after the last day of class instruction.

 

Topics (Order is approximate):

á          Von Neumann bottleneck/neurobiology primer

á          Advanced Backpropagation Concepts

o    On-line vs. Batch

o    Classification Based Learning

o    Other (Higher Order nets, Ontological Nets)

á          Recurrent Neural Networks (Elman Nets, BPTT, RTRL)

á          Support Vector Machines (with brief primer on Quadric/Higher Order Machines and RBF networks)

á          Hopfield Networks

á          Boltzmann Machines

á          Deep Learning

á          HMMs (with Baum Welch Learning - EM algorithm), with detailed speech recognition as the example platform

á          MULTCONS, Hopfield Extensions

á          Rule Based Learning (Sequential Covering, CN2)

á          Reinforcement Learning

á          Ensembles (Variations, BMC vs BMA, Oracle Learning, etc.)

 

Next Group of Topics as Time Allows (Based on Needs):

á         Bias: Interesting/computable problems, Bias-Variance Decomposition

á          Semi-Supervised Learning

á          ADIB (Automatic Discovery of Inductive Bias)/Latest lab Research

á          Structured Prediction

á          Manifold Learning/Non-Linear Dimensionality Reduction

á          Record Linkage/Family History Directions

á          Meta-Learning

á          Feature Selection

á          Computational Learning Theory

á          Transfer Learning

á          Transduction

á          Other Unsupervised Learning Models

 

Topics and Readings Schedule:

Assignments:

Readings Presentations Schedule:

Project Presentations Schedule:              

University and Other Items: