CS
778R – Readings in Machine Learning - Syllabus

Winter 2016, TuTh 9:30-10:45am,
134 TMCB

Professor: Tony Martinez, 3336 TMCB, x6464, **http://axon.cs.byu.edu/~martinez**

martinez at cs dot byu dot edu

Office Hours: by appointment

* Course Website: http://axon.cs.byu.edu/~martinez/classes/778/*

__Goals__: Advanced study and discussion of
relevant and recent machine learning research

__Prerequisites__: CS 478 (Introduction to Neural Networks
and Machine Learning), CS 678 (Topics in ML) recommended

__Text__: Mostly **papers** which will be made available from this website. You are
expected to read the assigned literature **before
**the scheduled lecture. We have
no specific topics that we must cover.
We will try to cover topics of interest to those attending the class.

__Teaching/Implementation
Assignment__: You will each teach one part of the
course on a topic/model of special interest to you, and also of interest to the
rest of the class. You will take
about 2 lectures (1 week) to do this (minimum 1 lecture, maximum two). You will provide one paper/reading on
the model/topic that all the class will read before the discussion. Your teaching should teach and motivate
the overall topic and then go into detail on a particular model within the
topic. Include active participation
from the class. Your teaching should draw on information beyond just the one
paper given to the class. This is
not just a chance to present a paper but to teach a topic as if you were a
professor in your own class. You may pattern it somewhat after the lectures you
have seen me give. First we motivate the area (e.g. Deep learning), introducing
and motivating the high level approach, including some history and other models
used to solve it. Then we go into depth on a particular model (e.g. Stacked
Auto-Encoders) to the extent that we are capable of walking through a full
example on the board or implementing the model, if needed. It is best if we can understand it at a
sufficient level of detail that we can walk through how the model would work
for specific data examples.
Discussion should also include pros, cons, latest current work from others,
and suggested directions for improving the chosen topic/model. Thus you should, at least at a high
level, be aware of, and briefly discuss with us other approaches/models that
are common for your topic. Note that you basically get to become a Òsemi-expertÓ
in a particular area and then teach it to us.

Power point slides are usually
helpful. You will also __implement
and test__ the model you teach before you teach it and show results during
your lectures on at least two data sets to help us better understand the
capacity of the model. Implementing
it will help you to understand it a lot better, and thus teach it better. You
could include demos or other pedagogical aids in your teaching to help us gain
insights. Note that the focus should be on us understanding the topic, with the
particular model being an in-depth tool to help us gain deeper understanding of
the topic. To clarify, here are a
few examples of topic/model pairs you may have learned from me in previous
classes.

Topic |
Model |

Decision Trees |
ID3 |

Clustering |
K-means, HAC |

Deep Learning |
DBN, SAE |

Recurrent Networks |
BPTT |

Rule Based Learning |
CN2 |

The model you present/implement should
be a model that we have not previously covered in 478 or 678. However, you may choose
extensions/variations on models previously covered, if the variations have
sufficient educational Òvalue-addedÓ beyond what we have covered before. For example, you might choose to teach
about a new variation of deep learning that we did not cover in our previous
classes. Do not choose a model that
you have already implemented for your own research (though it could be one you
have not yet implemented and which you may use in the future for your
research). Usually your choice will
be a relatively recent model. However, it could be an older topic/model if it
would be a good learning experience for the class. It could also be a theoretical topic,
model, paradigm, etc. as long as you can find a way to Òimplement itÓ in a
rationale and educational way to demonstrate/validate its effectiveness.

I will give the lectures for the first
4-5 weeks while you prepare to teach your topics. I will also then fill in any remaining
time with topics at the end of the semester.

Your first step is to put together and
e-mail me a ranked annotated list of three possible topics/models you would
like to teach/implement. Include a
link (and full reference) for a suggested paper that could be read by the class
for each potential topic/model. For
each topic/model include about a half page discussion, including why you think
it would be a good topic for this class. This will be due no later than January
12. Each person must present/implement
a different model. If two people
want the same model, the person that sends me the list the soonest will have
priority. Some good sources for
potential topics/models/papers include *Proceedings of the International
Conference on Machine Learning*, *Proceedings of Neural Information
Processing Systems*, *Machine Learning Journal*, *Journal of
Artificial Intelligence Research,* *Journal of Machine Learning Research*,
*Neural Networks*, *IEEE Transactions on Neural Networks*, and* Neural
Computation, *amongst others. Machine learning books and chapters
could also be a good option. Google
search is often a great way to start getting your arms around a topic.

Also include how early in the semester
you would be willing to present if your first choice is accepted. I will then
put together the order of presentation.

You may also attach to that e-mail a
short list of other topics that you would enjoy seeing covered in this
class. I will use this information
to attempt to insure that topics for which there is most interest get covered
during class.

__Teaching Outline__:

You will hand in a well thought out
outline of your teaching at the beginning of class on Jan 26^{th}. Following is an example of a general high level outline. Yours will have different time
allotments and be more detailed than this and may vary based on your
topic/model. I will then give you
feedback and suggestions on your outline.

Topic x, Model
y:

Reading(s): xxx

1.
Introduction of the topic at a general level (5 minutes)

2.
Motivation of why it is useful, etc. (5 minutes)

3.
Brief history of topic, early models, and related work (5
minutes)

4.
Detailed (easy to understand) description of the model you
chose (15 minutes) – Note: It
will often be best to choose the more common ÒvanillaÓ well known model within
the topic for teaching and possibly implementing, rather than one with too many
bells and whistles.

5.
Careful group example walk through of a simple hand made
data set. Lots of class
participation. You are asking us to
fill in what will happen as it learns, and helping us grasp it as we go, rather
than just showing us – (25 minutes)

6.
Discussion of your implementation, including challenges,
complexity, and analysis of actual results (tables, graphs, demos, etc) on multiple real world data sets. (15 minutes)

7.
Discussion of strengths and weaknesses of the model (10
minutes)

8.
Discussion of other models (bells and whistles) that try to
overcome weaknesses of the basic model. You may end up implementing one of
these as your main model implementation, which would change the time allocation
and order somewhat. You may end up
implementing both a basic and an extended models version. (10-30 minutes)

9.
Current directions, needs, and future work in the
topic/model area. (5 minutes)

10. Include a
slide(s) with a bibliography of the papers/sources you studied in learning your
topic.

11. Questions and
discussion (This will mostly happen throughout the overall teaching, but we
will usually also do a little at the end) (5 minutes)

We will make available both your
reading and your slides before your presentation for the class to have
available. You should have one
quality required reading for the class (if short you could have more than
one). You may also have optional
readings. Often the required
reading will be more of a understandable review of the
topic (book chapter, etc.) and the paper on your model will be an optional
reading, as the topic is the most important goal. This will vary depending on your
topic/model.

__Grading__: The largest portion of your grade will
be based on your presentation and implementation of your model. There will also be a midterm on the
topics that I teach during the first part of the class. Participation will also be important
part of the grade as I expect all students to read the assigned papers before
class and come prepared to contribute to our effort of understanding the topic/model. We can all gain insights through helping
each other understand difficult concepts.
I am currently thinking we will not have a final, but if you want one we
can discuss this and you could convince me.

Grading (~):

Topic Teaching/Implementation |
50% |

Participation in Model Discussions |
30% |

Midterm |
20% |

__Topics I
currently plan to cover at the beginning the semester (not necessarily in this
order): __

á
Rule Based Learning (Sequential Covering, CN2)

á
Bias: Interesting/computable problems, Bias-Variance Decomposition

á
Convolutional
Deep Neural Networks Intro

á
Ensembles (Variations, BMC vs BMA, Oracle Learning, etc.)

á
Semi-Supervised Learning

__Other topics
I/we may cover based on time and interest:__

á
Structured Prediction

á
Spiking Neural Networks

á
Manifold Learning/Non-Linear Dimensionality Reduction

á
Record Linkage/Family History Directions

á
Meta-Learning

á
Feature Selection

á
Computational Learning Theory

á
Transfer Learning

á
Transduction

á
Other Unsupervised Learning Models

Project Presentations Schedule: