CS 778R – Readings in Machine Learning - Syllabus

Winter 2016, TuTh 9:30-10:45am, 134 TMCB

Professor: Tony Martinez, 3336 TMCB, x6464, http://axon.cs.byu.edu/~martinez

martinez at cs dot byu dot edu

Office Hours: by appointment

Course Website: http://axon.cs.byu.edu/~martinez/classes/778/

Goals: Advanced study and discussion of relevant and recent machine learning research

Prerequisites: CS 478 (Introduction to Neural Networks and Machine Learning), CS 678 (Topics in ML) recommended

Text: Mostly papers which will be made available from this website. You are expected to read the assigned literature before the scheduled lecture. We have no specific topics that we must cover. We will try to cover topics of interest to those attending the class.

Teaching/Implementation Assignment: You will each teach one part of the course on a topic/model of special interest to you, and also of interest to the rest of the class. You will take about 2 lectures (1 week) to do this (minimum 1 lecture, maximum two). You will provide one paper/reading on the model/topic that all the class will read before the discussion. Your teaching should teach and motivate the overall topic and then go into detail on a particular model within the topic. Include active participation from the class. Your teaching should draw on information beyond just the one paper given to the class. This is not just a chance to present a paper but to teach a topic as if you were a professor in your own class. You may pattern it somewhat after the lectures you have seen me give. First we motivate the area (e.g. Deep learning), introducing and motivating the high level approach, including some history and other models used to solve it. Then we go into depth on a particular model (e.g. Stacked Auto-Encoders) to the extent that we are capable of walking through a full example on the board or implementing the model, if needed. It is best if we can understand it at a sufficient level of detail that we can walk through how the model would work for specific data examples. Discussion should also include pros, cons, latest current work from others, and suggested directions for improving the chosen topic/model. Thus you should, at least at a high level, be aware of, and briefly discuss with us other approaches/models that are common for your topic. Note that you basically get to become a “semi-expert” in a particular area and then teach it to us.

Power point slides are usually helpful. You will also implement and test the model you teach before you teach it and show results during your lectures on at least two data sets to help us better understand the capacity of the model. Implementing it will help you to understand it a lot better, and thus teach it better. You could include demos or other pedagogical aids in your teaching to help us gain insights. Note that the focus should be on us understanding the topic, with the particular model being an in-depth tool to help us gain deeper understanding of the topic. To clarify, here are a few examples of topic/model pairs you may have learned from me in previous classes.

Topic	Model
Decision Trees	ID3
Clustering	K-means, HAC
Deep Learning	DBN, SAE
Recurrent Networks	BPTT
Rule Based Learning	CN2

The model you present/implement should be a model that we have not previously covered in 478 or 678. However, you may choose extensions/variations on models previously covered, if the variations have sufficient educational “value-added” beyond what we have covered before. For example, you might choose to teach about a new variation of deep learning that we did not cover in our previous classes. Do not choose a model that you have already implemented for your own research (though it could be one you have not yet implemented and which you may use in the future for your research). Usually your choice will be a relatively recent model. However, it could be an older topic/model if it would be a good learning experience for the class. It could also be a theoretical topic, model, paradigm, etc. as long as you can find a way to “implement it” in a rationale and educational way to demonstrate/validate its effectiveness.

I will give the lectures for the first 4-5 weeks while you prepare to teach your topics. I will also then fill in any remaining time with topics at the end of the semester.

Your first step is to put together and e-mail me a ranked annotated list of three possible topics/models you would like to teach/implement. Include a link (and full reference) for a suggested paper that could be read by the class for each potential topic/model. For each topic/model include about a half page discussion, including why you think it would be a good topic for this class. This will be due no later than January 12. Each person must present/implement a different model. If two people want the same model, the person that sends me the list the soonest will have priority. Some good sources for potential topics/models/papers include Proceedings of the International Conference on Machine Learning, Proceedings of Neural Information Processing Systems, Machine Learning Journal, Journal of Artificial Intelligence Research, Journal of Machine Learning Research, Neural Networks, IEEE Transactions on Neural Networks, and Neural Computation, amongst others. Machine learning books and chapters could also be a good option. Google search is often a great way to start getting your arms around a topic.

Also include how early in the semester you would be willing to present if your first choice is accepted. I will then put together the order of presentation.

You may also attach to that e-mail a short list of other topics that you would enjoy seeing covered in this class. I will use this information to attempt to insure that topics for which there is most interest get covered during class.

Teaching Outline:

You will hand in a well thought out outline of your teaching at the beginning of class on Jan 26^th. Following is an example of a general high level outline. Yours will have different time allotments and be more detailed than this and may vary based on your topic/model. I will then give you feedback and suggestions on your outline.

Topic x, Model y:

Reading(s): xxx

1. Introduction of the topic at a general level (5 minutes)

2. Motivation of why it is useful, etc. (5 minutes)

3. Brief history of topic, early models, and related work (5 minutes)

4. Detailed (easy to understand) description of the model you chose (15 minutes) – Note: It will often be best to choose the more common “vanilla” well known model within the topic for teaching and possibly implementing, rather than one with too many bells and whistles.

5. Careful group example walk through of a simple hand made data set. Lots of class participation. You are asking us to fill in what will happen as it learns, and helping us grasp it as we go, rather than just showing us – (25 minutes)

6. Discussion of your implementation, including challenges, complexity, and analysis of actual results (tables, graphs, demos, etc) on multiple real world data sets. (15 minutes)

7. Discussion of strengths and weaknesses of the model (10 minutes)

8. Discussion of other models (bells and whistles) that try to overcome weaknesses of the basic model. You may end up implementing one of these as your main model implementation, which would change the time allocation and order somewhat. You may end up implementing both a basic and an extended models version. (10-30 minutes)

9. Current directions, needs, and future work in the topic/model area. (5 minutes)

10. Include a slide(s) with a bibliography of the papers/sources you studied in learning your topic.

11. Questions and discussion (This will mostly happen throughout the overall teaching, but we will usually also do a little at the end) (5 minutes)

We will make available both your reading and your slides before your presentation for the class to have available. You should have one quality required reading for the class (if short you could have more than one). You may also have optional readings. Often the required reading will be more of a understandable review of the topic (book chapter, etc.) and the paper on your model will be an optional reading, as the topic is the most important goal. This will vary depending on your topic/model.

Grading: The largest portion of your grade will be based on your presentation and implementation of your model. There will also be a midterm on the topics that I teach during the first part of the class. Participation will also be important part of the grade as I expect all students to read the assigned papers before class and come prepared to contribute to our effort of understanding the topic/model. We can all gain insights through helping each other understand difficult concepts. I am currently thinking we will not have a final, but if you want one we can discuss this and you could convince me.

Grading (~):

Topic Teaching/Implementation	50%
Participation in Model Discussions	30%
Midterm	20%

Topics I currently plan to cover at the beginning the semester (not necessarily in this order):

· Rule Based Learning (Sequential Covering, CN2)

· Bias: Interesting/computable problems, Bias-Variance Decomposition

· Convolutional Deep Neural Networks Intro

· Ensembles (Variations, BMC vs BMA, Oracle Learning, etc.)

· Semi-Supervised Learning

Other topics I/we may cover based on time and interest:

· Structured Prediction

· Spiking Neural Networks

· Manifold Learning/Non-Linear Dimensionality Reduction

· Record Linkage/Family History Directions

· Meta-Learning

· Feature Selection

· Computational Learning Theory

· Transfer Learning

· Transduction

· Other Unsupervised Learning Models

Topics and Readings Schedule:

Project Presentations Schedule: