CS 678 - Advanced Machine Learning and Neural Networks
Assignments – Winter 2013
NOTE: All written assignments are to be done with a word processor and be neat and professional. Good writing, grammar, punctuation, organization, etc. are important and will affect your grade.
Do a write-up of approximately ½ single spaced page each on three papers of your choosing in the area of machine learning which you will hand in at the beginning of class on the due date. Include the full reference for each paper. Also e-mail me PDF files for each paper. These papers should be ones you have not read before the semester (and areas we will not directly cover this semester) and the papers should cover three different areas in the field. This is a chance for us to explore. The majority of the papers should focus on innovative technical aspects of machine learning and not just applications of machine learning. As much as possible, choose papers which are relatively recent. In your half page, you should a) briefly describe the main approach and contribution of the paper and b) give your opinion on the specific strengths, weaknesses, and contributions of the approach. Grading will be based on the perceived effort and insights you have gained.
You will also give me a priority order on the three papers regarding which you would most like to present. I will let you know which paper you will present and will most likely have you present the highest priority paper, but may choose another if more appropriate. You will give a 15 minute presentation on the paper to the class including an explanation of the approach or concept proposed in the paper and also the most interesting aspects and insights of the paper. The goal is to educate the audience on the most interesting aspects of the research in the limited time you have. Be prepared to answer questions. We will do one presentation at the beginning of each class starting early February. The quality of the presentation will also be part of the grading. You are encouraged to use slides or any other appropriate teaching aids. I especially like when in addition to reviewing the main points of the paper, you also go beyond and add some or your own insights into interesting aspects of the approach.
Goals of this assignment include a) an opportunity to better learn how to read and analyze research papers, b) an opportunity to practice your oral presentation skills, and c) an opportunity through the presentations for all of us to gain new and diverse insights into machine learning areas.
Some good sources for potential papers include Proceedings of the International Conference on Machine Learning, Proceedings of Neural Information Processing Systems, Machine Learning, Journal of Artificial Intelligence Research, Journal of Machine Learning Research, Neural Networks, IEEE Transactions on Neural Networks, Neural Computation, among others.
All projects are due as a hardcopy at the beginning of class on the due date. Do not attach your source code. The write-up should be about 4-5 pages.
Recurrent Neural Networks Project - BPTT
Due: Feb. 7
1) Implement BPTT on the platform of your choice.
2) Test it on the Person Activity data set for different values of k. Explain your exact network structure (i.e. a) # of input, output, and hidden nodes and b) exactly how you represented the inputs and outputs). Analyze and discuss your results with appropriate graphs and tables. Note the you should only use the Tag ID and the corresponding x, y, z coordinates as inputs. Why? Also note that the events are not completely regular. But the net should learn anyways. Why?
3) Test BPTT on another time-series/recurrent task of your choice. Analyze and discuss your results.
4) Do a creative experiment with BPTT and discuss your findings.
Deep Learning Project
Due: March 28
1) You will solve the MNIST task using deep learning. We have the data in ARFF format here. There are lots of papers regarding MNIST which you can review if you would like. For consistency and comparison with each other, we will just use the given MNIST input features without initial pre-processing, though you are welcome to also try some pre-processing of the features to see what kind of results that allows. We will just use their given training set and test set for training, testing, and comparison.
2) Train on a standard single layer MLP with 200 hidden nodes and report test results as a baseline to compare with.
3) Implement an initial unsupervised deep network (Deep belief net or stacked sparse autoencoders) for preprocessing. Use those features as an input to an MLP. (Optionally you can also try to refine the weights of the entire network with Backpropagation after your initial learning). Test the MLP with 200 hidden nodes, but you should also try other variations, based on the number of features you have, etc.). Try some variations on your deep network and MLP to get the best accuracy possible.
4) Analyze and discuss your results.
Model of your choice Project and Presentation
Due: Hardcopy due at beginning of class on the day you give your oral presentation of the model
1) Implement an ML model of your choice. This should be a model not previously implemented for another class. You may choose one of the models discussed in the class, or one that you have an interest in. I will have you propose a ranked list of 3 possible models and give this list to me halfway through the semester. Then I will give you an OK to proceed forward. This will allow me to give you feedback on your choice, and allow some more diversity on the models we consider.
2) Test the model on at least 2 different data sets and for a reasonable spread of the learning algorithm parameters. Analyze and discuss your results.
3) Do a creative experiment with the model and discuss your findings.
4) In addition to your written report, you will prepare a slide presentation to be given to the class at the end of the semester. The talk should motivate your model, review how it works, discuss results on your tasks, and give your overall analysis of the model.