Delta Rule Simulation Assignment - CS 578

You will have the opportunity to code up a delta rule simulator. It should have the capacity to be set up with an arbitrary number of input and output nodes (remember the bias). You will need to be able to input a training and test set and be able to examine results (tss, accuracy, weights, etc.) For each experiment give a short (~paragraph) discussion of your findings and observations.

Note: For all assignments in this course requiring your observations (which is the typical case) the most important thing is not just mentioning what you observe occurring, but I want your explanation as to why it is occurring. If you can't figure it out, then give your best try at explaining it. This is where learning can best occur, when trying to figure out why the models do what they do.

1. Test a simple 2x1 or 3x1 network (always remember the bias weight) on linearly separable training sets. What is the effect of Learning rate and different initial weight settings.

2. What happens with the simple network for nonlinearly separable training sets.

3. Browse the different files of the Irvine Machine Learning Data Base (MLDB). The MLDB applications can be found from links on the class web page. Create networks for two or more MLDB applications (including analog inputs). Realize that you will not typically get perfect convergence. Test generalization and discuss your results!

4. Be creative. Try some experiment(s) of your own which may reveal some interesting aspects of the delta rule model. This discussion may be longer than those of the previous 4 experiments. For example:

• Test the Dichotomization Capacity by trying random training sets with varying numbers of patterns to see when convergence is attained.

• Observe dynamics of tss, pss, etc.

• Effect of different initial weight settings

• Try Delta Rule on some different MLDB applications.

Discuss why it does well on some but not others.

• Come up with some of your own research results!

Total pages: ~3-4