Connectionist Assignment

Adapted from Rich & Knight

Here is a toy problem for testing generalization in networks. Suppose that there are twenty political issues on which every political party must decide, and suppose further that those decisions are binary (for example, to legalize gambling or not, to increase military spending or not, etc.). We can then represent the platform of a political party as a vector of twenty ones and zeros. Individuals who belong to political parties may have beliefs that differ slightly from their party's platform. Your job is to train a backpropagation network to compute the political platform of the party that most closely matches a given individual's beliefs.

Generate four random 20-bit vectors to represent the platforms of four political parties. For each party, generate twenty individuals who belong to that party. The beliefs of an individual, like those of a party, are represented as a 20-bit vector. Each individual has a 50% chance of not agreeing with the party on each issue. Now generate 80 input- output pairs, by juxtaposing individuals with the platforms of their respective political parties. Each input is 20 bits, and each output is 20 bits.  (This data are already generated and are in the .mat files.)

Next, remove ten of the input-output pairs from each party. These forty will make up the "testing set"; the other forty will make up the "training set." Create a backpropagation network (pp. 504-506 and Fig. 18.14) from the skeleton code backprop.m with twenty input units, eight hidden units, and twenty output units. Train the network on the 40 vectors in the training set for 500 epochs. On each epoch also test the network on the input-output pairs it has never seen before.

Turn in the graph showing sse for the training and testing set for each epoch.  What is the general sse pattern for the training and testing sets? Why?