MATH 494: Mathematical Foundations of Machine Learning

Assignment #9

(Due Friday, April 26, 2024)

In this assignment you will use a skeleton of a simple neural network (will be emailed to you) with a single layer of hidden nodes to produce a very basic working NN that is able to learn the following task. Four input nodes take the 1/0 values (bits) and the output node should return a value that indicates the number of ones. The number of hidden nodes is up to you (note that often fewer is better). With complete input, this is not a difficult task, so the network should easily learn. The interesting part will be omitting some inputs in training, yet testing the network on them to see its generalization ability.

When the basic training is over, the script should display the average relative error between the targets and the actual results over all 16 patterns. Next, omit one pattern, train the network, and display the testing results for the missing pattern. Then, try to omit a second pattern, and display the results for both omitted patterns.

The program should display all the parameters that you used: the learning rate, the number of hidden nodes, and seed of the PRNG, if you use seeding. When the script runs with missing pattern(s), please display which patterns are missing. Make the display informative!

Technical hint: it is a standard that the values on the output node are in the range (0, 1), so you need to scale the results to get values in the range from 0 to 4.