Topics in Mathematical Modeling

Class Notes

Week 1

1. Tuesday, Jan 18
• Handout: Sections 0.2, 0.3 from T. Sauer's ``Numerical Analysis''. (Available on CLEo for a short time)
• Homework for Friday: Exercise 3(a), 4(a), 5(c), 6. (Changed- See Friday's notes)
2. Thursday, Jan 20: Meet in the Mathematics Computer Lab
• Introduction to the Lab (Log in, change password, etc.)
• Introduction to Matlab
• A summary of useful Matlab commands
• Much more in-depth documentation from Mathworks
• Matlab videos
3. Friday, Jan 21: Homework 1 was changed to: p. 15-16, 3(a), 4(a), 6, 7. DUE: Tuesday
   Today, we will finish talking about floating point numbers, and then on Tuesday we'll pick up with some Matlab.
   Use these instructions to move the window options (maximize, minimize, close) back over to the right side of the window.

Week 2

1. Tuesday, Jan 25: More discussion on Matlab, and the Case Study on Learning. The Matlab code is banditE.m and the script file is banditScript.m. Be sure to right-click the mouse and choose "Save Link As..." to download the Matlab code. Homework 2: Exercises 1-5, pg 2 of the Case Study Handout.
2. Thursday, Jan 27: Worked through the epsilon-greedy algorithm, worked in the computer lab a bit. Homework 3 is here.
3. Friday, Jan 28: Finish the Case Study. Homework 4: is here. Here is the Matlab code discussed:
   • softmax.m
   • banditS.m is the script to run softmax
   • winstay.m
   • banditW.m is the script to run win-stay, lose-shift

Week 3

1. Tuesday, Feb 1: Finished the n-armed bandit model, and began the linear algebra review- The four subspaces. No homework assigned.
2. Thursday, Feb 3: Introduction to linear models, and linear neural nets. Handout is here.
   
3. Friday, Feb 4: Continuing with linear models. Homework 5 Handout is here.
   Matlab stuff:
   • Code: Line1.m
   • Code: Line2.m
   • A Text File with Data: HgtWgt.txt

Week 4

1. Tuesday, Feb 8. Today we started linear neural networks, and discussed unsupervised learning. Here is a copy of the updated notes that were passed out in class. Other links:
   • Homework 6 (due Friday)
2. Thursday, Feb 10. Our focus is on training the linear network. Here are the Matlab files:
   • Matlab function: wid_hoff1.m as on p. 8
   • Script file from page 9: scriptP09.m
   • Script file from page 11: scriptP11.m
   • Script file from page 12: scriptP12.m (New link for Friday)
3. Friday, Feb 11: We will meet in the computer lab and try to get further on understanding the Matlab code. Homework 6 will not be due until Tuesday, Feb 15. Summary page with some homework notes.

Week 5

• Tuesday, Feb 15: Finish up the linear neural networks with a look at gradient descent and an application: Novelty Detection.
  • Matlab function: lag.m
  • Script file from page 17: scriptP17.m
• Thursday, Feb 17: Finish up linear networks, start Appendix A- The derivative and the gradient.
  • Appendix A: Derivative and Gradient
  • Homework 7
  • Matlab data file for HW 7: IrisData.mat
  • Script file for HW 7: scriptIris.m
• Friday, Feb 18: Continue with Appendix A.

Week 6

• Tuesday, Feb 22: Finish Appendix A, work in the lab for a bit.
• Thursday, Feb 24 - Review for Exam 1 (See links below)
• Friday, Feb 25: Exam 1

Week 7

• Tuesday, Mar 01: Handout with more linear algebra (Projections, the SVD, and the Pseudo-Inverse. No homework.
• Thursday, Mar 03: Continue with the handout. Homework Assigned today: p.6, 3, 6, and 7. (Try them all, but only turn in solutions for the three).
• Friday, Mar 04: Continue with the handout. Homework: Compute the SVD for a ``simple'' matrix by hand (1(a), p. 14).

Week 8

• Tuesday, Mar 8: Continue with the SVD. Code for class.
• Thursday, Mar 10. "Movie Data", file is author.mat. Other handouts:
  • How to solve systems using Matlab's slash commands.
  • Homework that is due on Tuesday, March 29, 10PM . Includes Exercise 3 (projection of the iris data) that was originally assigned on Tuesday, March 8th.
• Friday, Mar 11: Meet in the computer lab.

Week 9

• Tuesday, March 29: Homework is due. Start working through the notes: that were passed out last Thursday. Files: edm.m and InterpExamp1.m.
  Homework: p. 175, 1, 2, 3 and p. 178, 1 (not due yet, but work through them).
• Thursday, March 30: Continue with the RBFs. Here is some Matlab code for the homework:
  1. Matlab File: rbf1.m, needed for the transfer functions
  2. Matlab File: RBFSample0.m, the first example
  3. Matlab File: RBFSample1.m, the second example
• Friday, April 1:
  Homework 11:
  1. Write the solution to Exercise 1 (the determinant of the Vandermonde matrix).
  2. Write a Matlab script for classifying the Iris data. Here are the code details:
     1. Use the same data as before (X is 150 x 4, Y is 150 x 3).
     2. Use 10 points from the data (randomly selected) for the "centers".
     3. Use a Gaussian transfer function with a width of 3.
     4. For training, use 80 points chosen at random from the data.
     5. For testing, use all the data and construct the Confusion matrix (See HW 7, for example).

Week 10

• Tuesday, April 5: Script: RBFwidths.m
  Today, we'll be discussing the role of the Gaussian, Matlab structures.
• Thursday, April 7: Continue with Matlab structures and (time permitting) Orthogonal Least Squares.
  Homework 12: (due Fri)
  1. Exercise 2 on page 186.
  2. Compute the RBF by hand (First problem on today's handout weight vector corrected)
• Friday, April 8: No homework announced. We talked about Matlab's "Orthogonal Least Squares" training algorithm.

Week 11

• Tuesday, April 12: No classes (Undergrad. Conference)
• Thursday, April 14: Finish OLS.
  Homework 13: (due Tues)
  1. Finish writing the function "HW13A.m", which implements OLS between matrices X and Y. This is a link to a template you can start with.
  2. RBF model for the sunspot data below.
  3. Sunspot data: Sunspot11.mat
  4. A file to plot the original series, plotspots.m
  5. The original data, yearssn.dat
• Friday, April 15: Meet in the Lab

Week 12

• Tuesday, April 19: Review for Exam 2 (See below for materials)
• Thursday, April 21: Exam 2 (in class portion)
• Friday, April 22: No class. Work on the Take Home Exam.

Week 13

• Tuesday, April 26: Feed Forward Neural Nets. Here is the handout from today. Homework was to look over the exercises on pg. 5- Turn in date to be determined.
• Thursday, April 28: The backpropagation of error. Here is the handout from today which includes Matlab files to manually compute the backpropagation of error.
  Matlab file from the handout
  Homework 14: Last page of today's handout. Due: Tuesday, May 3.
• Friday, April 29: Training a feedforward network.

Week 14

• Tuesday, May 3: Training issues, part I.
• Thursday, May 5: Meet in the Lab. Handout for today, and the dataset diabetes1.mat/
• Friday, May 6: Paperwork Day. Finish up the course materials, do course evaluations.
  Matlab file (alphachars.mat) containing the alphabet as a 35 x 26 matrix.
  "Homework 15" (Not due, but look it over and try to work it out on your own.
  Solution (M-file) Try it out!

Week 15

• Tuesday, May 10: Discuss the final exam (take home)
• Wednesday, May 11: Reading Day (Math Picnic!)
• Take home exam due on Tuesday, May 17, 6PM.

Exam 1 Links

• Review questions
• Solutions to the review questions
• Homework solutions
  • Homework Set 1
  • Homework Set 2
  • Homework Set 3
  • Homework Set 4
  • Homework Set 5
  • Homework Set 6
  • Homework Set 7
• Take home portion of the exam. Data for the exam:
  • Exam01Prob01.mat (right-click the link and "Save Link As..."
  • FaceClass01.mat (right-click, etc).

Exam 2 Links

• Review questions
• Review solutions
• Homework Solutions
  • Homework Set 8
  • Homework Set 9
  • Homework Set 10
  • Homework Set 11
  • Homework Set 12
  • Homework Set 13: There are three Matlab files- A script file to test your function, the function file HW13Asol.m, and for the RBF file: HW13Bsol2.m.
• Exam 2 (Take home portion):
  • The take home exam.
  • The data file, MenAndWomen2.mat
  • A template script file for the first question.
  • A template script file for the second question.
  NOTE: You do not have to use the template files- You might try to code the solutions yourself before looking at my suggestions!

Final Exam Links

• Final Exam
• Data for the first problem, Ax=b
• Data for the linear classification
• Mushroom classification data