Math 350, Fall 2014

This course explores the process of building, analyzing, and interpreting mathematical descriptions of physical processes. Topics may include feature extraction, partial differential equations, neural networks, statistical models. The course will involve some computer programming, so previous programming experience is helpful, but not required.

There is no required text for this course. Class notes will be provided.

- Course Syllabus
- Draft List of Topics
- Introduction to Matlab (Updated Sep 8)
- Summary of Matlab Commands

- Introduction and Chapter 2
- Chapter 3 (Stats and Linear Regression)
- Chapter 4 (Genetic Algorithms)
- Chapter 5 (Linear Algebra Review, updated Nov 3)
- Chapter 6 Redone
- Optimization
- Chapter 9 (Data Clustering)
- Chapter 10 (Linear Networks)
- Chapter 11 (Radial Basis Functions) (Plus, the last couple of pages from Ch 10).

- Week 1
- Tue, Sep 2
- Reading from "A First Course in Chaotic Dynamical Systems", Chapters 2-5. See the "Resources" tab in CLEo.
- Homework: See Week 1 HW below.

- Thu, Sep 4: Discrete Systems, continued.
- Homework for Week 1. Due: 4, 5 from Tuesday, and 1, 4 from Thursday. These are due on Tuesday, Sep 9.

- Tue, Sep 2
- Week 2
- Mon, Sep 8
- Introduction to Matlab
- First set of text notes
- HW: Familiarize yourself with Matlab.

- Tue, Sep 9
- Addition to "Intro to Matlab" from last time
- Matlab function file, Chapter 3: banditE.m
- Matlab script file, Chapter 3: banditScript01.m
- Homework: Run the bandit script. Do Exercises 1-5 in the new "Addition to Intro to Matlab" passed out today.

- Thu, Sep 11: Finish the reading for the n-armed bandit. Be sure you can run the previous scripts. Finish up Exercises 1-5 assigned last time (Due on Monday).

- Mon, Sep 8

- Mon, Sep 15
- Tue, Sep 16: Genetic Algorithms.
- Homework updated to include today!
- Function RandChooseN.m This function is used to choose items with certain probabilities. We need it for the next script file.
- Script BinStrings.m This is the script file in the text that implements a GA on a population of binary strings of length 20.

- Thu, Sep 18: Finish Genetic Algorithms. Start
Optimization if time.
- PDF file of BinStrings.m
- Function testfunction.m for the script file below (Hills.m).
- Script Hills.m Gives a script for real valued strings.
- Picture of the surface, Hills.m

- Monday, Sep 22 (Today we finished GA's).
- Homework: Solve the Knapsack problem. Here is the fitness
function, called testfunction2.m:

testfunction2.m (Fitness function for the Knapsack problem).

SOLUTION to Knapsack problem (ExerciseKnap.m) - Tuesday, Sep 23: Start Optimization.
- Thursday, Sep 25: Finished the previous notes.

HOMEWORK DUE ON MONDAY: The knapsack problem, in Matlab.

HOMEWORK NOT DUE YET: Work on implementing Bisection and Newton in Matlab

EXAM NOTE: Exam 1 is scheduled for next week- By unanimous proclamation, we moved it to THURSDAY, OCT 2.

- Monday, Sep 29: Finish up the optimization.

Optimization notes, updated to include Gradient Descent

Solutions to Optimization Exercises

Matlab file: bisect.m

Matlab file: MultiNewton.m

Matlab file: Ex4.m (Solution to Exercise 4) - Tuesday, Sep 30: Review/Catch up day.

Exam 1 Review Sheet

Exam 1 Review Solutions - Thursday, Oct 2: EXAM 1.

- Mon, Oct 6 Start on Data Clustering (Updated Oct 8). Homework: 1, 2, 3, 4, 6. Turn in next Monday: 2, 6.
- Tue, Oct 7: K-Means Clustering
- Matlab Function: edm.m (This is the function that computes the Euclidean Distance Matrix between all the data and the cluster centers).
- Matlab Function: kmeansUpdate.m (This performs one step of the K-means algorithm).
- Matlab script: ClusterExample01.m (Gives an example of how to use the k-means code).
- Matlab script: ClusterExample02.m (Same as previous script, except shows the Voronoi cells
- HOMEWORK: Download the m-files, try to publish ClusterExample01.m.
- Thu, Oct 9: Neural Gas
- Matlab Function: paramUpdate.m
- Matlab Function: initng.m
- Matlab Function: NeuralGas01.m
- Matlab Data: SixDatasets.mat
- Matlab Data: obstacle1.mat
- For fun: Video showing "Growing Neural Gas"
- Training Examples on Concentric Circles:
- TestNG1: Good parameter selection.
- TestNG2: Bad in two respects- The value of lambda is too big, as is the learning rate. This shoves all the centers towards the center of mass. Secondly, the age cutoff is too old- We're retaining "bad" edges.
- TestNG3: Similar to TestNG2. Cut off age is still too old.
- TestNG4: The cut off age is too "young"- We're not retaining connections long enough, and they're being deleted too early.
- The learning rate is too small. Notice that the centers are really not moving at all from where they were initialized.

- Thu, Oct 16: Download HW6scriptA.m and tell me what each of
the lines of code does. Additionally, comment on what you see. Does
k-means depend on the initial centers? (Also, be sure to continue
workingg on the Neural Gas script from last week), the project on
the obstacle course. Some extra scripts (Added Mon, Oct 20):
- Matlab Script: NGScript01.m (Script file showing you how to use the Neural Gas)
- Matlab Script: NGScript02.m (Almost the same as before, except with obstacle data)
- Matlab Script: GrowingNeuralGas.m (Fun to watch)
- Matlab function plotng01.m for some visualization of Neural Gas.
- Matlab script plotNG.m for Growing Neural Gas visualization.

- Mon, Oct 20: Finish Neural Gas, start Linear Networks

Addition to Course Notes: Linear Networks - Tue, Oct 21: Some stats and linear regression.

Addition to Course Notes (Chapter 3): Statistics and Linear Regression

Homework Assigned Week 8 (some Neural Gas, some about Line of Best Fit). Due on Monday, Oct 27. You can upload the Matlab files to CLEO- Make an "Oct27" file folder, please! - Thu, Oct 23: Worked on Linear Neural Networks up through defining the Widrow-Hoff learning algorithm. No new homework assigned.

- Mon, Oct 27

Homework: Go through the 10.4.1: Derivation of Widrow-Hoff (5 exercises). - Tue, Oct 28

Today we finished the linear neural nets and the discussion on covariance.

- Homework and Matlab files:
- Summary of Homework, Week 9
- Data and Matlab files for the iris data classification example.
- Matlab M-file: BreastData.m. This is a *script file* containing the data from the homework, so once you download it, you can open it in Matlab's editor and look at it.

- Thu, Oct 30: Finished the discussion of the covariance and stats, started looking at the linear algebra notes. Nothing new for the HW (See Week 9 homework from Tuesday).

Breast Cancer Classifier 1

Breast Cancer Classifier 2

- Mon, Nov 03: Went through some linear algebra review. using updated notes here. Homework: #1, 4, 5 on pg. 55.
- Tue, Nov 04: More linear algebra. Looked at projections and subspaces, high and low dimensional representations.
- Thu, Nov 06: Finished up the linear algebra.

Week 10 HW Summary

Week 10 HW Solutions

- Mon, Nov 10: Eigenvalues.
- Tue, Nov 11: Review/Catch up.
- Thu, Nov 13:
**Exam 2**.

- Mon, Nov 17: Finished the Spectral Theorem and the SVD. No extra HW assigned (Take home exam due tonight).
- Tue, Nov 18: Finished the SVD, started Karhunen-Loeve Expansions.
- SVD Example 1: "SVDMandrill.m"
- Matlab output of previous file
- Homework: Do the SVD by hand for the matrix on the right, item 20, p. 63 (we did the SVD for the left matrix in class). Also, Exercises 1 and 2 on p. 65.

- Thu, Nov 20: Begin the "Best Basis" notes. Here are the new set of notes (NOTE: These
are now outdated notes, but have been kept here in case you want to
refer to them. The new notes are given below in Week 13.)

HW: The numbered items on pg. 71-72, 73.

- Mon, Dec 1 HOMEWORK due on THURSDAY: Exercise 2, p. 65 of the SVD notes and Exercises 1-3 in the "Best Basis" notes. You should run the "author" example before Thursday as well.
- SOLUTIONS to the HW
- Tues, Dec 2: Continue with Monday's topics.
- Thur, Dec 4: Finish the best basis, start Radial Basis Functions.

- Mon, Dec 8: Radial Basis Functions
- Tue, Dec 9: Orthogonal Least Squares outline
- Thu, Dec 11: Last Day of Class- Finish discussion of Eigenfaces, Finish up RBFs, discuss the final exam.

- Exam 1 (Thu, Oct 2)
- Exam 2 (Thu, Nov 13)
- Exam 2 Summary
- Exam 2 Review Questions
- Exam 2 Review Solutions (Updated for typos in 9, 6 and 23)