AMath 482/582
Computational Methods for Data Analysis

Winter 2014
http://www.atmos.washington.edu/~breth/classes/AM582/

MWF 8:30-9:20: Loew 216

This class is being offered on-line through EDGE, which provides live streaming of each class and archived lecture videos. You'll need to sign in with your UW NetID. If you are an EDGE student, you can also get to this site through the UWEO Moodle portal moodle.extn.washington.edu. If you are registered for the virtual section 582C, it is identical to being in the on-campus 'A' section, and you are welcome to come to the classroom to listen to the lectures live (we will have enough seats, at least after the first lecture or two). Assignments and grading for 482 sections will be identical to that for 582 sections.

A Canvas page will be used for announcements, discussions, homework submission and grading.

 
Instructor:
Prof. Chris Bretherton
breth@washington.edu
ATG 704, x5-7414
Office hours: MF 1:30-2:20, or by appointment.


TAs:
Xing Fu xingf@uw.edu
Meghana Velegar mvelegar@uw.edu
Office hours: Tu 12:30-1:30 (LEW 128) and Fr 12:30-1:30 (LEW 129); also via Canvas teleconferencing.


Schedule   Homework and Exams   Syllabus and supplementary notes     Matlab scripts  

Course Description

Exploratory and objective data analysis methods applied to the physical, engineering, and biological sciences. Statistics, including normal distributions, confidence intervals, linear regression. Fourier spectral analysis and filtering for time series, wavelet analysis, image processing and compression, principal component analysis, cluster analysis, Kalman filter.

Prerequisites

Prior experience with Matlab, linear algebra and Fourier analysis, as provided by Amath 301 and Amath 352 or Math 308 (or Amath 481/581 and/or Amath 584).

References

Useful references to supplement lecture notes:

Syllabus and lecture notes

Statistics potpourri (6 lectures)

Fourier spectral analysis and filtering (9 lectures)

Wavelet analysis and image compression (3 lectures)

Principal component and cluster analysis for dimensionality reduction (6 lectures)

Data assimilation and model-data fusion (3 lectures)

Grading

Schedule

No class:

Makeup classes for instructor travel days (please attend if you want to watch live; otherwise watch the on-line video before the following class)

Homework and Exams

Item Due Date Download Solutions
Homework #1; uses hw1_dat.mat due We 22 Jan HW #1 solution
Homework #2; uses snow.mat due Fr 31 Jan HW #2 solution
Homework #3; uses SP500.dat due Fr. 7 Feb HW #3 solution
Homework #4 (Midterm assignment; no collaboration) due Fr. 14 Feb HW #4 solution
Homework #5; uses raymo.mat due Fr. 21 Feb HW #5 solution
Homework #6; uses person.jpg due Fr. 28 Feb HW #6 solution
Homework #7; uses USTA.mat due Mo 10 Mar HW #7 solution
Final assignment; uses x_y_train.png and x_y_test.png. If you get stuck on problem 1, two 4D arrays containing 8x8 rescaled images for each of the 8x5 letters in the training and testing datasets that you can use for problem 2 are given in ScaledLetter.mat. due We 19 Mar 5 pm, no collaboration, no late submissions Final solutions

Matlab Scripts for Class Examples

For the scripts below with a .html link, I've used Matlab's publish capability to make self-documenting web page (html) versions of the scripts. To extract the original Matlab script from the web page, copy the URL (web address). In Matlab, type grabcode('URL') to bring up an untitled file with the code in an editor window, e. g. to get the file foo.m that generated the web page foo.html below, type grabcode('http://www.atmos.washington.edu/~breth/classes/AM582/matlab/html/foo.html') and save this to foo.m in the Matlab editor.

regression_example.html: Simple and multiple linear regression on a dataset of car MPG vs. weight and horsepower.

fft_hw1.m: DFT of HW1 dataset. Makes plot of the amplitudes of the complex-valued DFT components.

DFT_gauss.html: Power spectra of a Gaussian and a half-Gaussian.

nino1.html: Uses 1950-2012 monthly Nino3.4 sea-surface temperature dataset nino.mat. Plots the SST, its power spectrum, and the same for the SST anomaly after the mean and first three harmonics of the annual cycle of SST are removed.

nino2.html: Uses 1950-2012 monthly Nino3.4 sea-surface temperature anomaly dataset SSTA.mat optionally made by nino1.m. Plots subannual SSTA power spectrum with red noise fit, and plots of its autocovariance and autocorrelation sequences.

rednoise.m: Function to generate a sample of standardized Gaussian red noise with a given lag-1 covariance r.

nino3.html: Uses monthly Nino3.4 sea-surface temperature anomaly (SSTA) time series derived by nino1.m, given in SSTA.mat. Plots windowed power spectrum of SSTA, using 20-year overlapping Hann (cosine-taper) windows, both directly and using Matlab signal-processing toolbox function pwelch, and compares to red noise fit.

music.html. Plays and does windowed tapered power spectral analysis of short segment of Handel Messiah. A copy of the score suggests that the initial notes of the segment are D, A and F, as also visible from our analysis (which also suggests they are played slightly flat).

music2.html. Low, band and high pass filtering of short segment of Handel Messiah with Fourier and Butterworth filters.

runningmean.html. Power spectrum of running mean filter using DFT.

butterworth.html. Calculation of an Nth order Butterworth filter and plots showing its properties.

wavelet_leleccum_notoolbox.html. Application of single and multilevel Haar wavelet transform to an electricity consumption dataset. Uses leleccum.mat (included in Matlab wavelet toolbox) and my functions dwtHaar.m and idwtHaar.m in place of the Matlab wavelet toolbox, for portability. See wavelet_leleccum.html for an equivalent script from the wavelet toolbox documentation that uses the toolbox functions instead.

wavelet_leleccum_cwt.html. Continuous Haar wavelet transform on the electricity consumption dataset. Requires Matlab wavelet toolbox function cwt.

wavelet_image.html. Image compression example using 2D multilevel Haar wavelet transform. Uses ngc6543a.jpg and my functions dwt2Haar.m and idwt2Haar.m in place of the Matlab wavelet toolbox, for portability.

PCA_SSTA.html. Application of PCA to gridded tropical Pacific sea-surface temperature dataset. Uses SSTPac.mat.

PCA_cities.html. Application of PCA to multiparameter dataset of indices for 9 categories contributing to quality of life in 329 U.S. cities. Uses cities.mat, which is also already included in the Matlab Statistics toolbox.

cluster_cities.html. K-means cluster analysis of cities dataset. Uses cities.mat.

classify_one_two.html. Classify spoken 'one's and 'two's using DWT power spectral analysis and PCA. Uses dwtcolHaar.m, ones.m4a, twos.m4a, and ones-twos.m4a. If your Matlab doesn't have audioread or equivalent, load the following .mat files instead: ones.mat, twos.mat, ones-twos.mat ; these also include the 20 sounds being spoken in each file.

sequential_estimation_simple1D.html. Sequential state estimation (Kalman filter) on the simple 1D system x_n = ax_{n-1}.

kalman2.html. Kalman filter on a multivariate ball-tracking problem.