MATH 494: Mathematical Foundations of Machine Learning

Test #1 Review Topics

March 9, 2024

1. Calculus review: gradient and its properties, extrema of functions of several variables

2. Gradient descent method; use of grid

3. Linear algebra review: matrices, determinants, products, transposes, inverses, norms, symmetric matrices

4. Eigenvalues and eigenvectors of square matrices

5. Discrete and continuous random variables; probability distributions; uniform and normal distributions

6. Expected value, variance, standard deviation; data centering and standardization

7. Pseudo-random number generators; seeding

8. Conditional probability; Bayes theorem

9. Machine learning landscape: types of ML; challenges of ML (curse of dimensionality, insufficient data, poor data, overfitting, underfitting, bias

10. Validation methods

11. Iris and MNIST datasets

12. Single-variable linear regression; assumptions, procedure, coefficient of determination

13. Linearizing non-linear models for linear regression

14. Polynomial regression

15. Multiple linear regression

16. Derivation of the normal equation of linear regression

17. Logistic regression; assumptions, sigmoid function, cross-entropy cost function; algorithm