Numerical Optimization

1. Convex vs. Nonconvex

2. Differentiable vs. Nondifferentiable

3. Constrained vs. Unconstrained

4. Line search

5. Gradient descent

6. Hessian matrix, etc.

7. Stochastic optimization