Aims and Fit of Module

This module introduces techniques for formulating data science models as optimization problems, algorithms for solving data science problems with an emphasis on scalability, efficiency and parallelizability including gradient-descent based algorithms, derivative-free algorithms and randomized algorithms, theoretical analyses of algorithms, and approaches for recognizing the most suitable algorithm for solving a particular problem.

Learning outcomes

A. Demonstrate a deep understanding of the basic concepts, such as least squares, support vector machines, logistic regression, gradient, subgradient, convex function, momentum, coordinate descent and duality.
B. Analyze and implement classical smooth optimization, descent methods and gradient methods.
C. Analyze and implement stochastic gradient methods, coordinate descent methods, constrained optimization, and nonsmooth optimization.
D. Identify and implement optimization methods to solve data science related problems using software.

Method of teaching and learning

This module is delivered through a combination of lectures and tutorials over a period of 13 weeks. In lectures, students are introduced to the core principles, major methodology, and common topics and issues in the area of optimization. Tutorials are given as a platform to address any specific question or issue from individual students. Part of tutorials also are given as a platform to solve practical problem by using software.