THESIS
2023
1 online resource (xvi, 126 pages) : illustrations (some color)
Abstract
Efficient algorithms have become increasingly important in the field of scientific
computing and machine learning due to the growing size and complexity of
problems with higher dimensionality. This thesis focuses on the development of
computationally efficient algorithms.
The first part proposes the mean-field score-based transport modeling (MSBTM)
algorithm to solve mean-field Fokker-Planck equations. This algorithm provides
an efficient and accurate way to solve equations that are computationally challenging
using traditional numerical methods. The algorithm’s theoretical bound,
on the time derivative of the Kullback-Leibler (KL) divergence between the numerical
solution and the exact solution, is obtained. Moreover, an error analysis
between the samples from the MSBTM algorithm and t...[
Read more ]
Efficient algorithms have become increasingly important in the field of scientific
computing and machine learning due to the growing size and complexity of
problems with higher dimensionality. This thesis focuses on the development of
computationally efficient algorithms.
The first part proposes the mean-field score-based transport modeling (MSBTM)
algorithm to solve mean-field Fokker-Planck equations. This algorithm provides
an efficient and accurate way to solve equations that are computationally challenging
using traditional numerical methods. The algorithm’s theoretical bound,
on the time derivative of the Kullback-Leibler (KL) divergence between the numerical
solution and the exact solution, is obtained. Moreover, an error analysis
between the samples from the MSBTM algorithm and those of the associated
ordinary differentiable equation is provided. Numerical experiments validate the
MSBTM algorithm for different types of interacting particle systems.
The second part develops an algorithm for network pruning, which reduces the
number of parameters in neural networks while maintaining accuracy. The proposed
pruning algorithm improves the structured sparsity in DNNs from a new
perspective of evolution of features. In particular, the trajectories connecting features of adjacent hidden layers, namely feature flow, are considered. Our
pruning method, feature flow regularization (FFR), penalizes the length and the
total absolute curvature of the trajectories, which implicitly increases the structured
sparsity of the parameters. The algorithm is shown to be effective for
image classification on various datasets, including CIFAR10 and ImageNet.
Finally, the last part uses a modified alternating direction method of multipliers
(ADMM) with an increasing penalty parameter to solve a linearly constrained
nonconvex optimization problem for grain boundary structure in materials science.
The algorithm is shown to converge to the stationary point of the corresponding
augmented Lagrangian function. In addition, sufficient conditions of
quasi-convexity are identified, and the objective function is shown to be quasi-convex
and to have only one minimum over the given domain. Numerical experiments
demonstrate the modified ADMM improves the accuracy and efficiency
compared to the penalty method and the augmented Lagrangian method.
Post a Comment