Efficient algorithms have become increasingly important in the field of scientific computing and machine learning due to the growing size and complexity of problems with higher dimensionality. This thesis focuses on the development of computationally efficient algorithms.

The first part proposes the mean-field score-based transport modeling (MSBTM) algorithm to solve mean-field Fokker-Planck equations. This algorithm provides an efficient and accurate way to solve equations that are computationally challenging using traditional numerical methods. The algorithm’s theoretical bound, on the time derivative of the Kullback-Leibler (KL) divergence between the numerical solution and the exact solution, is obtained. Moreover, an error analysis between the samples from the MSBTM algorithm and t...[ Read more ]

Efficient algorithms have become increasingly important in the field of scientific computing and machine learning due to the growing size and complexity of problems with higher dimensionality. This thesis focuses on the development of computationally efficient algorithms.

The first part proposes the mean-field score-based transport modeling (MSBTM) algorithm to solve mean-field Fokker-Planck equations. This algorithm provides an efficient and accurate way to solve equations that are computationally challenging using traditional numerical methods. The algorithm’s theoretical bound, on the time derivative of the Kullback-Leibler (KL) divergence between the numerical solution and the exact solution, is obtained. Moreover, an error analysis between the samples from the MSBTM algorithm and those of the associated ordinary differentiable equation is provided. Numerical experiments validate the MSBTM algorithm for different types of interacting particle systems.

The second part develops an algorithm for network pruning, which reduces the number of parameters in neural networks while maintaining accuracy. The proposed pruning algorithm improves the structured sparsity in DNNs from a new perspective of evolution of features. In particular, the trajectories connecting features of adjacent hidden layers, namely feature flow, are considered. Our pruning method, feature flow regularization (FFR), penalizes the length and the total absolute curvature of the trajectories, which implicitly increases the structured sparsity of the parameters. The algorithm is shown to be effective for image classification on various datasets, including CIFAR10 and ImageNet.

Finally, the last part uses a modified alternating direction method of multipliers (ADMM) with an increasing penalty parameter to solve a linearly constrained nonconvex optimization problem for grain boundary structure in materials science. The algorithm is shown to converge to the stationary point of the corresponding augmented Lagrangian function. In addition, sufficient conditions of quasi-convexity are identified, and the objective function is shown to be quasi-convex and to have only one minimum over the given domain. Numerical experiments demonstrate the modified ADMM improves the accuracy and efficiency compared to the penalty method and the augmented Lagrangian method.