THESIS
2017
xxii, 150 pages : illustrations ; 30 cm
Abstract
While perception tasks such as visual object recognition and text understanding play
an important role in human intelligence, the subsequent tasks that involve inference,
reasoning, and planning require an even higher level of intelligence. The past few
years have seen major advances in many perception tasks using deep learning models.
In terms of higher-level inference, however, probabilistic graphical models (PGM),
with their ability to describe properties of variables and various probabilistic relations
among variables, are still more powerful and flexible.
To achieve integrated intelligence that involves both perception and inference,
we have been exploring a research direction we call Bayesian deep learning (BDL).
BDL tightly integrates deep learning and Bayesian models (e...[
Read more ]
While perception tasks such as visual object recognition and text understanding play
an important role in human intelligence, the subsequent tasks that involve inference,
reasoning, and planning require an even higher level of intelligence. The past few
years have seen major advances in many perception tasks using deep learning models.
In terms of higher-level inference, however, probabilistic graphical models (PGM),
with their ability to describe properties of variables and various probabilistic relations
among variables, are still more powerful and flexible.
To achieve integrated intelligence that involves both perception and inference,
we have been exploring a research direction we call Bayesian deep learning (BDL).
BDL tightly integrates deep learning and Bayesian models (e.g., PGM) within a
principled probabilistic framework. The aim of this thesis is to advance the fields
of both deep learning and Bayesian learning by demonstrating BDL's power and
flexibility in different real-world problems such as recommender systems and social
network analysis. Its main contributions are as follows.
First, we propose a general framework, BDL, to combine the power of deep
learning and PGM in a principled way to get the best of both worlds. Specifically,
PGM formulations of deep learning models are first designed and then incorporated
into the main PGM, after which joint learning is performed for the unified models.
Second, we devise several concrete models under the BDL framework: Collaborative
Deep Learning (CDL) for recommender systems, Collaborative Recurrent
Autoencoders (CRAE) for joint sequence generation and recommendation, Relational
Stacked Denoising Autoencoders (RSDAE) for relational representation learning,
and Relational Deep Learning (RDL) for link prediction and social network analysis.
Third, we propose Natural-Parameter Networks (NPN) as a backpropagation-compatible
and sampling-free Bayesian treatment for deep neural networks. Such a
treatment can then be naturally incorporated into BDL models to facilitate efficient
Bayesian learning of parameters.
Post a Comment