Keyphrase generation aims to produce a set of phrases summarizing the essentials of a given document. Conventional methods apply an encoder-decoder architecture to generate the output keyphrases for an input document, and they are designed to focus on the current document, so they inevitably omit crucial global contexts carried by other relevant documents, e.g., the cross-document dependency and latent topics.

In this thesis, we firstly focus on scientific documents and propose CDKGEN, a Transformer-based keyphrase generator. CDKGEN expands the Transformer to global attention with cross-document attention networks to incorporate available documents as references so as to generate better keyphrases with the guidance of topic information. In addition to the scientific domain, we verify th...[ Read more ]

Keyphrase generation aims to produce a set of phrases summarizing the essentials of a given document. Conventional methods apply an encoder-decoder architecture to generate the output keyphrases for an input document, and they are designed to focus on the current document, so they inevitably omit crucial global contexts carried by other relevant documents, e.g., the cross-document dependency and latent topics.

In this thesis, we firstly focus on scientific documents and propose CDKGEN, a Transformer-based keyphrase generator. CDKGEN expands the Transformer to global attention with cross-document attention networks to incorporate available documents as references so as to generate better keyphrases with the guidance of topic information. In addition to the scientific domain, we verify the effectiveness of our approach in the social media domain as well. The nature of social media content makes it difficult to directly transfer keyphrase generation methods to this domain, mainly because they are often short in length and extremely informal, making their information insufficient to infer keyphrases. To address this, we leverage relevant posts and their conversations (replying and reposting messages) and relevant entity relations to enrich the contexts of the original post. Specifically, we propose MOCHA (Multi-grained glObal Contexts Hashtag generAtor), a hashtag generation model consisting of two novel modules: RC-ATTENTION and RE-GRAPH. The RC-ATTENTION module uses cross-document attention to retrieve relevant posts and conversations, while the RE-GRAPH module employs a graph attention network to model the relevant entity relations.

Experimental results on five scientific document datasets and two social media datasets illustrate the validity and effectiveness of our models, which achieves the state-of-the-art performance on all datasets. Further analyses show that our models are able to generate keyphrases consistent with the topics and conversations while maintaining sufficient diversity.