Among the many genres of language that have been studied in computational linguistics and spoken language processing, there has been a dearth of work on lyrics in music, despite the major impact that this form of language has across almost all human cultures. In this thesis, we propose theoretically motivated symbolic and distributed models for improvising lyrics in music and we choose the genre of hip hop lyrics as our domain. Through our work, we model the issues in song lyric improvisation using modern statistical language technologies and attempt to bridge the gap between language and music in natural language processing (NLP).

Firstly, we describe a novel Hidden Markov Model (HMM) based rhyme scheme detection module which identifies the rhyming scheme within a given stanza...[ Read more ]

Among the many genres of language that have been studied in computational linguistics and spoken language processing, there has been a dearth of work on lyrics in music, despite the major impact that this form of language has across almost all human cultures. In this thesis, we propose theoretically motivated symbolic and distributed models for improvising lyrics in music and we choose the genre of hip hop lyrics as our domain. Through our work, we model the issues in song lyric improvisation using modern statistical language technologies and attempt to bridge the gap between language and music in natural language processing (NLP).

Firstly, we describe a novel Hidden Markov Model (HMM) based rhyme scheme detection module which identifies the rhyming scheme within a given stanza in a completely unsupervised fashion without using any linguistic or phonetic features. We use this rhyme scheme detection module to select the training data for our improvisation models so as to generate fluent and rhyming output and demonstrate that using the rhyme scheme detection module improves the model performance considerably.

Secondly, we improvise hip hop lyrics by generating responses to challenges similar to a freestyle rap battle. We model the problem of improvisation as a machine translation problem where the challenge needs to be “translated” into a response and train a bottom-up token based bracketing inversion transduction grammar (BITG) model to perform the transduction. We also propose a search heuristic in our decoding algorithm and disfluency handling strategies to improve our model output. We contrast our model with an off-the-shelf phrase based SMT (PBSMT) model and show that our model generates significantly better responses that are more fluent and rhyme better with the challenges.

We also propose a novel model that improvises rhyming and fluent responses for a hip hop lyric challenge by combining both bottom-up token based rule induction and top-down rule segmentation strategies to learn a stochastic transduction grammar. We demonstrate that a smoothed interpolation of rules generated by top-down rule segmentation and bottom-up rule induction outperforms a linear interpolation of the segmental and token based rules. We also show that both our segmental models outperform the token based BITG model on the criterion of generating rhyming and fluent responses to hip hop lyrical challenges. We also show good model performance on Maghrebi French hip hop lyrics demonstrating the language independence of our models.

Another improvisation algorithm using TRAAM, a fully bilingual generalization of Pollack’s (1990) monolingual Recursive Auto-Associative Memory neural network model, in which each distributed vector represents a bilingual constituent is also presented. TRAAM models capture cross-lingual generalizations via soft bilingual categories and hence have attractive properties which can be used for the tasks such as bilingual grammar induction and statistical machine translation approaches. Using a novel pattern completion decoding algorithm, we use a trained TRAAM model to improvise hip hop lyrics.

Lastly, we discuss the challenges in evaluating the performance on the improvisation task of evaluating hip hop lyrics as a first step toward designing robust evaluation strategies for improvisation tasks, a relatively neglected area to date. We discuss our observations regarding inter-evaluator agreement on judging improvisation quality as a means to better understand the high degree of subjectivity at play in improvisation tasks, thereby enabling the design of more discriminative evaluation strategies to drive future model development.