How Did English Songs Evolve? Retrieving Information from Song Lyrics Via Natural Language Processing and Statistical Topic Modeling

Master's Thesis from the year 2019 in the subject Mathematics - Statistics, grade: 1,0, University of Bamberg (Statistik und Ökonometrie), language: English, abstract: In this thesis, the similarity, the complexity, as well as the evolution of English song lyrics over the past five decades will be examined with the help of statistical methods. Hence, the central research question of this thesis is: Can information gained by Natural Language Processing and statistical topic modelling be used to determine whether and to what extent song lyrics of various genres changed over the course of the past 50 years? Based on this, the goals of this thesis are:determining how similar songs of five diverse genres (alternative, country, pop, rock, and hip-hop) are, as measured by text statistics and text features that are composed by Natural Language Processing (NLP) and text mining methods. Additionally, using these methods as well as an attempt to find out whether song lyrics are becoming less complex and therefore less sophisticated And, finally, the main target of this thesis set for itself, computing statistical topic models by applying Latent Dirichlet Allocation (LDA), to analyse how similar the topics of songs are and whether they changed over time. This will be conducted by calculating similarity measures on the per-topic-per-word probability distributions that are output of the LDA models.