GramError: a quality metric for machine generated songs.
Davies, Craig; Wiratunga, Nirmalie; Martin, Kyle
Professor Nirmalie Wiratunga firstname.lastname@example.org
Dr Kyle Martin email@example.com
This paper explores whether a simple grammar-based metric can accurately predict human opinion of machine-generated song lyrics quality. The proposed metric considers the percentage of words written in natural English and the number of grammatical errors to rate the quality of machine-generated lyrics. We use a state-of-the-art Recurrent Neural Network (RNN) model and adapt it to lyric generation by re-training on the lyrics of 5,000 songs. For our initial user trial, we use a small sample of songs generated by the RNN to calibrate the metric. Songs selected on the basis of this metric are further evaluated using ”Turinglike” tests to establish whether there is a correlation between metric score and human judgment. Our results show that there is strong correlation with human opinion, especially at lower levels of song quality. They also show that 75% of the RNN-generated lyrics passed for human-generated over 30% of the time.
DAVIES, C., WIRATUNGA, N. and MARTIN, K. 2018. GramError: a quality metric for machine generated songs. In Bramer, M. and Petridis, M. (eds.) Artificial intelligence XXXV: proceedings of the 38th British Computer Society's Specialist Group on Artificial Intelligence (SGAI) International conference on innovative techniques and applications of artificial intelligence (AI-2018), 11-13 December 2018, Cambridge, UK. Lecture notes in computer science, 11311. Cham: Springer [online], pages 184-190. Available from: https://doi.org/10.1007/978-3-030-04191-5_16
|Conference Name||38th British Computer Society's Specialist Group on Artificial Intelligence (SGAI) International conference on innovative techniques and applications of artificial intelligence (AI-2018)|
|Conference Location||Cambridge, UK|
|Start Date||Dec 11, 2018|
|End Date||Dec 13, 2018|
|Acceptance Date||Sep 3, 2018|
|Online Publication Date||Nov 16, 2018|
|Publication Date||Dec 31, 2018|
|Deposit Date||Jan 21, 2019|
|Publicly Available Date||Jan 21, 2019|
|Series Title||Lecture notes in computer science|
|Book Title||Artificial intelligence XXXV|
|Keywords||Natural language generation; Quality metric; Recurrent neural; Network|
DAVIES 2018 GramError
Publisher Licence URL
You might also like
DisCERN: discovering counterfactual explanations using relevance features from neighbourhoods.
Reasoning with counterfactual explanations for code vulnerability detection and correction.
FedSim: similarity guided model aggregation for federated learning.
Autonomous CPSoS for cognitive large manufacturing industries.