Parallel Stylometric Document Embeddings with Deep Learning Based Language Models in Literary Authorship Attribution

Reference

Škorić, Mihailo, Ranka Stanković, Milica Ikonić Nešić, Joanna Byszuk, and Maciej Eder. “Parallel Stylometric Document Embeddings with Deep Learning Based Language Models in Literary Authorship Attribution”, Mathematics, 10.5 (2022), 838. DOI: 10.3390/math10050838

Abstract

This paper explores the effectiveness of parallel stylometric document embeddings in solving the authorship attribution task by testing a novel approach on literary texts in 7 different languages, totaling in 7051 unique 10,000-token chunks from 700 PoS and lemma annotated documents. We used these documents to produce four document embedding models using Stylo R package (word-based, lemma-based, PoS-trigrams-based, and PoS-mask-based) and one document embedding model using mBERT for each of the seven languages. We created further derivations of these embeddings in the form of average, product, minimum, maximum, and l2 norm of these document embedding matrices and tested them both including and excluding the mBERT-based document embeddings for each language. Finally, we trained several perceptrons on the portions of the dataset in order to procure adequate weights for a weighted combination approach. We tested standalone (two baselines) and composite embeddings for classification accuracy, precision, recall, weighted-average, and macro-averaged F1-score, compared them with one another and have found that for each language most of our composition methods outperform the baselines (with a couple of methods outperforming all baselines for all languages), with or without mBERT inputs, which are found to have no significant positive impact on the results of our methods.

Keywords

Document embeddings, Authorship attribution, Language modelling, Corpus, Stylometry, Stylo R, POS, Tagging

Direct Access

• PDF
• Article

BibTex

@article{skoric_parallel_2022,
	title = {Parallel {Stylometric} {Document} {Embeddings} with {Deep} {Learning} {Based} {Language} {Models} in {Literary} {Authorship} {Attribution}},
	volume = {10},
	copyright = {http://creativecommons.org/licenses/by/3.0/},
	issn = {2227-7390},
	url = {https://www.mdpi.com/2227-7390/10/5/838},
	doi = {10.3390/math10050838},
	abstract = {This paper explores the effectiveness of parallel stylometric document embeddings in solving the authorship attribution task by testing a novel approach on literary texts in 7 different languages, totaling in 7051 unique 10,000-token chunks from 700 PoS and lemma annotated documents. We used these documents to produce four document embedding models using Stylo R package (word-based, lemma-based, PoS-trigrams-based, and PoS-mask-based) and one document embedding model using mBERT for each of the seven languages. We created further derivations of these embeddings in the form of average, product, minimum, maximum, and l2 norm of these document embedding matrices and tested them both including and excluding the mBERT-based document embeddings for each language. Finally, we trained several perceptrons on the portions of the dataset in order to procure adequate weights for a weighted combination approach. We tested standalone (two baselines) and composite embeddings for classification accuracy, precision, recall, weighted-average, and macro-averaged F1-score, compared them with one another and have found that for each language most of our composition methods outperform the baselines (with a couple of methods outperforming all baselines for all languages), with or without mBERT inputs, which are found to have no significant positive impact on the results of our methods.},
	language = {en},
	number = {5},
	urldate = {2022-03-07},
	journal = {Mathematics},
	author = {Škorić, Mihailo and Stanković, Ranka and Ikonić Nešić, Milica and Byszuk, Joanna and Eder, Maciej},
	month = jan,
	year = {2022},
	note = {Number: 5
Publisher: Multidisciplinary Digital Publishing Institute},
	keywords = {type\_publication},
	pages = {838},
}