Abstract

This study compares BERTurk, mBERT, XLM-RoBERTa and ELECTRA-based language models for topic classification of Turkish news texts. A new dataset of 48,000 news articles in 12 categories was compiled and made publicly available. The Turkish pre-trained BERTurk model achieved the highest performance with a macro F1 score of 93.6%. The effect of agglutinative morphology on subword segmentation was examined; increasing vocabulary size improved performance, especially for short texts.

Keywords:natural language processingtext classificationBERTTurkishtransformer

Full Text

PDF (TR) Download · 7.9 KB

Declarations

Ethics Approval
Bu çalışma etik kurul onayı gerektirmemektedir.
Conflict of Interest
Yazarlar herhangi bir çıkar çatışması olmadığını beyan eder.
Funding
Bu çalışma Selçuk Üniversitesi BAP Koordinatörlüğü tarafından desteklenmiştir (Proje No: 24003).

References 5

  1. Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT 2019 (pp. 4171–4186). https://doi.org/10.18653/v1/N19-1423
  2. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
  3. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
  4. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
  5. Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In International Conference on Learning Representations.

How to Cite

Öztürk, S., & Arslan, C. (2024). Performance of Transformer-Based Language Models in Classifying Turkish News Texts. International Journal of Science and Technology Research, 6(2), 1–14. https://doi.org/10.99999/ubtad.2024.4
BibTeX RIS (EndNote, Zotero, Mendeley)

License

CC BY 4.0

© 2024 Selin Öztürk, Cem Arslan. This article is distributed under the terms of the CC BY 4.0 license, which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. License text