CYBERBULLYING TWEET CLASSIFICATION USING LANGUAGE MODELS
Keywords:
Classification, Language Models, Online Safety, Transformers, Natural Language ProcessingAbstract
Cyberbullying is a severe problem in the age of social media and round-the-clock connectivity. Detecting cyberbullying is the first step in the right direction to enhancing online safety and improving mental health. Some organizations have a legal and ethical responsibility to carry out this task proactively to prevent harm to their employees or users. There is a lot of research on sentiment analysis in the realm of Natural Language Processing. However, for detecting and flagging cyberbullying, there are very few datasets out there (unlike toxicity detection or hate detection). In this paper, we explore the problem of cyberbullying tweet detection by approaching this as a classification problem. By leveraging the power of finetuning large language models (like LLAMA3 and Llama-Guard) using PEFT, we perform classification (after extracting the embeddings). We compare these techniques with traditional transformer-like approaches (SBERT). These conventional approaches have much smaller architectures than LLAMA3 and Llama-Guard. However, these models can help extract embeddings with power classification approaches such as Random Forests. The classification techniques we employ here show that the performance of LLAMA3 and Llama-Guard (trained with PEFT) compared to SBERT is more balanced.
References
Larxel, Cyberbullying dataset, Kaggle.
Putri, N. L. P. M. S., Nurjanah, D., C Nurrahmi, H. (2022). Cyberbullying detection on twitter using support vector machine classification method. Building of Informatics, Technology and Science (BITS), 3(4), 661-666.
Yin, D., Xue, Z., Hong, L., Davison, B. D., Kontostathis, A., C Edwards, L. (2009). Detection of harassment on web 2.0. Proceedings of the Content Analysis in the WEB, 2(0), 1-7.
Nurrahmi, H., C Nurjanah, D. (2018, March). Indonesian twitter cyberbullying detection using text classification and user credibility. In 2018 International Conference on Information and Communications Technology (ICOIACT) (pp. 543-548). IEEE.
Lee, P. J., Hu, Y. H., Chen, K., Tarn, J. M., C Cheng, L. E. (2018). Cyberbullying Detection on Social Network Services. PACIS, 61.
Van Hee, C., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., De Pauw, G., C Hoste, V. (2015, September). Detection and fine-grained classification of cyberbullying events. In Proceedings of the international conference
recent advances in natural language processing (pp. 672-680).
Nahar, V., Li, X., Pang, C., C Zhang, Y. (2013, November). Cyberbullying detection based on text-stream classification. In The 11th Australasian Data Mining Conference (AusDM 2013).
Sen, M., Masih, J., C Rajasekaran, R. (2024, January). From Tweets to Insights: BERT-Enhanced Models for Cyberbullying Detection. In 2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS) (pp. 1289-1293). IEEE.
Paul, S., C Saha, S. (2022). CyberBERT: BERT for cyberbullying identification: BERT for cyberbullying identification. Multimedia Systems, 28(6), 1897-1904.
Muralidhar, A. (2024). BERT-BASED DETECTION OF CYBERBULLYING IN ONLINE TEXTS. Scientific and practical cyber security journal.
Reimers, N. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv preprint arXiv:1S08.10084.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., ... C Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.
Kumar, Y., Huang, K., Perez, A., Yang, G., Li, J. J., Morreale, P., ... C Jiang, R. (2024). Bias and Cyberbullying Detection and Data Generation Using Transformer Artificial Intelligence Models and Top Large Language Models. Electronics, 13(17), 3431.
Ottosson, D. (2023). Cyberbullying Detection on social platforms using LargeLanguage Models.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., ... C Ganapathy, R. (2024). The llama 3 herd of models. arXiv preprint arXiv:2407.21783.
META LLAMA-3, https://ai.meta.com/blog/meta-llama-3/
Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., ... C Khabsa, M. (2023). Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674.
HuggingFace Sentence Transformers, https://huggingface.co/sentence- transformers/paraphrase-MiniLM-L6-v2
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Rahul Kavi , Jeevan Anne (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
