Publication Details
Abstract
Automatic short-answer assessment plays a crucial role in modern educational technology by providing scalable and consistent evaluation of student responses. However, traditional similarity-based approaches often fail to capture the deeper semantic meaning of answers, limiting their effectiveness. This study investigates the performance of a fine-tuned AraBERT v2 model for Arabic automatic assessment using the AR-ASAG dataset. The model processes student answers by preprocessing, tokenizing, and extracting contextual representations from the [CLS] token, with a Dropout probability of 0.3 to reduce overfitting. Experimental results show that the fine-tuned AraBERT v2 model significantly outperforms traditional cosine similarity methods. Specifically, cosine similarity between student and model answers achieved a Mean Squared Error (MSE) of 0.1834 and a Pearson correlation of 0.3098, while cosine similarity with the question yielded weaker results with an MSE of 0.1640 and a correlation of 0.0139.
In contrast, the fine-tuned AraBERT v2 achieved superior performance, reaching an MSE of 0.0187 and a Pearson correlation of 0.8866 after three training epochs.
These results demonstrate the model’s high robustness, its effectiveness in capturing deep semantic similarity, and its reliability as a solution for automatic short-answer assessment in Arabic educational contexts.