Developing a Higher-Order Thinking Skills Assessment Instrument for Akidah Akhlak Learning: An ADDIE and Rasch Model Approach
DOI:
https://doi.org/10.35445/alishlah.v18i3.9743Keywords:
higher-order-thinking skills, Mathematics Assessment Instruments, akidah akhlak, Science process skills Rasch model, Islamic educationAbstract
The assessment of Higher-Order Thinking Skills (HOTS) in Akidah Akhlak learning remains dominated by lower-order cognitive questions, limiting the evaluation of students’ analytical, evaluative, and creative thinking abilities. This study aimed to develop and evaluate the psychometric quality of a HOTS-based assessment instrument for Madrasah Aliyah students. This research employed a Research and Development approach using the ADDIE model, integrated with Rasch analysis. The instrument comprised 40 multiple-choice items targeting cognitive levels C4, C5, and C6. Content validity was assessed by five experts using Aiken’s V. Following a preliminary trial involving 30 students, field testing was conducted with 300 students from five madrasahs in Jambi Province. Item fit, reliability, and difficulty were analyzed using the QUEST program. Expert validation yielded an average Aiken’s V of 0.93, indicating strong content validity. Rasch analysis identified one misfitting item, resulting in a final instrument of 39 items. All retained items satisfied the reported Outfit t criterion (≤ 2.00). Item reliability reached 0.97, while person reliability was 0.69. The mean student ability (2.05 logits) exceeded the mean item difficulty (0.00 logits), indicating that the instrument was relatively easy for the sampled students. The instrument demonstrates promising content validity and item-level psychometric properties for assessing HOTS in Akidah Akhlak learning. However, moderate person reliability and suboptimal item–person targeting indicate that further refinement is needed before its use for individual diagnostic assessment.
References
Afikah, A., Astuti, S. R. D., Suyanta, S., Jumadi, J., & Rohaeti, E. (2022). Mobile Learning in Science Education to Improve Higher-Order Thinking Skills (HOTS) and Communication Skills: A Systematic Review. International Journal of Advanced Computer Science and Applications, 13(7). https://doi.org/10.14569/IJACSA.2022.0130782
Alghadir, A. H., Anwer, S., Iqbal, A., & Iqbal, Z. A. (2018). Test-retest reliability, validity, and minimum detectable change of visual analog, numerical rating, and verbal rating scales for measurement of osteoarthritic knee pain. Journal of Pain Research, 11. https://doi.org/10.2147/JPR.S158847
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikhshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (2023). Revisi atas Taksonomi Bloom (et.al.) A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives (Abridge Edition). TAKSONOMI ANDERSON (et.Al.): Revisi Atas Taksonomi Bloom (et.Al.).
Anwar, N. (2025). Report review: Reimagining our futures together: a new social contract for education. Globalisation, Societies and Education, 23(5). https://doi.org/10.1080/14767724.2023.2269103
Asnawi, M. F. A., & Mas’ad, Z. (2025). Effectiveness of Islamic Education Learning Model Based on Higher Order Thinking Skills (HOTS): Systematic Review. Progressive Journal of Educational Science, 1(2). https://doi.org/10.65729/progressive.v1i2.330
Bond, T. G., Yan, Z., & Heene, M. (2020). Applying the rasch model: Fundamental measurement in the human sciences. In Applying the Rasch Model: Fundamental Measurement in the Human Sciences. https://doi.org/10.4324/9780429030499
Boone, W. J. (2016). Rasch analysis for instrument development: Why,when,and how? CBE Life Sciences Education, 15(4). https://doi.org/10.1187/cbe.16-04-0148
Branch, R. M. (2010). Instructional design: The ADDIE approach. In Instructional Design: The ADDIE Approach. https://doi.org/10.1007/978-0-387-09506-6
Brookhart, S. M. (2010). How to Assess Higher-Order Thinking Skills in Your Classroom. In Ascd.
Caselli, S., Leonardi, M., Magnani, F. G., Cacciatore, M., Barbadoro, F., Ippoliti, C., Kreiner, S., Pellicciari, L., & La Porta, F. (2025). Comparing the Different Sets of Item-Level Diagnostic Criteria of the Coma Recovery Scale-Revised (CRS-R): A Measurement-Based Approach Driven by Rasch Analysis. Archives of Physical Medicine and Rehabilitation, 106(7). https://doi.org/10.1016/j.apmr.2024.12.009
Dong, Y., Xu, W., Huang, J., & Yann, K. (2025). Validating and refining a multi-dimensional scale for measuring AI literacy in education using the Rasch Model. Humanities and Social Sciences Communications, 12(1). https://doi.org/10.1057/s41599-025-05670-6
Fadkhurrohmad, M. N., & Nursikin, M. (2023). Pendekatan dalam inovasi materi dan metode pembelajaran Kurikulum PAI berbasis HOTS Era Industri 4 . 0 di MTs NU Salatiga. Jurnal Pengabdian Kepada Masyarakat Nusantara (JPkMN), 4(1), 386–395.
Fauzi, A., & Inayati, N. L. (2023). Implementasi Evaluasi Pembelajaran Pendidikan Al Islam di Sekolah Menengah Atas Muhammadiyah. Munaddhomah, 4(2), 272–283. https://doi.org/10.31538/munaddhomah.v4i2.438
Helmiyatun, Taqiudin, M., & Ahmad. (2025). Content Validity of Student Perception Instrument on School Educational Management. Journal of Practice Learning and Educational Development, 5(1). https://doi.org/10.58737/jpled.v5i1.422
Johanson, G. A., & Brooks, G. P. (2010). Initial scale development: Sample size for pilot studies. Educational and Psychological Measurement, 70(3). https://doi.org/10.1177/0013164409355692
Karim, R., Mandang, F. H., Luntungan, G., & Rotty, V. N. J. (2020). Improving Competency Hosted Skills through Think-Pair-Share Cooperative Learning Models in Class XI Science Students of SMA Negeri 1 Ratahan. Asia Pacific Journal of Management and Education, 3(3), 32–36. https://doi.org/10.32535/apjme.v3i3.965
Kemendikbudristek. (2024). Peraturan Menteri Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia Nomor 8 Tahun 2024 Tentang Standar Isi Pada PAUD,Jenjang pendidikan Dasar dan Jenjang Pendidikan Menengah. In Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi.
Kunselman, A. R. (2024). A brief overview of pilot studies and their sample size justification. In Fertility and Sterility (Vol. 121, Issue 6). https://doi.org/10.1016/j.fertnstert.2024.01.040
Linacre, J. M. (2002). What do infit and outfit, mean-square and standardized mean? Rasch measurement transaction. In https://www.rasch.org/rmt/rmt162.pdf (Vol. 16, Issue 2).
Maxnun, L., Kristiani, K., & Sulistyaningrum, C. D. (2024). Development of hots-based cognitive assessment instruments: ADDIE model. Journal of Education and Learning, 18(2). https://doi.org/10.11591/edulearn.v18i2.21079
Mufrihah, A., Yudha, E. S., Supriatna, M., Ahman, A., & Nurihsan, J. (2025). Rasch Model Analysis of Santri Reverence Morals Scale. Islamic Guidance and Counseling Journal, 8(1), 1–18. https://doi.org/10.25217/0020258548700
Ocy, D. R., Rahayu, W., & Arthur, R. (2024). Rasch model analysis: Evaluating students’ spatial thinking ability in higher-order thinking skills trigonometric comparison assessment (HOTS-TCA). Jurnal Gantang, 9(2). https://doi.org/10.31629/jg.v9i2.7386
OECD. (2023). PISA 2022 Assessment and Analytical Framework. In OECD Publishing.
Pratiwi, Z. I., & Maharani, D. (2020). Penerapan Pembelajaran Pendidikan Agama Islam ( Pai ) Berbasis Higher Order Thinking Skills ( HOTS ) ( Studi Analisis Pada Kelas XI dI SMA Dharma Karya UT Tangerang Selatan). Jurnal Qiro’ah, 10(2), 57–72.
Rohman, M. H., Marwoto, P., Nugroho, S. E., & Supriyadi, S. (2024). Effectiveness of Ethnoecological-STEM Project-Based Learning Model to Improve Critical Thinking Skills, Creativity, and Science Concept Mastery. International Journal of Cognitive Research in Science, Engineering and Education, 12(3). https://doi.org/10.23947/2334-8496-2024-12-3-521-534
Sugiyono. (2020). Metode Penelitian Kuantitatif, Kualitatif, dan R&D. ALFABETA.
Susilawati, S., Yaqin, M. Z. N., Wahidmurni, Ulfa, F. A., & Krismoneta. (2023). Training on Preparation of Islamic Integrated HOTS-Based Questions and Their Application in Online Learning in Elementary Schools. Munaddhomah, 4(4), 867–877. https://doi.org/10.31538/munaddhomah.v4i4.459
Udin By Arifin, M. B., Nurdyansyah, Rindaningsih, I., & Kalimah, S. (2021). Development of Smart Play Wheel Learning Media to Improve Student Learning Outcomes in Islamic Elementary schools. Journal of Physics: Conference Series, 1779(1). https://doi.org/10.1088/1742-6596/1779/1/012049
Utami, W. S., Kusaeri, Ridho, A., Yusuf, A., Widodo, B. S., Prastiyono, H., & Murtini, S. (2025). Unraveling social and economic problems: what basics of critical thinking skills are needed? International Journal of Evaluation and Research in Education, 14(1), 566–574. https://doi.org/10.11591/ijere.v14i1.29180
Downloads
Published
How to Cite
Issue
Section
Citation Check
License
Copyright (c) 2026 Thoiroh Thoiroh
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
