AN APPLICATION OF MULTIDIMENSIONAL ITEM RESPONSE THEORY FOR DEVELOPING STANDRAD SETTING OF MATHEMATICAL PROFICIENCY LEVEL FOR THE MIXED FORMAT TEST การประยุกต์ใช้ทฤษฎีการตอบสนองข้อสอบแบบพหุมิติเพื่อกำหนดมาตรฐาน การประเมินระดับความสามารถทางคณิตศาสตร์ของแบบทดสอบรูปแบบผสม
Article Sidebar
Main Article Content
Abstract
The present study were intended to establish the standards setting of mathematical proficiency for the mixed-format test by setting the criterion zone on Wright Map, to compare quality of standards setting of mathematical proficiency evaluation for proficiency estimation methods between MLE and EAP, and to design the standards setting of mathematical proficiency for the mixed-format test. The samples were 517 Mattayomsuksa 1 students. This study was conducted through design research. The data were analyzed and interpreted based on the MRCML model. The results are as follows:
1. On standards setting of mathematical proficiency evaluation using the criterion zone based on Wright Map, it was found that mathematical processes as the first dimension consisted of five levels with four cut scores; the lowest and highest scores were -1.41, -0.69, 0.49, and 1.75 respectively. Similarly, conceptual structures as the second dimension featured five levels and four cut scores of which the lowest and highest were -0.98, -0.14, 0.44, and 1.70 respectively.
2. In terms of comparing quality of standards setting methods between MLE and EAP, it was found that the standard errors of MLE -based estimation fulfilled the acceptable criteria, considering the duration, values, and efficiency of mathematical than EAP-based estimation.
3. In designing the standards setting of mathematical proficiency in the mixed-format test, it included four steps as follows: 1) prepare test-score data, 2) analyze estimation of students and threshold of test, 3) calculate mathematical proficiency based on the cut-score formula = , and 4) compare cut scores with raw scores estimated through MLE methods to convert such scores to scale scores (
d) and mathematical proficiency = 50+ (
*10).
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The owner of the article does not copy or violate any of its copyright. If any copyright infringement occurs or prosecution, in any case, the Editorial Board is not involved in all the rights to the owner of the article to be performed.
References
AERA, APA, & NCME. (2014). Standards for educational and psychological testing (6th ed.). Washington, DC: American Educational Research Association.
Bock, R. D., & Aitkin, M (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika volume 46, 443 – 459.
Bock, R. D., & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431 - 444.
Chanayota, K., & Chanpen, P. (2018). Developing a framework for assessing competence levels mathematical subjects of algebra: Applications of structural models. Thailand Measurement Evaluation and Research Conference 26th (pp. 289-311). Khon Kaen: Khon Kaen University. [in Thai]
Chinjunthuk, S., & Junpeng, P. (2020). Assessment guidelines for student’s personalized mathematical proficiency development. Journal of Educational Measurement Mahasarakham University, 26(1), 47-64. [in Thai]
Hejri, S., & Jalili, M. (2014). Standard setting in medical education: Fundamental concepts and emerging challenges. Medical journal of the Islamic Republic of Iran, 28, 34.
Jungpeng, P. (2018). Application of multidimensional item response theory for research. Khon Kean: Khonkean University. [in Thai]
Jungpeng, P., Maweang, M., Chinjunthuk, S., Suwannatrai, P., Krotha, J., & Chanayotha, K. (2020). Developing students’ mathematical proficiency level diagnostic tools through information technology in assessment for learning report (Research report). KhonKean: Khonkean University. [in Thai]
Junpeng, P, Inprasitha, M., & Wilson, M. (2018). Modelling of the open-ended items for assessing multiple proficiencies in mathematical problem solving. Turkish Online Journal of Educational Technology, 2(Special Issue), 142-149.
Junpeng, P., Krotha, J., Chanayota, K., Tang, K. N., & Wilson, M. (2019). Constructing progress maps of digital technology for diagnosing mathematical proficiency. Journal of Education and Learning, 8(6), 90-102.
Kanchanawasi, S. (2009). Classical test theory. Bangkok: Chulalongkorn University Press. [in Thai]
National Institute of Educational Testing. (2020). Results testing of O-net M.3 (2019). Retrieved December 17,2020, from www.newonetresult.niets.or.th [in Thai]
Pasiphol, S. (2019). Computerized adaptive testing system in information and communication. Journal of Educational Measurement Mahasarakham University, 25(2), 96 – 11. [in Thai]
Reeves, T. C. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research (pp. 52-66). London: Routledge.
SaiKang, T. (2020). developing a digital tool for assessing multidimensional mathematical proficiency level of the seventh-grade students (Master thesis) Khon Kaen: Khon Kaen University. [in Thai]
Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, N.J: Routledge.
Wongwanich, S. (2020). Design research for education. Bangkok: Chulalongkorn University. [in Thai]
Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ACER Con Quest: Generalized item response modelling software. Australia: ACER Press.
Wyse, A.E. (2013). Construct Maps as a foundation for standard setting. Measurement Interdisciplinary Research and Perspectives, 11(4), 139-170.
Zheng, Y., & Chang, H. H. (2015). On-the-fly assembled multistage adaptive testing. Applied Psychological Measurement, 39(2), 1–15.