A development of an objective structured clinical examination in performance assessment for Advance emergency medical technician

Authors

  • Nuchpiya Thongchot Naval Medical Department

Keywords:

Performance assessment, Objective structured clinical examination, Advance emergency medical technician student

Abstract

To develop a performance procedure to assess clinical skills for Advance emergency medical technician (AEMT) students. This study aimed to develop and validate structured clinical examination (OSCE). This study modified the logical development steps proposed to incorporate the following four steps: 1) blueprints and mapping 2) scenario development 3) examiner training 4) validity and reliability testing. An OSCE of 3 situations of 10 min each were adopted. A sample consisted of 20 AEMT students from Naval operational medical school.

The results were summarized as follows: 1. Item-level analyses 1) Task difficulty index in rater 1 of situation 1, 2, 3 were 0.69, 0.71, 0.69 and rater 2 of situation 1, 2, 3 were 0.70, 0.73, 0.69. 2) Discrimination index in rater 1 of situation 1, 2, 3 were 0.74, 0.26, 0.74 and rater 2 of situation 1, 2, 3 were 0.74, 0, 0.38. 2.Test level analysis 1) Item Objective Congruence: IOC was 0.66 2) Inter-rater reliability by Pearson product-moment correlation coefficient of situation 1, 2, 3 were 0.88, 0.98, 0.97 and use Generalizability Coefficient (G-coefficient) PXSXR design were G-coefficient for relative decision ( gif.latex?\rho^ gif.latex?\delta2)  were 0.49 and G-coefficient for absolute decision ( gif.latex?\rho^gif.latex?\Delta2)  were 0.48. To keep reliability less than 0.7, then use rater five persons in the test.

References

กมลวรรณ ตังธนกานนท์. (2563). การวัดและประเมินทักษะการปฏิบัติ. กรุงเทพฯ: สำนักพิมพ์จุฬาลงกรณ์มหาวิทยาลัย.

รุ่งนภา นาคะภากร. (2540). การพัฒนาการสอบทางคลินิกเชิงโครงสร้างแบบปรนัยในการวัดทักษะทางคลินิกสำหรับนักศึกษาพยาบาล. (วิทยานิพนธ์ครุศาสตรมหาบัณฑิต). จุฬาลงกรณ์มหาวิทยาลัย. กรุงเพทฯ.

ศิริชัย กาญจนวาสี. (2555). ทฤษฎีการทดสอบแนวใหม่. พิมพ์ครั้งที่ 4. กรุงเทพฯ: โรงพิมพ์แห่งจุฬาลงกรณ์มหาวิทยาลัย.

ศิริวรรณ ชูกำเนิด และสุระพร ปุ้ยเจริญ. (2560). The use of Objective Structured Clinical Examinationin Nursing Education. วารสารเครือข่ายวิทยาลัยพยาบาลและการสาธารณสุขภาคใต้, 4(3), 240.

American Heart Association. (2017). Get with the guidelines-resuscitation overview. Retrieved 14 December 2020, from https://www.heart.org/en/professional/quality-improvement/get-with-the-guidelines/get-with-the-guidelines-resuscitation/get-with-the-guidelines-resuscitation-overview#.WPefZ4jyv6Q.

Conn, C. A., Bohan, K. J., Pieper, S. L., & Musumeci, M. (2020). Validity inquiry process: Practical guidance for examining performance assessments and building a validity argument. Studies in Educational Evaluation, 65. doi:10.1016/j.stueduc.2020.100843.

Kamran Khan, Kathryn Gaunt, Sankaranarayanan Ramachandran, & Pushkar, P. (2013). The Objective Structured Clinical Examination (OSCE) AMEE Guide No.81. Part2: Organisation & Administration. MedicalTeacher, 35,e1447-e1463. doi:https://doi.org/10.3109/0142159X.2013.818635.

Lee, K. C., Ho, C. H., Yu, C. C., & Chao, Y. F. (2020). The development of a six-station OSCE for evaluating the clinical competency of the student nurses before graduation: A validity and reliability analysis. Nurse Educ Today, 84, 104247. doi:10.1016/j.nedt.2019.104247

Miller, G. (1990). The assessment of clinical skill/competence/performance. Academic Medical education, 65, S63-S67.

Shinnick, M. A., & Woo, M. A. (2018). Validation of time to task performance assessment method in simulation: A comparative design study. Nurse Educ Today, 64, 108-114. doi:10.1016/j.nedt.2018.02.011.

.

Downloads

Published

17-08-2021

Issue

Section

Research Article