Abstract:

Objective To design the role-play rubric and to evaluate its psychometric properties. Methods Literature review,classroom observation and content analysis were used to analyze and refine the initial evaluation dimensions and evaluation methods. Then a role-play rubric consultation questionnaire was prepared. Delphi technique was used to conduct two-round letter consultation among 24 experts. The role-play rubric was distributed to the 11 role-play videos of the undergraduates of grade 2018 and invite 6experts to use the rubric to conduct assessment. The reliability was tested by using the consistency among evaluators. Content validity index was used for validity evaluation by inviting 24 experts. Results The positive coefficients of the two rounds of expert consultation are 92.3% and 100% respectively,the Cr was 0.89. The Kendall’s W in the second round of “evaluation dimensions” and “evaluation criteria” were 0.466 and 0.469 respectively. The final role-play rubric included 5 dimensions,4 levels,and the corresponding evaluation criteria. The results of the reliability and validity test of the rubric showed that the consistency of the expert score was 0.832~0.962,the S-CVI was 0.99. Conclusion The design process of role-play rubric is scientific and the content setting is reasonable. The rubric shows good reliability and validity,which can provide reference for the implementation and evaluation of role-play teaching method.

Key words: Role-play, Delphi Technique, Rubric, Interrater agreement