Browsing by Subject "Comparability"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Open Access A comparability and classification analysis of computerized adaptive and conventional paper- based versions of an English language proficiency reading subtest(Bilkent University, 2022-01) Kaya, ElifThe current study compares the computerized adaptive test (CAT) and paper-based test (PBT) versions of an English language proficiency reading subtest in terms of psychometric qualities. The study also investigates classification performance of CATs not designed for classification purposes with reference to its PBT version. Real data-based simulations were conducted under varying test conditions. The results demonstrate that ability levels estimated by CATs and PBT are similar. A relatively larger item reduction can be obtained with 0.50 and 0.40 standard error thresholds and CATs terminated with 20, 25, and 30 items performed well with acceptable SE values. Reliability of CAT ability estimates was comparable and highly correlated with PBT estimates. For classification analysis, classification accuracy (CA) and classification consistency (CC) was also estimated using the Rudner method. Classification analyses were conducted on single and multiple cut-off points. The results showed that the use of a single cut-off score produced better classification performance, particularly for high and low ability groups. On the other hand, the use of multiple cut-off scores simultaneously yielded significantly lower classification performance. Overall, the results highlight the potential for CATs not designed specifically for classification to serve classification purposes and indicate avenues for further research.Item Open Access Measurement invariance of student evaluation of teaching across groups defined by course-related variables(International Online Journal of Educational Sciences, 2015) Kalender, İ.In the present study, comparability of scores from student evaluation of teaching forms was investigated. This is an important issue because scores given by students are used in decision making in higher education institutions. Three course-related variables (grade level, course type, and course credit) were used to define student subgroups. Then, multi-group confirmatory factor analysis was used to assess invariance of factorial structure, factor loadings and factor means across groups. It was found that although a common factorial structure held across groups, fully invariant factor loadings were observed only across instructors who teach different course types. For other groups, only partial invariance of factor loadings was obtained. Analyses also revealed that none of the subgroups had invariant factor means, indicating a possible bias. Results indicate that comparison of instructors based on student ratings may not be valid as it is mostly assumed.