EFFECT OF SAMPLE SIZE, ABILITY DISTRIBUTION AND TEST LENGTH ON DETECTION OF DIFFERENTIAL ITEM FUNCTIONING USING MANTEL-HAENSZEL STATISTIC

Abstract

Differential Item Functioning (DIF) is a statistical method that determines if test measurements distinguish abilities by comparing two sub-population outcomes on an item. The Mantel-Haenszel (MH) statistic provides an effect size measure that can give the magnitude of DIF. The purpose of the study was to investigate through simulation the effects of sample size, ability distribution and test length on the Effect Size (ES) of DIF and their influence on detection of DIF using MH method. A Factorial research design was used in the study. The population of the study consisted of 2000 examinee responses. A stratified random sampling technique was used with the stratifying criteria as the reference (r) and focal (f) groups. A small sample size (60r/60f) and a large sample size (1000r/1000f) were established. WinGen3 statistical software was used to generate dichotomous item response data which was replicated 1000 times. The findings of the study showed that whereas sample size and ability distribution had significant effects on the ES of DIF items when MH was used, test length had no statistically significant effect on the ES of DIF items. However, the number of DIF detections using MH statistic increased with test length regardless of the nature of Ability Distribution, The findings of the study are of great significance to teachers, educational policy makers, test developers and test users.