This study examined the measurement invariance of a Likert scale and a graphic rating scale using two configurations of the scale numeric values. The graphic rating scale and the Likert scale were each be measured using two different response category sets, a scale ranging from 1 to 5 and a scale ranging from -2 to +2. Participants were asked to respond to a scale measuring Employee Engagement and were randomly assigned to one of the four scaling conditions.

Following the guidelines proposed by Vanderberg & Lance (2000), a multi-step approach to testing measurement invariance using a multiple group confirmatory factor analysis (CFA) models with the maximum likelihood estimation was employed to test for configural invariance, metric invariance, and scalar invariance. Additionally, the scales were tested to see if subjects display a tendency to utilize the scales differently due to the numeric anchoring conditions.

In the first step of the invariance testing, the four scaling techniques were found to have good model fit using a confirmatory factor analysis. Next the factor structures were constrained to be equal across the groups. Good fit was found with the factor structures constrained to be equal which is evidence for configural invariance. To test for metric invariance the model was additionally constrained to have equal factor loadings. When tested the model showed good fit indicating metric invariance. Next, to test for scalar invariance the model was additionally constrained to have equal intercepts of the indicators. When tested the model showed good fit indicating scalar invariance. Finally, each of the scaling techniques were then tested with a one-way analysis of variance to determine if there were means score differences for any of the four scale types. No significant differences were found.

Results of this study would indicate that Likert scales and graphic numeric rating scales have measurement invariance, however further research should be done to address the limitations of this study.