Problem with Satorra-Bentler Chi-Squared in WLS

The OpenMx users and the OpenMx Development Team have found a problem with the current implementation of the Satorra-Bentler scaled chi-squared difference test which is used for weighted least squares (WLS) models. The problem is isolated to models that use WLS, and further isolated to the use of mxCompare() with WLS. The statistics reported by summary() are unaffected. The anova() and mxCompareMatrix() functions are wrappers around mxCompare() and thus show the same problem.

The values of the 'SBchisq' column in mxCompare() do not behave how they theoretically should. The 'SBchisq' values are frequently preposterously large and sometimes negative, the latter of which is theoretically impossible. We recommend that users ignore all 'SBchisq' values until the problem is resolved. In the source code, we have added a warning to mxCompare(), mxCompareMatrix(), and anova() that advises users to not use 'SBchisq', and instead use the robust chi-squared value for chi-squared difference testing.

The procedure for chi-squared difference testing is quite simple: pchisq(diffchisq, df=diffdf, lower.tail=FALSE) where 'diffchisq' is the difference in the chi-squared values of models you are testing, 'diffdf' is the difference in the degrees of freedom of the models you are testing, and the output gives you the p-value for the test.

We anticipate resolving this issue in the next few weeks, but in the meantime want to protect users from inaccurate information.