In OpenMx v. 2.11.5, the fit statistics output is the same whether type="cov" or type="cor". In a previous version of OpenMx, I recall that the manifest variable variances would be subtracted from the number of degrees of freedom and observed statistics, and the fit statistics were calculated using the reduced degrees of freedom. Now, the number of observed statistics number is reduced, but the degrees of freedom and fit statistics are not similarly adjusted.
Also, previously the warning about standard errors and fit statistics for type="cor" being potentially incorrect with the Steiger (1980) citation was printed when summary.MxModel() was called as when mxRun() was called. Now the warning only prints with mxRun()
For example, using the HS.ability.data example in help(mxModel), the output for type="cov" is:
Model Statistics: | Parameters | Degrees of Freedom | Fit (-2lnL units) Model: 21 24 10970.90 Saturated: 45 0 10928.42 Independence: 9 36 11882.01 Number of observations/statistics: 301/45 chi-square: χ² ( df=24 ) = 42.48505, p = 0.01138111 Information Criteria: | df Penalty | Parameters Penalty | Sample-Size Adjusted AIC: -5.514954 84.48505 87.79687 BIC: -94.485600 162.33436 95.73430 CFI: 0.9798547 TLI: 0.9697821 (also known as NNFI) RMSEA: 0.05058496 [95% CI (0.01638508, 0.07931089)] Prob(RMSEA <= 0.05): 0.4530891
The output for type="cor" is:
Model Statistics: | Parameters | Degrees of Freedom | Fit (-2lnL units) Model: 21 15 1788.898 Saturated: 45 -9 1746.413 Independence: 9 27 2700.000 Number of observations/statistics: 301/36 chi-square: χ² ( df=24 ) = 42.48505, p = 0.01138111 Information Criteria: | df Penalty | Parameters Penalty | Sample-Size Adjusted AIC: 12.48505 84.48505 87.79687 BIC: -43.12161 162.33436 95.73430 CFI: 0.9798547 TLI: 0.9697821 (also known as NNFI) RMSEA: 0.05058496 [95% CI (0.01638508, 0.07931089)] Prob(RMSEA <= 0.05): 0.4530891
I'm not sure which version this behavior changed in.
I'm not sure when the change occurred. Was this an intentional change? If so, was it determined that the unadjusted df/fit statistics are more likely to be correct (the problems with doing covariance structure analyses with correlations acknowledged)?
Printer-friendly version
I'm pretty sure this is unintentional and indicates a bug.
I'm searching back. There has been no change in behavior since v2.9.9 (2018-05-11).
It's hard to test earlier version of OpenMx. Can we narrow this bug down? Is the problem in summary()?
I would expect it to be in summary() as that is where the fit indices are computed and the Model statistics table constructed.
Did a change occur in OpenMx:::computeOptimizationStatistics()? That is where the DoF are calculated.
I installed v2.6.9 (2016-07-28). Cov has observed statistics: 45, degrees of freedom: 24. Cor has observed statistics: 36, degrees of freedom: 15. These look the same. However, I do notice differences in the chi-squared and Information Criteria fit stats,
For cov:
chi-square: X2 ( df=24 ) = 88.7134, p = 2.351412e-09 Information Criteria: | df Penalty | Parameters Penalty | Sample-Size Adjusted AIC: 40.71340 130.7134 NA BIC: -48.25725 208.5627 141.9627For cor:
chi-square: X2 ( df=24 ) = 88.7134, p = 2.351412e-09 Information Criteria: | df Penalty | Parameters Penalty | Sample-Size Adjusted AIC: 58.713401 130.7134 NA BIC: 3.106747 208.5627 141.9627There are lots of differences here compared to current output. What specifically do you think is incorrect?
When I run a RAM model with type = "cor" data , I get warning message:
This is with the current OpenMx: 2.12.2.267 [GIT v2.12.2-267-g3cbee07]
An upcoming version of OpenMx will constrain the variances of the manifest portion of the
Smatrix to 1 iftype="cor"in RAM models. Secondly, we'll likely stop allowing cor in non-RAM models, with a note to the user to asserttype = covand add constraints where they need to be based on their matrix usage, or warn people that they need to take care of this.Automating the constraint on the diagonal of the model-expected covariance/correlation matrix is going to be trickier than I thought when I volunteered to tackle this bug. A model of
type="RAM"has no MxMatrix or MxAlgebra for its model-expected matrix. The constraint would have to exist solely in the backend. We do have a precedent for creating new constraints in the backend: we do that to implement the constrained representation of the confidence-limit problem. But, those constraints don't persist to be returned to the frontend, and they don't do things like adjust the degrees-of-freedom of the model. Maybe we should just deprecatetype="cor"?Analyses with correlation matrices are really common, and returning to the previous behavior where the sole change for type="cor" is to display a warning about standard errors and adjusting the degrees of freedom and associated fit indices would be much preferable I think to users just analyzing correlation matrices as covariance matrices without any warning or adjustment.
Rather than doing the diagonal constraint implicitly, would adding a new mxCorrelationConstraint() function that adds an explicit constraint as part of the model be easier? Then, type="cor" could error out if that constraint were not added?
It does warn, though. I just tested it in version 2.12.2. I notice in your OP you said you were running v2.11.5. Have you updated since then?
Sorry I wasn't clear there. The current version does warn during estimation (though not during summary() which would also be useful I think), but does not adjust df or any fit statistics.
My comment about "without any warning or adjustment" is regarding the proposed option of deprecating type="cor". The consequence of doing that, I think, is that users would just analyze the correlation matrix as a covariance, but then not even with the warning.
The create.vechsR() function in the metaSEM package can be used to create a model-implied covariance matrix ensuring that the diagonals are always ones. It does not use the mxConstraint() function in OpenMx. It treats the error variances of the dependent variables as computed parameters rather than as free parameters.
It was written to fit a correlation structure with weighted least squares in the context of meta-analysis. But it can be modified for maximum likelihood. Please see the attached examples.
I still have an installation of R v2.15.3, running OpenMx v1.3, on my 32-bit Windows 7 laptop. When I analyze the Holzinger-Swineford correlation matrix with
type="cov", I get this:When I analyze the correlation matrix with
type="cor", I get this (along with status BLUE):So, if we go back THIS far, OpenMx didn't handle correlation matrices correctly, and didn't warn about it, either.