Login

News

Change in type="cor" behavior?

Posted on
Picture of user. bwiernik Joined: 01/30/2014
Forums
OpenMx 2.0 Discussion
In OpenMx v. 2.11.5, the fit statistics output is the same whether type="cov" or type="cor". In a previous version of OpenMx, I recall that the manifest variable variances would be subtracted from the number of degrees of freedom and observed statistics, and the fit statistics were calculated using the reduced degrees of freedom. Now, the number of observed statistics number is reduced, but the degrees of freedom and fit statistics are not similarly adjusted.

Also, previously the warning about standard errors and fit statistics for type="cor" being potentially incorrect with the Steiger (1980) citation was printed when summary.MxModel() was called as when mxRun() was called. Now the warning only prints with mxRun()

For example, using the HS.ability.data example in help(mxModel), the output for type="cov" is:
Model Statistics:
| Parameters | Degrees of Freedom | Fit (-2lnL units)
Model: 21 24 10970.90
Saturated: 45 0 10928.42
Independence: 9 36 11882.01
Number of observations/statistics: 301/45

chi-square: χ² ( df=24 ) = 42.48505, p = 0.01138111
Information Criteria:
| df Penalty | Parameters Penalty | Sample-Size Adjusted
AIC: -5.514954 84.48505 87.79687
BIC: -94.485600 162.33436 95.73430
CFI: 0.9798547
TLI: 0.9697821 (also known as NNFI)
RMSEA: 0.05058496 [95% CI (0.01638508, 0.07931089)]
Prob(RMSEA <= 0.05): 0.4530891

The output for type="cor" is:
Model Statistics:
| Parameters | Degrees of Freedom | Fit (-2lnL units)
Model: 21 15 1788.898
Saturated: 45 -9 1746.413
Independence: 9 27 2700.000
Number of observations/statistics: 301/36

chi-square: χ² ( df=24 ) = 42.48505, p = 0.01138111
Information Criteria:
| df Penalty | Parameters Penalty | Sample-Size Adjusted
AIC: 12.48505 84.48505 87.79687
BIC: -43.12161 162.33436 95.73430
CFI: 0.9798547
TLI: 0.9697821 (also known as NNFI)
RMSEA: 0.05058496 [95% CI (0.01638508, 0.07931089)]
Prob(RMSEA <= 0.05): 0.4530891

I'm not sure which version this behavior changed in.

I'm not sure when the change occurred. Was this an intentional change? If so, was it determined that the unadjusted df/fit statistics are more likely to be correct (the problems with doing covariance structure analyses with correlations acknowledged)?

Replied on Mon, 03/25/2019 - 15:39
Picture of user. jpritikin Joined: 05/23/2012

I'm searching back. There has been no change in behavior since v2.9.9 (2018-05-11).
Log in or register to post comments
Replied on Wed, 03/27/2019 - 15:51
Picture of user. jpritikin Joined: 05/23/2012

I installed v2.6.9 (2016-07-28). Cov has observed statistics: 45, degrees of freedom: 24. Cor has observed statistics: 36, degrees of freedom: 15. These look the same. However, I do notice differences in the chi-squared and Information Criteria fit stats,

For cov:

chi-square:  X2 ( df=24 ) = 88.7134,  p = 2.351412e-09
Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:       40.71340               130.7134                       NA
BIC:      -48.25725               208.5627                 141.9627

For cor:

chi-square:  X2 ( df=24 ) = 88.7134,  p = 2.351412e-09
Information Criteria: 
      |  df Penalty  |  Parameters Penalty  |  Sample-Size Adjusted
AIC:      58.713401               130.7134                       NA
BIC:       3.106747               208.5627                 141.9627

There are lots of differences here compared to current output. What specifically do you think is incorrect?

Log in or register to post comments
Replied on Fri, 03/29/2019 - 10:13
Picture of user. tbates Joined: 07/31/2009

When I run a RAM model with type = "cor" data , I get warning message:


In FUN(X[[i]], ...) :
OpenMx does not yet correctly handle mxData(type='cor') standard errors and fit statistics.
See Steiger (1980), "Tests for
comparing elements of a correlation matrix".

This is with the current OpenMx: 2.12.2.267 [GIT v2.12.2-267-g3cbee07]

Log in or register to post comments
Replied on Fri, 04/05/2019 - 10:36
Picture of user. tbates Joined: 07/31/2009

An upcoming version of OpenMx will constrain the variances of the manifest portion of the `S` matrix to 1 if `type="cor"` in RAM models. Secondly, we'll likely stop allowing cor in non-RAM models, with a note to the user to assert `type = cov` and add constraints where they need to be based on their matrix usage, or warn people that they need to take care of this.
Log in or register to post comments
Replied on Tue, 04/09/2019 - 15:46
Picture of user. AdminRobK Joined: 01/24/2014

Automating the constraint on the diagonal of the model-expected covariance/correlation matrix is going to be trickier than I thought when I volunteered to tackle this bug. A model of `type="RAM"` has no MxMatrix or MxAlgebra for its model-expected matrix. The constraint would have to exist solely in the backend. We do have a precedent for creating new constraints in the backend: we do that to implement the constrained representation of the confidence-limit problem. But, those constraints don't persist to be returned to the frontend, and they don't do things like adjust the degrees-of-freedom of the model. Maybe we should just deprecate `type="cor"`?
Log in or register to post comments
Replied on Tue, 04/09/2019 - 18:46
Picture of user. bwiernik Joined: 01/30/2014

Analyses with correlation matrices are really common, and returning to the previous behavior where the sole change for type="cor" is to display a warning about standard errors and adjusting the degrees of freedom and associated fit indices would be much preferable I think to users just analyzing correlation matrices as covariance matrices without any warning or adjustment.

Rather than doing the diagonal constraint implicitly, would adding a new mxCorrelationConstraint() function that adds an explicit constraint as part of the model be easier? Then, type="cor" could error out if that constraint were not added?

Log in or register to post comments
Replied on Tue, 04/16/2019 - 10:40
Picture of user. AdminRobK Joined: 01/24/2014

and returning to the previous behavior where the sole change for type="cor" is to display a warning about standard errors and adjusting the degrees of freedom and associated fit indices would be much preferable I think to users just analyzing correlation matrices as covariance matrices without any warning or adjustment.

It _does_ warn, though. I just tested it in version 2.12.2. I notice in your OP you said you were running v2.11.5. Have you updated since then?

Log in or register to post comments
Replied on Tue, 04/16/2019 - 12:20
Picture of user. bwiernik Joined: 01/30/2014

Sorry I wasn't clear there. The current version does warn during estimation (though not during summary() which would also be useful I think), but does not adjust df or any fit statistics.

My comment about "without any warning or adjustment" is regarding the proposed option of deprecating type="cor". The consequence of doing that, I think, is that users would just analyze the correlation matrix as a covariance, but then not even with the warning.

Log in or register to post comments
Replied on Wed, 04/10/2019 - 03:07
Picture of user. Mike Cheung Joined: 10/08/2009

The create.vechsR() function in the metaSEM package can be used to create a model-implied covariance matrix ensuring that the diagonals are always ones. It does not use the mxConstraint() function in OpenMx. It treats the error variances of the dependent variables as computed parameters rather than as free parameters.

It was written to fit a correlation structure with weighted least squares in the context of meta-analysis. But it can be modified for maximum likelihood. Please see the attached examples.

File attachments
cor_structure.pdf
Log in or register to post comments
Replied on Wed, 04/17/2019 - 10:16
Picture of user. AdminRobK Joined: 01/24/2014

I still have an installation of R v2.15.3, running OpenMx v1.3, on my 32-bit Windows 7 laptop. When I analyze the Holzinger-Swineford correlation matrix with `type="cov"`, I get this:

observed statistics: 45
estimated parameters: 23
degrees of freedom: 22
-2 log likelihood: 2050.228
saturated -2 log likelihood: 1714.86
number of observations: 301
chi-square: 335.3673
p: 7.71944e-58
Information Criteria:
df Penalty Parameters Penalty Sample-Size Adjusted
AIC: 291.3673 381.3673 NA
BIC: 209.8109 466.6309 393.6879
CFI: 0.6698406
TLI: 0.4597392
RMSEA: 0.2175366

When I analyze the correlation matrix with `type="cor"`, I get this (along with status BLUE):

observed statistics: 36
estimated parameters: 23
degrees of freedom: 13
-2 log likelihood: 2050.228
saturated -2 log likelihood: 1714.86
number of observations: 301
chi-square: 335.3673
p: 9.242367e-64
Information Criteria:
df Penalty Parameters Penalty Sample-Size Adjusted
AIC: NA NA NA
BIC: NA NA NA
CFI: 0.6698406
TLI: 0.4597392
RMSEA: 0.2175366

So, if we go back THIS far, OpenMx didn't handle correlation matrices correctly, and didn't warn about it, either.
Log in or register to post comments
  Login

News