You are here

Fit Indices in OpenMx path modeling

13 posts / 0 new
Last post
HAMED's picture
Offline
Joined: 06/15/2012 - 02:29
Fit Indices in OpenMx path modeling

Hello,

I am using OpenMx for latent variable path modeling. I have faced two problems in the results of the summary() function, about which any help would be much appreciated:

1- When I use the "raw" data type in my mxData() function, some of the resulted fit indices (i.e., RMSEA, CFI, TLI, and Chi-square) returns "NA" as the output! However, when I change my data type to "cov" in the mxData() function, the problem is resolved and it gives me some non-"NA" values for those fit indices. I have tested this on the examples in the "OpenMxUserGuide" and I have seen the same problem there too. Is there any reason for this? or is it a bug in the OpenMx?

2- The summary() function reports very limited number of fit indices (i.e., chi-square, AIC, BIC, CFI, TLI, and RMSEA). However, many other common fit indices that reviewers usually ask for them are missing from the output of summary() (e.g., GFI, AGFI, SRMR, NFI, NNFI, 95% CI for RMSEA). I was wondering is there is any function/solution for calculating these missing fit indices in OpenMx?

Thank you,

mhunter's picture
Offline
Joined: 07/31/2009 - 15:26
We know about these issues

We know about these issues and I'm happy to say there is a solution for both.

For the first issue: when raw data is used, absolute fit indices (like RMSEA, CFI, TLI, and Chi-Square) require one or more null comparison models to be fit. The short answer is that OpenMx does not fit these null models automatically on raw data in the interest of speed. When covariance data is used, the null models are trivial and always fast to estimate, so we return the absolute fit indices in that case. The binary release of OpenMx does not yet have a single line of code solution to this issue. However, the svn version does have a a helper function to fit the null models in a single line, and then you can obtain absolute fit indices. In upcoming binary releases of OpenMx you will be able to type something like the following:

myModelFit <- mxRun(myModel)
myModelSat <- omxSaturatedModel(myModel, run=TRUE)
summary(myModelFit, SaturatedLikelihood=myModelSat)

For the second issue: OpenMx does return a limited number of fit indices. However, the semTools package has a function called fitMeasuresMx() which returns a vector of fit indices when given an mxModel. It has 37 elements the last time I checked. These include Chi-Square, CFI, TLI, NFI, NNFI, RFI, RNI, IFI, RMSEA (including confidence intervals thereon), SRMR, RMR, GFI, AGFI, PGFI, MFI, AIC, BIC, and so on. You should be able to simply obtain all the fit indices you want with code like the following:

# install.packages("semTools") # if needed
require(semTools)
myModelFit <- mxRun(myModel)
fitMeasuresMx(myModelFit)

Hope this helps!

HAMED's picture
Offline
Joined: 06/15/2012 - 02:29
Thank you so much for such a

Thank you so much for such a prompt and complete answer. It certainly helped.

Best

neale's picture
Offline
Joined: 07/31/2009 - 15:14
Null vs. saturated models

I agree with mhunter that the fit index issue with raw data is speed. However, it is fitting the saturated comparison model (every variance, covariance and mean or threshold is a free parameter) that is the problem for speed. The null model (diagonal covariance matrix, separate means or thresholds for every variable) can be fitted without optimization so it would be fast.

Unfortunately, most fit indices (chi-squared and those based on it) do need the saturated model fit.

tbates's picture
Offline
Joined: 07/31/2009 - 14:25
saturated model in the submodel case

hi all,
What is the correct saturated fit to return for a supermodel with submodels?
Just that supermodel, with it's contained models turned into saturated (or independence) models?

Currently the helper doesn't support these models.

I see that SEMtools handles multiple groups in its nullMx() and saturateMx() functions.
It does seem like core functionality to me, and best maintained inside OpenMx package.

Best, tim

ya's picture
ya
Offline
Joined: 11/28/2013 - 06:10
Hi, I am trying to get the

Hi,

I am trying to get the model fit indices and standardized estimates using semtools. The model terminated normally, but the error occurred when I ask for the fit indices and standardized estimates. Could you give me some suggestions please? Thank you very much.

Here is my code:

> model2=mxModel('model2',
+ mxData(HolzingerSwineford1939[,7:15],type='raw'),
+ mxMatrix(name='loadingregress',type='Full',nrow=12,ncol=12,
+ values=c(
+ 0,0,0,0,0,0,0,0,0,1,0,0,
+ 0,0,0,0,0,0,0,0,0,1,0,0,
+ 0,0,0,0,0,0,0,0,0,1,0,0,
+ 0,0,0,0,0,0,0,0,0,0,1,0,
+ 0,0,0,0,0,0,0,0,0,0,1,0,
+ 0,0,0,0,0,0,0,0,0,0,1,0,
+ 0,0,0,0,0,0,0,0,0,0,0,1,
+ 0,0,0,0,0,0,0,0,0,0,0,1,
+ 0,0,0,0,0,0,0,0,0,0,0,1,
+ 0,0,0,0,0,0,0,0,0,0,0,0,
+ 0,0,0,0,0,0,0,0,0,0,0,0,
+ 0,0,0,0,0,0,0,0,0,1,1,0
+ ),
+ free=c(
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,T,F,F,
+ F,F,F,F,F,F,F,F,F,T,F,F,
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,F,T,F,
+ F,F,F,F,F,F,F,F,F,F,T,F,
+
+ F,F,F,F,F,F,F,F,F,F,F,T,
+ F,F,F,F,F,F,F,F,F,F,F,T,
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,F,F,F,
+ F,F,F,F,F,F,F,F,F,T,T,F
+ ),
+ byrow=T),
+ mxMatrix(name='vcov',type='Symm',nrow=12,ncol=12,
+ values=c(
+ 1,0,0,0,0,0,0,0,0,0,0,0,
+ 0,1,0,0,0,0,0,0,0,0,0,0,
+ 0,0,1,0,0,0,0,0,0,0,0,0,
+ 0,0,0,1,0,0,0,0,0,0,0,0,
+ 0,0,0,0,1,0,0,0,0,0,0,0,
+ 0,0,0,0,0,1,0,0,0,0,0,0,
+ 0,0,0,0,0,0,1,0,0,0,0,0,
+ 0,0,0,0,0,0,0,1,0,0,0,0,
+ 0,0,0,0,0,0,0,0,1,0,0,0,
+ 0,0,0,0,0,0,0,0,0,1,1,0,
+ 0,0,0,0,0,0,0,0,0,1,1,0,
+ 0,0,0,0,0,0,0,0,0,0,0,1
+ ),
+ free=c(
+ T,F,F,F,F,F,F,F,F,F,F,F,
+ F,T,F,F,F,F,F,F,F,F,F,F,
+ F,F,T,F,F,F,F,F,F,F,F,F,
+ F,F,F,T,F,F,F,F,F,F,F,F,
+ F,F,F,F,T,F,F,F,F,F,F,F,
+ F,F,F,F,F,T,F,F,F,F,F,F,
+ F,F,F,F,F,F,T,F,F,F,F,F,
+ F,F,F,F,F,F,F,T,F,F,F,F,
+ F,F,F,F,F,F,F,F,T,F,F,F,
+ F,F,F,F,F,F,F,F,F,T,T,F,
+ F,F,F,F,F,F,F,F,F,T,T,F,
+ F,F,F,F,F,F,F,F,F,F,F,T
+ ),
+ byrow=T),
+ mxMatrix(name='filter',type='Full',nrow=9,ncol=12,
+ dimnames=list(
+ c(names(HolzingerSwineford1939[,7:15])),
+ c(names(HolzingerSwineford1939[,7:15]),'f1','f2','f3')
+ ),
+ values=c(
+ 1,0,0,0,0,0,0,0,0,0,0,0,
+ 0,1,0,0,0,0,0,0,0,0,0,0,
+ 0,0,1,0,0,0,0,0,0,0,0,0,
+ 0,0,0,1,0,0,0,0,0,0,0,0,
+ 0,0,0,0,1,0,0,0,0,0,0,0,
+ 0,0,0,0,0,1,0,0,0,0,0,0,
+ 0,0,0,0,0,0,1,0,0,0,0,0,
+ 0,0,0,0,0,0,0,1,0,0,0,0,
+ 0,0,0,0,0,0,0,0,1,0,0,0
+ ),
+ free=F,
+ byrow=T),
+ mxMatrix(name='means',type='Full',nrow=1,ncol=12,
+ dimnames=list(
+ c('means'),
+ c(names(HolzingerSwineford1939[,7:15]),'f1','f2','f3')
+ ),
+ values=c(0,0,0,0,0,0,0,0,0,0,0,0),
+ free=c(T,T,T,T,T,T,T,T,T,F,F,F),
+ byrow=T),
+ mxRAMObjective('loadingregress','vcov','filter','means')
+ )
> fit2=mxRun(model2)
Running model2
> fitMeasuresMx(fit2)
Error in mxFIMLObjective(covariance = "S", means = "M", dimnames = colnames(data@observed)) :
'dimnames' argument cannot be an empty vector
> standardizeMx(fit2)
Error in standardizeMxSingleGroup(object) :
trying to get slot "values" from an object of a basic class ("NULL") with no slots
> fitMeasuresMx(fit2)
Error in mxFIMLObjective(covariance = "S", means = "M", dimnames = colnames(data@observed)) :
'dimnames' argument cannot be an empty vector
>

martonandko's picture
Offline
Joined: 02/19/2015 - 06:55
fitMeasuresMx with matrix specification

I'm trying to get additional fit indexes for a univariateACE model based on Neale's script (http://openmx.psyc.virginia.edu/sites/default/files/UnivariateTwinAnalysis_MatrixRaw-3.R) using the above menthiond fitMeasuresMx funciont of semTools, but I get an error:
> fitMeasuresMx(myModelFit)
Error in FUN(X[[1L]], ...) :
no slot of name "objective" for this object of class "MxModel"

Is there a way to get these fit indexes for models specified using matrixes? Do I need to calculate the saturated model first? If so, what function should I use for this?

mhunter's picture
Offline
Joined: 07/31/2009 - 15:26
semTools out of date

The semTools package seems to be out of date with OpenMx 2.0. You should still be able to get many of the fit indices you want from the following.

myModelFit <- mxRun(myModel)
myModelSat <- mxRefModels(myModel, run=TRUE)
summary(myModelFit, refModels=myModelSat)

The mxRefModels function estimates saturated and independence models. Using summary like above should give you chi-square, AIC, BIC, CFI, TLI, and RMSEA (and I think 95% CI for RMSEA in OpenMx 2.0.1). We still don't offer support for GFI, AGFI, SRMR, NFI, and NNFI. There's nothing I'm aware of that's stopping this support. We've just been developing other features.

martonandko's picture
Offline
Joined: 02/19/2015 - 06:55
Additional fit indexes would be great

Thank you for your reply! Additional fit indexes in the summary would be highly appreciated I think by everyone.

mhunter's picture
Offline
Joined: 07/31/2009 - 15:26
Update

After a bit of digging, (1) I realized that the NNFI is another name for the TLI, (2) GFI and AGFI are not generally recommended because simulations have found they don't perform well (heavily influenced by sample size and very high Type I error rates), and (3) the SRMR only makes sense for models of covariance data because it doesn't have a good analog for missing data situations.

tbates's picture
Offline
Joined: 07/31/2009 - 14:25
For helpers use umxSummary and umxFitIndices

The umx package can help some. grab it with the usual install.packages("umx")

umxSummary will detect raw-data and run the sat and ind models.
umxFitIndices (beta) may have the additional fit stats you desire.

require(umx)
data(demoOneFactor)
manifests = c("x1", "x2", "x3", "x4", "x5")
m1 <- umxRAM("OneFactor", data = demoOneFactor,
    umxPath("g", to = manifests),
    umxPath(var = manifests),
    umxPath(var = "g", fixedAt = 1)
)
umxSummary(m1, showEstimates = "std")
umxFitIndices(m1)
plot(m1)

http://openmx.psyc.virginia.edu/thread/765#comment-6757

http://openmx.psyc.virginia.edu/node/1301

HAMED's picture
Offline
Joined: 06/15/2012 - 02:29
Thank you for your

Thank you for your information. "umx" sounds like very interesting and helpful for reporting the results of a SEM model in a paper. However, the problem is that when I insert the following line in R:
source("http://timbates.wdfiles.com/local--files/start/umx.lib.R")
It gives me this error:
Error in mxOption(model, "Calculate Hessian", "No") : object 'm1' not found

Am I missing anything?

tbates's picture
Offline
Joined: 07/31/2009 - 14:25
fixed now

That was a bit of debugging code running a model ... commented out now. Sorry.