| Attachment | Size |
|---|---|
 Data | 1.26 MB |
 Mplus code | 158.41 KB |
Hi,
Is it possible to estimate multilevel path/structural equation models with OpenMx?
If it is possible, is there an example code for multilevel mediation (path) model?
Attached files are the example of multilevel path model using Mplus.
Source: Heck, R. H., & Thomas, S. L. (2015). An introduction to multilevel modeling techniques: MLM and SEM approaches using Mplus. Routledge.
Hope I can do the same analysis using OpenMx.
Thank you in advance.
Soyoung.
Printer-friendly version
It is possible to estimate multilevel structural equations in OpenMx. This paper by Pritikin et al. discusses many of the details and provides some example scripts.
Many-Level Multilevel Structural Equation Modeling: An Efficient Evaluation Strategy
Here's a similar model (see attachment). You probably need to add/remove a few paths to make it identical to what you posted. I find it tedious to read Mplus model diagrams or tables.
Thank you all.
Two more questions.
Please check the attached file.
[1]
I revised the code, and i though I made the same model.
BUT I found these results are not identical in terms of estimates.
Why did these have different results?
Which Estimation method the OpenMx use?
Mplus uses Robust-ML.
[2]
I want to estimate indirect effects in Multilevel SEM.
How can I set a product term (indirect effect parameter) in OpenMx?
See imxRobustSE
Create an mxAlgebra and use mxCI or mxSE to request confidence intervals on it. See attached code.
Thank you so much for your kind response.
It was very helpful.
All the best,
Soyoung.
Hi Soyoung
Could you confirm that you were able to get the same estimates and goodness-of-fit statistics with the multilevel model in OpenMx when compared to Mplus?
Thanks
Mike
Hi, Mike.
I still could not get the identical results.
I am curious why these are different.
First, in terms of model fit statistics, two results were almost same.
BUT I don't think this example is suitable here, because these fit values are too perfect. It should be tested in the other data.
Second, in terms of estimates, I could not get the same estimates.
I'm trying to figure out why there are different results.
I am pretty sure that
imxRobustSE()does not correctly handle multilevel data; the matter of whether or not it does is currently a matter of disagreement among the OpenMx development team. I suggest you not rely too much on the robust SEs you get out of OpenMx.Soyoung, could you post the OpenMx script you're currently working from?
Hi Rob
Following code is based on what Dr.Pritikin provided.
You can find the data in post #1 .
Under x86_64 Linux/GNU, I substantively reproduce the OpenMx results you report with all 3 gradient-descent optimizers (script and output are attached).
Could you post the full output from MPlus? In particular, I'm curious if the -2logL matches.
Yes. Attachment is full output.
Mplus reports LL -40521.828.
(-2) * (-40521.828) = 81043.656
Thank you.
Soyoung.
Hi Soyoung, I notice that the estimator in MPlus is MLR, whereas in OpenMx it is FIML. Could you try it with ML (FIML) in Mplus to see if that changes the results?
Cheers
Mike
Hi Mike.
Attachment is the result of FIML,
but this is almost same as the result of MLR of Mplus,
and the estimates does not match the result of OpenMx.
I wonder why the two results are different...Where does the difference come from?
Is there a way to check local-maxima in OpenMx, by any chance?
Thank you.
Soyoung.
One way to debug this is to load the parameter vectors into the OpenMx model using omxSetParameters and then evaluate the fit at that point using mxRun(mxModel(model, mxComputeOnce('fitfunction','fit'))). If the Mplus fit matches at a specific parameter vector then the models are likely the same. If the fit doesn't match then the models are different.
I already tried that (see my post, presently #17, below). The fit values don't match.
Hi Soyoung
OpenMx does have facilities for testing for local minima, such as mxTryHard(). However, in this case it seems to me that Mplus is the software that might need it, because the fit function from OpenMx's estimation is indicating a much higher likelihood (41000.78 vs 40522.071) than is Mplus. You could perhaps try putting the OpenMx estimates into Mplus and fixing all the parameters and seeing what it thinks the answer is with those values. If the answers still don't agree, it indicates that the models are not the same, and we should try to figure out why they aren't.
Cheers
Mike
Good point Rob. I am not sure what Robust-ML means for multilevel models.
It appears that
imxRobustSE()does not calculate robust SEs in a manner theoretically correct for multilevel data (e.g., Freedman 2006).It also appears that what OpenMx and MPlus are doing in this case are different enough that their results shouldn't be compared to each other. See the attached script and output. When OpenMx calculates the fit at MPlus' solution, OpenMx returns a different fit from MPlus. Further, I can start OpenMx at MPlus' solution, and OpenMx still returns the same solution it has before (with a fit value of 82001.55).
I don't know OpenMx's multilevel feature very well, and I don't know MPlus at all, so I'm not sure what might explain the difference (or even if Soyoung's syntax is even fitting equivalent models in the two software packages). Perhaps a difference in how the two programs internally handle zero-variance exogenous regressors? Does MPlus drop normalizing constants from its loglikelihood?
I have been told that the Mplus software excludes any variable with no variance from analysis.
and Mplus does not drop normalizing constants.
In my eyes, OpenMx and Mplus are different in the manner of providing results.
In OpenMx, when there are level-1 manifest variables causes level-2 latent variables,
intercepts are showed at the lower level results.
In Mplus, when there are level-1 manifest variables causes level-2 latent variables,
intercepts are showed at the between level (upper level) results.
When I look at the mathematical formulas of the model, I feel easy to understand to treat the intercept at the upper level.
(probably because I got used to it)
Will the outcome differ depending on the level of handling the intercept?
Thank you again,
Soyoung.
> Will the outcome differ depending on the level of handling the intercept?
No and you can prove it by trying it both ways.
OK, there is an OpenMx bug. Try the model again with,
Hi Joshua
I am so pleased to see these estimates are almost same as the results of Mplus :)
Where are the results of indirect effects such as a*b1 and a*b2?
I am curious whether they are also same as the results of Mplus.
Thank you so much,
Soyoung.
Please never mind.
I am going to try! :)
Hello,
l'd like to confirm that I can get the same estimates.
and I also want to ask a question if I can ignore small difference of lbound/ubound.
When I ran the following code,
I got the following result.
|
Comparing the results of Mplus(Estimator=ML),
(attachment is the full output of Mplus).
Fit valus are same,
path coefficents are same,
BUT, lower bound / upper bound of indirect effects are not perfectly identical.
followings are from OpenMx,
and followings are from Mplus.
CONFIDENCE INTERVALS OF MODEL RESULTS Lower 2.5% Estimate Upper 2.5% IND1 -0.269 -0.139 -0.010 IND2 -0.252 -0.131 -0.009Can I ignore this difference?
Thank you,
Soyoung.
Use
summary(model, verbose=TRUE)to get detailed information about the CIs. I'm not sure if Mplus makes this information available for comparison, but that's not a fault of OpenMx.Hi,
I would like to ask about the other example of Multilevel SEM.
Now, I am trying 2-1-1 mediation model (using the attachment ex9.5.dat)
I checked the values of -2LnL are almost same; 4303.159 (OpenMx); 4303.158 (Mplus),
but some estimates are not identical.
If you look at the diagram file (attachment), a path and a intercept are different between two programs,
others are almost identical.
Could you please check if there is a way to get the same result?
I want to make this 2-1-1 model based on the article:
Preacher, K. J., Zhang, Z., & Zyphur, M. J. (2011). Alternative methods for assessing mediation in multilevel data: The advantages of multilevel SEM. Structural Equation Modeling, 18, 161-182.
Appendix of this article is here.
my 2-1-1 model is based on E. 2-1-1 model (MSEM) in the Appendix above.
Thank you in advance.
Soyoung.
P.S. following code is what I have run in OpenMx.
and i got the following result.
First, thank you for providing an example that runs nicely, and for clearly documenting the differences! This helps tremendously. Second, I have been able to reproduce the OpenMx side of the analysis and recover your estimates. The Mplus output is a bit brief and it doesn't seem to contain expected means or covariances - though these would seem likely to equal those of OpenMx because the model fit is the same. Sometimes expectations for the model implied means differ, and this may be the case here. I note the warning in the Mplus output:
and I'm wondering if some of the components of the expected mean are omitted because of this 'treatment.' The difference between the two expected means of Y under OpenMx vs Mplus is .251*1.595*-.13-(-.085*1.595*-.13) = -0.0696696, which is the same as the expected mean 'due' to the path from X to Mb to M to Y, .336*(1.595*-.13) = -0.0696696. So it seems that the 'treated as y-variable' translates into 'the expected mean of Y lacks the effect of the mean of X transmitted through Mb to M to Y, but includes the effect of the mean of X transmitted through Yb.' Just how reasonable this is I will defer to others - maybe the Muthens will chime in here ;).
Hope this helps!
Hi, Mike.
Thank you so much for your comments. :)
I have emailed the Mplus support team, and I have got the mail that the WARNING in Mplus is not error message. I think we can ignore the warning.
I have one more question.
I would like to ask about the way to handle level-1 variables.
Does OpenMx do latent variable decomposition?
If I want to see the latent variable decomposition results using OpenMx, what should I do?
If you look at the diagram file (attachment) variable M and Y are decomposed into two parts.
Mplus does a latent variable decomposition into a Within and Between part of M and Y.
There is some explanation about latent variable decomposition:
Please click here to view page 274 of Mplus User's Guide.
I would be happy, if I can find the same solution with two programs.
Hello,
This forum has been immensely helpful for me to set up my code properly for a multilevel path analysis I'm working on for my dissertation. I am running into issues when I compute my fit statistics with the following code:
The error message I get is the following:
I know this is a very complicated model, although I get the same error message when I break it down to only three variables. I am wondering if I am using the wrong reference model given that it's a two-level model or something that could be a similar easy fix. Thank you in advance for any help you can offer.
Best,
Gina
I have a feeling that
mxRefModels()wasn't designed to be used with multilevel models.Hello, I am new in this forum. I am very interested in OpenMx capabilities to analyze multilevel SEM models.
I have read this node to see examples of how I could implement my own model. The problem is that I don't know if the cases presented here would be appropriate for my study.
I want to evaluate a 2-2-1 mediation model. The independent variable X is "leadership", a latent variable with 16 indicators, grouped into 3 factors (second-order factorial model). The mediating variable M is "job satisfaction", a latent variable measured by 20 indicators. Both variables X and M were measured in a sample of 80 middle-level workers. Finally, the response variable Y is "customer satisfaction", latent variable with 6 indicators, measured in a large sample of approx. 9000 customers.
These data are nested. These are customers and workers of 8 hotels. Clients evaluate the hotel in general and five departments, from where we extract the sample of 80 middle-level workers.
The examples of multilevel mediation that I have seen so far have no latent variables in the upper levels. In fact, I have seen few examples 2-2-1. I don't know if I can build a model with these characteristics with OpenMx. If someone could show me an example, I would greatly appreciate it! :-)
I also wanted to ask if it would be better in my case, first to analyze the responses of the workers separately. Build a model with two levels for this group. In the first one, I would include the latent variables and in the second the variance explained by the structure of the data (workers nested in departments and hotels) would be collected. Once adjusted this model I could treat it as a submodel, as in the example of Pritikin (message 10 of this node), and join it to a new model with customers. Another option would be to calculate the factor scores of the latent variables X and M, and treat them as variables observed in a 2-2-1 model, in which the only latent variable would be in the first level and correspond to the customers. In the second level would be the observed variables X and M, and the grouping structure.
Please, I would be very grateful if you could guide me, since I have been looking for models similar to what I want to study for several weeks and I have not been lucky.