You are here

Bootstrap Likelihood Ratio Test

17 posts / 0 new
Last post
matthewcgraham's picture
Offline
Joined: 02/20/2018 - 01:17
Bootstrap Likelihood Ratio Test
AttachmentSize
Binary Data GrahamM_BLRT.R6.34 KB

I'm interested in using the new bootstrap function in mxCompare to evaluate nested growth mixture models (GMM) using the Bootstrap Likelihood Ratio Test (BLRT), but am a uncertain how to do so. For example, in the attached code building on the GMM example provided in the OpenMx documentation, say I wanted to compare a three-class model to a two-class model. How would I conduct the BLRT?

OpenMx version: 2.8.3 [GIT v2.8.3]
R version: R version 3.4.1 Patched (2017-08-09 r73082)
Platform: x86_64-w64-mingw32
Default optimiser: CSOLNP
NPSOL-enabled?: Yes
OpenMP-enabled?: No

jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
read the man page?

Did you try mxCompare(gmm3Fit, gmm2Fit, boot=TRUE)? You should find detailed documentation in the mxCompare man page. Let us know if we might improve the documentation.

matthewcgraham's picture
Offline
Joined: 02/20/2018 - 01:17
Bootstrap Function Does Not Seem to Function when Comparing GMMs

mxCompare (gmm3Fit, gmm2Fit, boot = TRUE) returns the error message:

Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘genericGenerateData’ for signature ‘"NULL"’
Error: Cannot bootstrap null model 'GMM2Class'

OpenMx version: 2.8.3 [GIT v2.8.3]
R version: R version 3.4.1 Patched (2017-08-09 r73082)
Platform: x86_64-w64-mingw32
Default optimiser: CSOLNP
NPSOL-enabled?: Yes
OpenMP-enabled?: No

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
expectation

I'm pretty sure an MxModel needs to contain an "expectation" object in order for OpenMx to know how to generate data based on it. Look into using mxExpectationMixture() and mxFitFunctionML().

Edit: that's probably something the documentation needs to say...

jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
oh! mixture

Wait, how is this suppose to work? Will you resample each mixture component independently or do you want to resample the data once and have the same resampled data used by each mixture component? I may need to write some code to make this work.

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
parametric

The bootstrap LRT uses parametric bootstrapping, remember. OpenMx has to generate data under the null model from a parametric distribution (multivariate normal).

jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
null distribution

Yeah but it sounds like Matthew is trying to determine the best number of mixture components for a non-parametric (case resampling) bootstrap of the data.

matthewcgraham's picture
Offline
Joined: 02/20/2018 - 01:17
Struggling to Adapt GMM Code

I've been attempting to adapt the GMM code to incorporate mxExpectationMixture() and mxFitFunctionML(), but am having little success. Any insight would be greatly appreciated.
MCG

jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
false start

mxGenerateData is not implemented for mxExpectationMixture. So that wouldn't help anyway. You probably want to treat the mixture proportions as parameters to be estimated and not included as part of the bootstrap distribution. So you don't want the mixture proportions to affect the bootstrap data generation.

You'll have to do this by hand. Write a loop around the whole script. Generate a random weight vector. Pass it to mxData(.., weight=weight). Execute the whole script (inner part of the loop). Aggregate and summarize output from mxCompare.

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
mxGenerateData
mxGenerateData is not implemented for mxExpectationMixture.

That's a rather serious oversight...!

Edit: And I'm pretty sure you're still confused about what Matthew is trying to accomplish here.

jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
goal?

Yeah, that's the question. What Matthew is trying to accomplish?

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
I am pretty sure he's simply

I am pretty sure he's simply trying to do the bootstrap LRT via mxCompare(), to test a null model with (say) 2 mixture components against an alternative model with (say) 3 mixture components. To do so, he needs to be able to generate data under the null model, fit the null model to the generated data, fit the alternative model to the generated data, store the difference in -2logL, and repeat the process for B replications. Unfortunately, the generic for data-generation currently has no method defined for MxExpectationMixture, so OpenMx cannot do what he wants in an automated fashion via mxCompare().

AdminNeale's picture
Offline
Joined: 03/01/2013 - 14:09
High priority

I would say that implementing the data generation under mxExpectationMixture is now a high priority. Apologies to the OP for our oversight!

matthewcgraham's picture
Offline
Joined: 02/20/2018 - 01:17
Yes, Exactly

AdminRobK describes what I'm trying to do (well, reviewers want me to do).
Thank you all for your support,
MCG

jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
gotcha

Alright, you need version v2.9.2-11-g49758bdfa or better of OpenMx. The script to perform the desired analysis is attached.

File attachments: 
jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
windows?

You use windows, correct? I built a windows binary package for you, OpenMx_2.9.2-17.zip.

matthewcgraham's picture
Offline
Joined: 02/20/2018 - 01:17
Yes, I use Windows

Yes, and the script seems to work correctly in both the example and my current research project.

Thank you all so much for your attention and support!

Regards,
MCG