Hi,

I have been following the other post about latent class analysis using EasyMx https://openmx.ssri.psu.edu/comment/reply/4282/7227. I still have some questions and concerns. My goal is to run a two component finite mixture model (aka Latent Profile analysis with two classes).

Is the optimization being done gradient descent or some other algorithm like EM algorithm. I would like to be using the EM algorithm.

If Easy MX is not using the EM Algorithm is there a way to use the EM algorithm with EasyMx? Or is there a way to set up a latent profile analysis using the EM Algorithm. Thank you for your help!

OpenMx offers mxComputeEM, but we don't have any demonstration scripts to show how to use it for latent profile analysis.

If I am using mxExpectationMixture() and mxFitFunctionML() to construct a finite mixture model, which kind algorithm I am applying? I assume it still should be sorts of iterative methods, but may I know more details, like the name or steps? Thanks in advance.

The default optimizer is used. That's usually CSOLNP which does gradient descent.

Got it. Thank you!

Note of clarification: in some disciplines, "gradient descent" means "steepest descent", i.e. referring to algorithms that use only the first derivatives of the objective function. None of CSOLNP, SLSQP, or NPSOL is a steepest-descent optimizer. Rather, all 3 are quasi-Newton optimizers that use the BFGS approximation of the objective-function Hessian.

Yeah. All of the test scripts that use

`mxComputeEM()`

are IRT models that use the IFA (item factor analysis) module. But it's possible to use`mxComputeEM()`

with an arbitrary MxAlgebra, isn't it?Thanks for your feedback!

Is the general process the same? I build my model using pathways (as opposed to matrix algebra) then I just need to use mxComputeEM() to execute the optimization as opposed to something like mxExpectationMixture() and mxFitFunctionML(). I guess I am struggling to see where mxComputeEM fits in. Or is the process different and I actually wouldn't need to specify the model using pathways ? Thanks for your help!

In order to use mxComputeEM() my model needs to be specified in terms of matrix algebras as opposed to the pathways? Is that correct?

Knowing about compute plans would probably be useful for both Keighly and Jin. For the most commonly-used analyses done in OpenMx, the MxModel's compute plan is created automatically, without user involvement. Jin, you can see what the default compute plan would be, in the case of ML estimation and requesting confidence intervals, with

and inspect the compute steps of the plan. They are, in order:

1. MxComputeGradientDescent, which is the primary quasi-Newton optimization to find point estimates.

2. MxComputeConfidenceInterval, which is a series of additional optimizations to find the confidence limits of profile-likelihood confidence intervals.

3. MxComputeNumericDeriv, during which the fitfunction gradient and Hessian are numerically calculated using central differences and several iterations of Richardson extrapolation.

4. MxComputeStandardError, during which standard errors of the point estimates are estimated from the diagonals of the inverted Hessian.

5. MxComputeHessianQuality, during which the condition number of the Hessian is computed.

6. MxComputeReportDeriv, during which the results of the numeric derivatives are reported to the frontend.

7. MxComputeReportExpectation, during which the MxModel's expectation object is populated with its results from the backend.

So, to use

`mxComputeEM()`

, the syntax might be something likein order to replace the gradient-descent step with an EM step, and then, put

`plan`

into your MxModel. There is probably more to it, though, since I do not think all of the default steps are compatible with EM. I've never actually used`mxComputeEM()`

myself, so I'll have to ask Joshua, who implemented it, for the details.The first thing I want to know is what constitutes the E step in this model? Typically you'd create an mxAlgebra to perform the E step with fixed=TRUE. Then CompteEM can recompute this algebra with mxComputeOnce.

The E step for a latent profile model is the calculating of the subject-specific posterior class probabilities where k is the number of classes. These probabilities are a function of the prior class probabilities (mixing proportions) and the subject normal densities/likelihoods which are a function of the class means and standard deviations, as shown below:

P(class=k|i, p_k, L_k) = p_k * L_ik / sum_k=1^m (p_k * L_ik)

So if I am understanding correctly, I need to create an MxAlgebra object which specifies the algebraic equation for these subject specific posterior probabilities. If that is correct, I have a few practical questions about creating this MxAlgebra (please stop me if I am way off base!) ... below.

I have created the prior probabilities using an MxMatrix object and I know I can use MxMatrix objects in the MxAlgebra ... so that is not an issue. What I am unsure of is how to refer to or call the data points in the likelihood (L_ik) as well as the means and standard deviations. My means and standard deviations are specified using MxPathways. So, should I refer to them in the equation by their labels? Can I refer to the data points in the likelihood using data.x (this is a guess)?

Thank you!

I think the most efficient way to proceed is to exchange working code. Can you reply with your preliminary code attached and comments where you have specific questions?

Dr. Kirk,

Thank you very much! It's really very helpful. I need to add this page as my reference.

There are some bugs in the v2.12.x that prevent this kind of model from working. However, things are working better in the master branch. Here's an example,

I just want to point out a crucial detail concerning Joshua's script: it gets different results from an MxModel that uses

`mxExpectationMixture()`

and`mxFitFunctionML()`

. See the syntax below (which also avoids the label-switching problem by ensuring that class 2 is always the "responder" class):Hi -- Forgive a total newbie who has not used this site before if I have done things the wrong way, and offended any local etiquette... In particular I may be entirely in the wrong place, I realise.

I am using OpenMx to fit mixtures of normal distributions to data, and with univariate data it seemed to work well. Essentially, I started with the example of a mixture of growth models on p.89 of the User Guide (v2.17.2) and slowly altered things until I got the results I wanted with univariate data.

All was fine until I started working on some bivariate data and I kept running into a new error which said, "Error in runHelper(model, frontendStart, intervals, silent, suppressWarnings, : ModelC2.fitfunction: component class1.fitfunction must be in probability units"

This I didn't understand at all as the example talks about fitting proportions and then making them into probabilities later. More frustratingly I have scoured every detail of my model and cannot find where on earth it is different from the GMM model, and neither can I find anything on how to make the fitfunction be in probability units.

At this point I should confess that I find the internal workings of OpenMx a little opaque, despite having used LISREL and laavan for many years (but they don't do mixture modelling!). So forgive once more.

I think I have attached a file which generates dummy data to fit two bivariate normal distributions to some data (and the plot shows they are clearly a mixture of two components).

Many thanks for any help you can offer,

Chris

PS OpenMx version: 2.17.4 [GIT v2.17.4]

R version: R version 3.6.0 (2019-04-26)

Platform: x86_64-w64-mingw32

Default optimizer: CSOLNP

NPSOL-enabled?: No

OpenMP-enabled?: No

Hello Chris - my undergrad prof from Bedford College no less! I have kicked your model around a bit and I don't see anything obvious yet. The error message makes me think that the likelihood of one of the classes (probably the first) is negative at some point, but both class1 and class2 run fine on their own, e.g., mxRun(ModelC2$class1).

Not sure if I ever stumped you on a question in class, but if I did you may be enjoying mentor's revenge :). I'll bring it up with the rest of the team, hopefully in tomorrow's meeting.

Best wishes

Mike

When using the mixture model you need to have

`mxFitFunctionML(vector=TRUE)`

in the component models. So changeto

and similarly for the class2 model. Everything should work fine then.

This means that you can't run them individually, but they will work in the mixture.

Good stuff. I think we should try to improve the error message... "Very sorry, but it looks like you are trying to use a mixture distribution function, but at least some of the components of the model don't have the required argument to mxFitFunctionML(), i.e., they don't say mxFitFunctionML(vector=TRUE)." This is probably some variety of check prior to attempting to evaluate the likelihood.