# Versions 0.9.0 vs. 0.5.2

11 posts / 0 new
Offline
Joined: 10/08/2009 - 22:37
Versions 0.9.0 vs. 0.5.2

Hello all,

I am fitting some variance known models (akin to meta-analysis). To handle models with possible covariates, I use a RAM model. The results based on 0.5.2 are comparable to those based on other statistical packages. After upgraded to 0.9.0, the results look odd.

The followings are an example. The -2LL of the 0.5.2 and 0.9.0 versions are 27.79916 and 29.02565, respectively. Any suggestions are highly appreciated. Thanks.

Regards,
Mike

yi <- c(-0.264,-0.230,0.166,0.173,0.225,0.291,0.309,0.435,0.476,0.617,0.651,0.718,0.740,0.745,0.758,0.922,0.938,0.962,1.522,1.844)
vi <- c(0.086,0.106,0.055,0.084,0.071,0.078,0.051,0.093,0.149,0.095,0.110,0.054,0.081,0.084,0.087,0.103,0.113,0.083,0.100,0.141)
my.df <- cbind(yi,vi)
test <- mxModel("test", mxData(observed=my.df, type="raw"),
mxMatrix("Zero", ncol=1, nrow=1, free=FALSE, values=0, name="A"),
mxMatrix("Full", ncol=1, nrow=1, free=FALSE, values=0, labels="data.vi", name="V"),
mxMatrix("Full", ncol=1, nrow=1, free=TRUE, values=0.1, lbound=0.0000001, name="Tau"),
mxMatrix("Iden", ncol=1, nrow=1, name="F", dimnames=list(c("yi"), c("yi"))),
mxMatrix("Full", ncol=1, nrow=1, free=T, values=0, name="M"),
mxAlgebra(V+Tau, name="S"),
mxRAMObjective("A", "S", "F", "M")
)
summary(mxRun(test))

## In model 'test' NPSOL returned a non-zero status code 1. The final iterate satisfies the optimality conditions to the accuracy requested, but the sequence of iterates has not yet converged. NPSOL was terminated because no further improvement could be made in the merit function (Mx status GREEN).

Offline
Joined: 07/31/2009 - 15:24
RAM and raw data is evaluated

RAM and raw data is evaluated as a FIML model. We had a bug in the -2 LL calculation that was corrected in 0.9.0. This bug affected all the 0.5.x releases (unless my memory serves me wrong). Could someone run this model with 0.3.3 and report the results? I'm not making a claim as to which value is correct, only that a third datapoint will help. Old releases can be downloaded at openmx.psych.virginia.edu/packages.

Offline
Joined: 10/08/2009 - 22:37

Thanks for the prompt reply. Attached are the output for 0.3.3. They are the same as those in 0.5.2. I have also run the analyses (but with a different model parametrization) in Mx 1.70a and Mplus 6. They are similar to those in 0.5.2.

Mx:
Parameter Estimate Std Error Lower 95% Upper 95%

  2     0.579035E+00  0.105101E+00  0.373036E+00  0.785034E+00
1     0.131520E+00  0.737999E-01 -0.131281E-01  0.276168E+00


Mplus 6:
MODEL RESULTS
Two-Tailed
Estimate S.E. Est./S.E. P-Value
Means
U 0.579 0.107 5.406 0.000
Variances
U 0.132 0.078 1.689 0.091

## In model 'test' NPSOL returned a non-zero status code 1. The final iterate satisfies the optimality conditions to the accuracy requested, but the sequence of iterates has not yet converged. NPSOL was terminated because no further improvement could be made in the merit function (Mx status GREEN).

Offline
Joined: 07/30/2009 - 14:03
Thanks for the input, Mike!

Thanks for the input, Mike! Any chance we can add your script to our test suite so that we can catch any future discrepancies?

Offline
Joined: 07/31/2009 - 15:14
Hmmm. Seems like a problem

Hmmm. Seems like a problem concerning definition variables, but there is a definition variable example in the test suite which still runs fine. However, that example does not use a continuous definition variable.

Offline
Joined: 07/31/2009 - 15:12
First, thanks for running

First, thanks for running your code with the new version. Finding problems like these really helps the project.

I've attached full output for runs under 0.5.2 and 0.9.0. While I can't identify the problem, I find the difference in numbers of iterations across the two versions interesting. The successful 0.5.2 run goes for 9 major iterations, while 0.9.0 runs for three. This is interesting because it means that 0.9.0 (a) got stuck, as indicated by the non-invertible hessian at iteration 3 and the invalid standard errors, but (b) successfully ran through two complete iterations before hitting the problem. Its unfortunate that we get the same NPSOL message for both runs, though the hessian/SE differences should indicate a problem to users who look at them.

Ryne

Offline
Joined: 07/31/2009 - 15:10
Thanks for catching this so

Thanks for catching this so quickly.

These discrepancies resulted from a bug in the latest optimization for FIML RAM models with definition variables.

The latest source revision, r1419, contains the fix.

Would you mind if we add your script to the test suite to make sure we catch any similar bugs that crop up in the future?

Offline
Joined: 07/31/2009 - 15:24
This has been fixed in OpenMx

This has been fixed in OpenMx 0.9.1. Thanks for spotting the bug so quickly.

Offline
Joined: 10/08/2009 - 22:37
Thanks a lot for all the

Thanks a lot for all the responses and fixes. The latest version (r1420) works fine!

Please feel free to use the script. The original data were adopted from Joop Hox's "Multilevel analysis: Techniques and Applications" at http://www.ats.ucla.edu/stat/hlm/examples/ma_hox/chapter8.htm

By the way, I built OpenMx r1420 from source. When I ran the analysis, it still indicates "0.9.0-1417." It would be better to reflect the correct versions being used.

Offline
Joined: 07/31/2009 - 15:24
When OpenMx is compiled, the

When OpenMx is compiled, the version number is hard coded into the Makefile and the DESCRIPTION file. I've thought about using the command-line program 'svnversion' to extract the subversion revision number. It would be easier to place this number into the Makefile, and more difficult to place this number into the DESCRIPTION file. The only schemes we have come up with are way too complicated for the problem. So we just tell OpenMx developers to realize they are compiling from source and ignore the version number.

Offline
Joined: 10/08/2009 - 22:37
Thanks. It is not a big

Thanks. It is not a big issue.