You are here

mxCI results in 2.18.1 and 2.19.1 are different

3 posts / 0 new
Last post
shufai's picture
Offline
Joined: 10/08/2020 - 02:48
mxCI results in 2.18.1 and 2.19.1 are different

I found that the confidence interval computed by adding mxCI in 2.19.1 are different from those in 2.18.1.

This is the code from the example of mxCI:

# From the example of mxCI()
 
# generate data
covariance <- matrix(c(1.0, 0.5, 0.5, 1.0), 
    nrow=2, 
    dimnames=list(c("a", "b"), c("a", "b")))
 
data <- mxData(covariance, "cov", numObs=100)
 
# create an expected covariance matrix
expect <- mxMatrix("Symm", 2, 2,
    free=TRUE,
    values=c(1, .5, 1),
    labels=c("var1", "cov12", "var2"),
    name="expectedCov")
 
# request 95 percent confidence intervals   
ci <- mxCI(c("var1", "cov12", "var2"))
 
# specify the model
model <- mxModel(model="Confidence Interval Example",
    data, expect, ci,
    mxExpectationNormal("expectedCov", dimnames=c("a", "b")),
    mxFitFunctionML())
 
# run the model 
results <- mxRun(model, intervals=TRUE)
 
# view confidence intervals
print(summary(results)$CI)

These are the intervals from 2.18.1 with mxVersion() output, ran on a fresh installation of R 4.0.5 with only the packages required by OpenMx installed. 2.18.1 compiled from source:

> mxVersion()
OpenMx version: 2.18.1 [GIT v2.18.1]
R version: R version 4.0.5 (2021-03-31)
Platform: x86_64-w64-mingw32
Default optimizer: SLSQP
NPSOL-enabled?: No
OpenMP-enabled?: No
> print(summary(results)$CI)
        lbound estimate   ubound note
var1  0.759576    0.990 1.323856     
cov12 0.304192    0.495 0.752543     
var2  0.759576    0.990 1.323856 

These are the intervals from 2.19.1, with mxVersion() output:

> mxVersion()
OpenMx version: 2.19.1 [GIT v2.19.1]
R version: R version 4.0.5 (2021-03-31)
Platform: x86_64-w64-mingw32
Default optimizer: SLSQP
NPSOL-enabled?: No
OpenMP-enabled?: No
> print(summary(results)$CI)
         lbound estimate    ubound note
var1  0.7595122    0.990 1.3243122     
cov12 0.3039181    0.495 0.7526147     
var2  0.7595122    0.990 1.3243122     

Are the differences due to a change in the implementation, the optimization method, or both? The summary results, with verbose = TRUE, suggest that the method is the same (Neale-Miller-1997):

2.18.1 Results:

CI details:
  parameter  side    value      fit diagnostic statusCode            method      var1     cov12      var2
1      var1 lower 0.759576 173.0632    success         OK neale-miller-1997 0.7595760 0.3798989 0.9320089
2      var1 upper 1.323856 173.0632    success         OK neale-miller-1997 1.3238564 0.6619067 1.0734424
3     cov12 lower 0.304192 173.0632    success         OK neale-miller-1997 0.8712903 0.3041920 0.8712904
4     cov12 upper 0.752543 173.0632    success         OK neale-miller-1997 1.2200943 0.7525430 1.2200942
5      var2 lower 0.759576 173.0632    success         OK neale-miller-1997 0.9320089 0.3798989 0.7595760
6      var2 upper 1.323856 173.0632    success         OK neale-miller-1997 1.0734424 0.6619067 1.3238564

2.19.1 Results:

CI details:
  parameter  side     value      fit diagnostic statusCode            method      var1     cov12      var2
1      var1 lower 0.7595122 173.0658    success         OK neale-miller-1997 0.7595122 0.3799756 0.9318342
2      var1 upper 1.3243122 173.0730    success         OK neale-miller-1997 1.3243122 0.6597274 1.0739407
3     cov12 lower 0.3039181 173.0758    success         OK neale-miller-1997 0.8726362 0.3039181 0.8726362
4     cov12 upper 0.7526147 173.0651    success         OK neale-miller-1997 1.2194169 0.7526147 1.2194169
5      var2 lower 0.7595122 173.0658    success         OK neale-miller-1997 0.9318342 0.3799756 0.7595122
6      var2 upper 1.3243122 173.0730    success         OK neale-miller-1997 1.0739407 0.6597274 1.3243122

The fit values are the same for 2.18.1, but are different in 2.19.1. If the method is the same, are the differences due to a change in the optimization method?

-- Shu Fai

jpritikin's picture
Offline
Joined: 05/24/2012 - 00:35
constrained optimization

From the release notes:

SLSQP no longer ignores the value of mxOption "Feasibility tolerance" for non-linear constraints. That is, in prior releases, SLSQP would always prefer to eliminate constraint violations completely regardless of the loss in fit. Now SLSQP will correctly permit "Feasibility tolerance" worth of constraint violation in exchange for improved fit.

shufai's picture
Offline
Joined: 10/08/2020 - 02:48
Thanks!

Many thanks for your prompt reply! My bad. I checked the release note but overlooked that part. I successfully reproduced, in 2.19.1, the old 2.18.1 SLSQP results (to the 5th or 6th decimal places) in 2.19.1 by setting the option "Feasibility tolerance".