mxCI results in 2.18.1 and 2.19.1 are different

Posted on Fri, 04/23/2021 - 23:32

shufai Joined: 10/08/2020

Forums

I found that the confidence interval computed by adding mxCI in 2.19.1 are different from those in 2.18.1.

This is the code from the example of mxCI:

# From the example of mxCI()


# generate data

covariance <- matrix(c(1.0, 0.5, 0.5, 1.0),

    nrow=2,

    dimnames=list(c("a", "b"), c("a", "b")))
data <- mxData(covariance, "cov", numObs=100)
# create an expected covariance matrix

expect <- mxMatrix("Symm", 2, 2,

    free=TRUE,

    values=c(1, .5, 1),

    labels=c("var1", "cov12", "var2"),

    name="expectedCov")
# request 95 percent confidence intervals

ci <- mxCI(c("var1", "cov12", "var2"))
# specify the model

model <- mxModel(model="Confidence Interval Example",

    data, expect, ci,

    mxExpectationNormal("expectedCov", dimnames=c("a", "b")),

    mxFitFunctionML())
# run the model

results <- mxRun(model, intervals=TRUE)

# view confidence intervals print(summary(results)$CI)

These are the intervals from 2.18.1 with mxVersion() output, ran on a fresh installation of R 4.0.5 with only the packages required by OpenMx installed. 2.18.1 compiled from source:

> mxVersion() OpenMx version: 2.18.1 [GIT v2.18.1] R version: R version 4.0.5 (2021-03-31) Platform: x86_64-w64-mingw32 Default optimizer: SLSQP NPSOL-enabled?: No OpenMP-enabled?: No > print(summary(results)$CI) lbound estimate ubound note var1 0.759576 0.990 1.323856 cov12 0.304192 0.495 0.752543 var2 0.759576 0.990 1.323856

These are the intervals from 2.19.1, with mxVersion() output:

> mxVersion() OpenMx version: 2.19.1 [GIT v2.19.1] R version: R version 4.0.5 (2021-03-31) Platform: x86_64-w64-mingw32 Default optimizer: SLSQP NPSOL-enabled?: No OpenMP-enabled?: No > print(summary(results)$CI) lbound estimate ubound note var1 0.7595122 0.990 1.3243122 cov12 0.3039181 0.495 0.7526147 var2 0.7595122 0.990 1.3243122

Are the differences due to a change in the implementation, the optimization method, or both? The summary results, with verbose = TRUE, suggest that the method is the same (Neale-Miller-1997):

2.18.1 Results:
CI details: parameter side value fit diagnostic statusCode method var1 cov12 var2 1 var1 lower 0.759576 173.0632 success OK neale-miller-1997 0.7595760 0.3798989 0.9320089 2 var1 upper 1.323856 173.0632 success OK neale-miller-1997 1.3238564 0.6619067 1.0734424 3 cov12 lower 0.304192 173.0632 success OK neale-miller-1997 0.8712903 0.3041920 0.8712904 4 cov12 upper 0.752543 173.0632 success OK neale-miller-1997 1.2200943 0.7525430 1.2200942 5 var2 lower 0.759576 173.0632 success OK neale-miller-1997 0.9320089 0.3798989 0.7595760 6 var2 upper 1.323856 173.0632 success OK neale-miller-1997 1.0734424 0.6619067 1.3238564

2.19.1 Results:
CI details: parameter side value fit diagnostic statusCode method var1 cov12 var2 1 var1 lower 0.7595122 173.0658 success OK neale-miller-1997 0.7595122 0.3799756 0.9318342 2 var1 upper 1.3243122 173.0730 success OK neale-miller-1997 1.3243122 0.6597274 1.0739407 3 cov12 lower 0.3039181 173.0758 success OK neale-miller-1997 0.8726362 0.3039181 0.8726362 4 cov12 upper 0.7526147 173.0651 success OK neale-miller-1997 1.2194169 0.7526147 1.2194169 5 var2 lower 0.7595122 173.0658 success OK neale-miller-1997 0.9318342 0.3799756 0.7595122 6 var2 upper 1.3243122 173.0730 success OK neale-miller-1997 1.0739407 0.6597274 1.3243122

The fit values are the same for 2.18.1, but are different in 2.19.1. If the method is the same, are the differences due to a change in the optimization method?

-- Shu Fai

Replied on Sat, 04/24/2021 - 10:11

jpritikin Joined: 05/23/2012

constrained optimization

From the release notes:

SLSQP no longer ignores the value of mxOption "Feasibility tolerance" for non-linear constraints. That is, in prior releases, SLSQP would always prefer to eliminate constraint violations completely regardless of the loss in fit. Now SLSQP will correctly permit "Feasibility tolerance" worth of constraint violation in exchange for improved fit.

Log in or register to post comments

Replied on Sat, 04/24/2021 - 14:10

shufai Joined: 10/08/2020

Thanks!

Many thanks for your prompt reply! My bad. I checked the release note but overlooked that part. I successfully reproduced, in 2.19.1, the old 2.18.1 SLSQP results (to the 5th or 6th decimal places) in 2.19.1 by setting the option "Feasibility tolerance".