How to Extract Model Fit from Individual Converged Models in mxBootstrap?

Posted on
No user picture. Vivian Joined: 04/30/2025

Hi All,

I'm running the bivariate CCC model, and while some estimates have 95% confidence intervals, I needed to run bootstrapping using mxBootstrap to obtain the rest. However, when I ran 20,000 bootstrap replications, only about 1% of the models converged. I'm wondering if it's possible to extract model fit statistics from each converged model to assess whether the resulting 95% CIs are robust. Additionally, is there any way to improve the convergence rate? I also tried using OKStatusCodes = c(0, 1, 6) with 1,000 replications, but the percentage of converged models did not increase. I appreciate any suggestions or advice that might help me troubleshoot this issue. Thank you!

The script:

mxOption(fitbACiE_yafu, "Major iterations", 5000)
mxOption(fitbACiE_yafu, "Function precision", "1e-8")
yafuboot3 <- mxBootstrap(model=fitbACiE_yafu,replications=20000)
betacoef_yafu <- summary(yafuboot3, boot.quantile=c(0.025,0.975))

Output:

> betacoef_yafu <- summary(yafuboot3, boot.quantile=c(0.025,0.975))
Warning:
Only 1% of the bootstrap replications converged. Accuracy is much less than the 20000 replications requested 
> betacoef_yafu
Summary of twobAiCpE 
 
free parameters:
           name matrix row col    Estimate   Std.Error         2.5%       97.5%
1  thremarevery  threG   1   1 0.989956741 0.331644623  0.375534495 1.816599693
2  thremarhighy  threG   1   2 1.251162004 0.355536362  0.553216862 1.943450874
3          VA11     VA   1   1 0.845478557 0.025729225  0.790479741 0.885122870
4          VC22     VC   2   2 0.170762861 0.076068573  0.081992846 0.346701526
5          VE11     VE   1   1 0.154521443 0.025729225  0.114877130 0.209520259
6          VE22     VE   2   2 0.098970777 0.059415843  0.011642173 0.163718242
7           bIP     pB   2   1 0.854556381 0.071956177  0.722317386 0.952086449
8           bIA     bA   1   1 0.046552902 0.014977678  0.018908745 0.086883725
9           bPA     bA   1   2 0.016813070 0.015598169 -0.015814559 0.047590341
10          bIS     bS   1   1 0.246246949 0.059606609  0.116710150 0.341973820
11          bPS     bS   1   2 0.402846910 0.070245771  0.241117982 0.506784160

To obtain confidence intervals re-run with intervals=TRUE

Model Statistics: 
               |  Parameters  |  Degrees of Freedom  |  Fit ( units)
       Model:             11                   3472               NA
   Saturated:             NA                     NA               NA
Independence:             NA                     NA               NA
Number of observations/statistics: 1183/3483

Constraint 'Var1' contributes 2 observed statistics. 

timestamp: 2025-04-21 18:30:29 
Wall clock time: 156517.61 secs 
OpenMx version number: 2.21.12 
Need help?  See help(mxSummary) 

bIP is the beta pathway from the first outcome (thremarevery) to the second outcome (thremarhighy). bIA and bPA are the coefficients of age for each outcome, and bIS and bPS are the coefficients of sex for each outcome.

I also try the following parameters but the outputs are all NULL.

yafuboot3_try <- mxBootstrap(model = fitbACiE_yafu, replications = 200, only.successful = TRUE, save = TRUE)

yafuboot3_try$boot.models #output is NULL

 

yafuboot3_try3 <- mxBootstrap(fitbACiE_yafu, replications = 1000, onlyCompute = FALSE, OKStatusCodes = c(0, 1, 6))

yafuboot3_try3$bootData #output is NULL

-------------------------------------------------------------------------------------------------

OpenMx version: 2.21.12 [GIT v2.21.12]

R version: R version 4.4.1 (2024-06-14 ucrt)

Platform: x86_64-w64-mingw32

Default optimizer: SLSQP

NPSOL-enabled?: No

OpenMP-enabled?: No

Replied on Tue, 05/06/2025 - 12:41
Picture of user. AdminRobK Joined: 01/24/2014

You probably want to do something like this: yafuboot3@compute@output$raw.  That should give you what you want.

You may also want to set the appropriate option as follows, before you bootstrap: mxOption(NULL, "Status OK",as.statusCode(c(0,1,5,6))).

> I also try the following parameters but the outputs are all NULL.

> yafuboot3_try <- mxBootstrap(model = fitbACiE_yafu, replications = 200, only.successful = TRUE, save = TRUE)

> yafuboot3_try$boot.models #output is NULL

>

> yafuboot3_try3 <- mxBootstrap(fitbACiE_yafu, replications = 1000, onlyCompute = FALSE, OKStatusCodes = c(0, 1, 6))

> yafuboot3_try3$bootData #output is NULL

I'm not sure what you were expecting here?

Replied on Tue, 05/06/2025 - 19:42
No user picture. Vivian Joined: 04/30/2025

In reply to by AdminRobK

Hi Robert,

Thank you so much for solving my problem! The code you provided worked well.

I had added the parameters: only.successful = TRUE, save = TRUE, or onlyCompute = FALSE in an attempt to retrieve the model fit for each run, but that didn’t work. However, they weren’t correct.