Login

News

item factor analysis performance issues across R versions

Posted on
No user picture. cffalk2 Joined: 10/06/2025
Forums
OpenMx General Help

I'll try to describe the problem briefly. Can try to provide reprex if this sounds novel.

I have some code based on this paper (https://doi.org/10.1007/s11136-021-02873-7) that fits item factor analysis models. With version R 3.6.2, OpenMx 2.17.2, Windows 10, the models were quick to estimate (e.g., 3 seconds). I can still run them in that environment. I recently revisited the code with R 4.5.1, OpenMx 2.22.10 on the same hardware and estimation is painfully slow (e.g., almost 4 minutes).

Wondering if anyone can point me in the right direction for figuring out why and/or if there are any known performance issues. Going back to older R versions on the same machine, this seems to be an issue with anything R 4.0 (e.g., 4.0.5) and later. I see something in release notes for OpenMx versions (e.g., 2.18) around that time about CSOLNP no longer being the default optimizer in part due to performance issues (changing the optimizer in this case does not seem to make a difference). I also understand that it might be that many on the development team are not using Windows. Though, if I install Ubuntu in a virtual machine and go with the latest R and OpenMx version, I seem to encounter the same performance issues.

Any advice is greatly appreciated. Understand if you folks would ask for more info.

 

Replied on Fri, 02/06/2026 - 09:45
Picture of user. AdminNeale Joined: Mar 01, 2013

Hi Carl

Sorry to be so late responding!  Can you please send a reproducible example?

It seems like this is a problem with R 4.x.  The optimizers are running in the C++ backend so should not be affected by the R change.  Possibly memory management by R has changed in a bad way.  We will investigate.

Log in or register to post comments
Replied on Tue, 04/28/2026 - 11:33
No user picture. cffalk2 Joined: Oct 06, 2025

Thank you, Mike, sorry I'm slow to reply now.

I am attempting to attach an RMarkdown's pdf output (.rmd is not allows for attachments?). This is for the new-ish version of R (4.5.2) and OpenMx (2.22.10).

I was going to then also run this under the older environment, but it turns out I would need to jump through hoops to install rmarkdown. So... I'll paste code and relevant part of the output below. Note this is with R 3.6.2 and OpenMx 2.17.2.

There is a tiny bit of benchmarking in here. Under the newer version, it's something like 12 seconds on average to fit some models. Under the older version, microbenchmark switches to milliseconds because it's only something like 0.56 seconds per model on average.

Thank you for taking a look!

All of the code:

# some utility functions I'm using... be nice, I never got around to polishing these much...
# which was what I was hoping to do.
# devtools::install_github("falkcarl/mpirt")
library(mpirt)

library(microbenchmark)

library(psych) # uh, for some data
data(bfi)
dat <- bfi[,1:25]
dat.openmx<-mxFactor(dat,levels=1:6)
ni<-ncol(dat.openmx) # number of items

# This should default to GRMP, but with 1st order polynomial = Graded response model
fitk0<-fitMP(dat.openmx, se=FALSE)

## Code I would like to run; but by the second fitted model it's already too slow...
## with the newer version of R and OpenMx. With older version it's fast enough.

## Use simulated annealing to obtain MP-based model
## set up matrix to store possible polynomial orders
# kmat<-newkmat(0,3,ni) #k = 3 -> 7th order polynomial
# 
# # Same priors used (tau=-1, variance of 500) as in GRMP paper (Falk, 2020)
# set.seed(1938)
# sa.result<-simAnneal(dat.openmx,kmat,itemtype=rep("grmp",ni),itermax=1200,
#                      inittemp=5,type="aic",pvar=500,taumean=-1,temptype="logarithmic",
#                      items=1)

## Example of a slow-ish model
# This will return an MxModel object, but fitting is a bit "slow"

fitmodel <- fitMP(dat.openmx, k = c(1,rep(0,24)), fit = TRUE, itemtype = rep("grmp",ni), 
                  priors = TRUE, startimat = fitk0$itemModel@matrices$item, qpoints = 101, 
                  qwidth = 5, se = FALSE)

microbenchmark(
  fitMP(dat.openmx, k = c(1,rep(0,24)), fit = TRUE, itemtype = rep("grmp",ni), 
                  priors = TRUE, startimat = fitk0$itemModel@matrices$item, qpoints = 101, 
                  qwidth = 5, se = FALSE),
  times = 5
)

devtools::session_info()

And some of the relevant output:

> microbenchmark(
+   fitMP(dat.openmx, k = c(1,rep(0,24)), fit = TRUE, itemtype = rep("grmp",ni), 
+                   priors = TRUE, startimat = fitk0$itemModel@matrices$item, qpoints = 101, 
+                   qwidth = 5, se = FALSE),
+   times = 5
+ )
Running Model with 152 parameters
Running Model with 152 parameters               
Running Model with 152 parameters
Running Model with 152 parameters              
Running Model with 152 parameters
Unit: milliseconds
                                                                                                                                                                                       expr
 fitMP(dat.openmx, k = c(1, rep(0, 24)), fit = TRUE, itemtype = rep("grmp",      ni), priors = TRUE, startimat = fitk0$itemModel@matrices$item,      qpoints = 101, qwidth = 5, se = FALSE)
      min       lq     mean   median      uq      max neval
 549.1338 557.2917 561.0982 559.5095 562.394 577.1618     5
> 
> devtools::session_info()
- Session info -----------------------------------------------------------------------------------------------------------------
 setting  value                       
 version  R version 3.6.2 (2019-12-12)
 os       Windows 10 x64              
 system   x86_64, mingw32             
 ui       RStudio                     
 language (EN)                        
 collate  English_Canada.1252         
 ctype    English_Canada.1252         
 tz       America/New_York            
 date     2026-04-28                  

- Packages ---------------------------------------------------------------------------------------------------------------------
 package        * version   date       lib source           
 assertthat       0.2.1     2019-03-21 [1] CRAN (R 3.6.0)   
 backports        1.1.5     2019-10-02 [1] CRAN (R 3.6.1)   
 cachem           1.1.0     2024-05-16 [1] standard (@1.1.0)
 callr            3.4.3     2020-03-28 [1] CRAN (R 3.6.3)   
 cli              2.0.2     2020-02-28 [1] CRAN (R 3.6.3)   
 codetools        0.2-16    2018-12-24 [1] CRAN (R 3.6.0)   
 colorspace       1.4-1     2019-03-18 [1] CRAN (R 3.6.0)   
 crayon           1.3.4     2017-09-16 [1] CRAN (R 3.6.0)   
 desc             1.2.0     2018-05-01 [1] CRAN (R 3.6.0)   
 devtools         2.3.0     2020-04-10 [1] CRAN (R 3.6.3)   
 digest           0.6.25    2020-02-23 [1] CRAN (R 3.6.3)   
 dplyr            0.8.4     2020-01-31 [1] CRAN (R 3.6.2)   
 ellipsis         0.3.0     2019-09-20 [1] CRAN (R 3.6.1)   
 fansi            0.4.1     2020-01-08 [1] CRAN (R 3.6.2)   
 fastmap          1.2.0     2024-05-15 [1] standard (@1.2.0)
 fs               1.3.1     2019-05-06 [1] CRAN (R 3.6.0)   
 furrr          * 0.1.0     2018-05-16 [1] CRAN (R 3.6.0)   
 future         * 1.16.0    2020-01-16 [1] CRAN (R 3.6.2)   
 ggplot2        * 3.2.1     2019-08-10 [1] CRAN (R 3.6.1)   
 globals          0.12.5    2019-12-07 [1] CRAN (R 3.6.1)   
 glue             1.4.1     2020-05-13 [1] CRAN (R 3.6.3)   
 gtable           0.3.0     2019-03-25 [1] CRAN (R 3.6.0)   
 htmltools        0.4.0     2019-10-04 [1] CRAN (R 3.6.2)   
 httpuv           1.5.2     2019-09-11 [1] CRAN (R 3.6.1)   
 ifaTools       * 0.21      2020-02-04 [1] CRAN (R 3.6.2)   
 later            1.0.0     2019-10-04 [1] CRAN (R 3.6.2)   
 lattice          0.20-38   2018-11-04 [2] CRAN (R 3.6.2)   
 lazyeval         0.2.2     2019-03-15 [1] CRAN (R 3.6.0)   
 lifecycle        0.1.0     2019-08-01 [1] CRAN (R 3.6.1)   
 listenv          0.8.0     2019-12-05 [1] CRAN (R 3.6.2)   
 magrittr         1.5       2014-11-22 [1] CRAN (R 3.6.0)   
 MASS             7.3-51.4  2019-03-31 [2] CRAN (R 3.6.2)   
 Matrix           1.2-18    2019-11-27 [2] CRAN (R 3.6.2)   
 memoise          2.0.1     2021-11-26 [1] standard (@2.0.1)
 microbenchmark * 1.5.0     2024-09-04 [1] standard (@1.5.0)
 mime             0.9       2020-02-04 [1] CRAN (R 3.6.2)   
 mnormt           1.5-6     2020-02-03 [1] CRAN (R 3.6.2)   
 mpirt          * 0.1.1     2025-11-16 [1] local            
 multcomp         1.4-12    2020-01-10 [1] CRAN (R 3.6.2)   
 munsell          0.5.0     2018-06-12 [1] CRAN (R 3.6.0)   
 mvtnorm          1.0-12    2020-01-09 [1] CRAN (R 3.6.0)   
 nlme             3.1-142   2019-11-07 [2] CRAN (R 3.6.2)   
 OpenMx         * 2.17.2    2020-02-15 [1] CRAN (R 3.6.2)   
 parallelly       1.45.1    2025-07-24 [1] CRAN (R 3.6.2)   
 pillar           1.4.3     2019-12-20 [1] CRAN (R 3.6.2)   
 pkgbuild         1.0.6     2019-10-09 [1] CRAN (R 3.6.2)   
 pkgconfig        2.0.3     2019-09-22 [1] CRAN (R 3.6.1)   
 pkgload          1.0.2     2018-10-29 [1] CRAN (R 3.6.0)   
 plyr             1.8.5     2019-12-10 [1] CRAN (R 3.6.2)   
 prettyunits      1.1.1     2020-01-24 [1] CRAN (R 3.6.0)   
 processx         3.4.2     2020-02-09 [1] CRAN (R 3.6.2)   
 promises         1.1.0     2019-10-04 [1] CRAN (R 3.6.2)   
 ps               1.3.2     2020-02-13 [1] CRAN (R 3.6.2)   
 psych          * 1.9.12.31 2020-01-08 [1] CRAN (R 3.6.2)   
 purrr            0.3.3     2019-10-18 [1] CRAN (R 3.6.2)   
 R6               2.4.1     2019-11-12 [1] CRAN (R 3.6.2)   
 Rcpp             1.0.3     2019-11-08 [1] CRAN (R 3.6.3)   
 remotes          2.1.1     2020-02-15 [1] CRAN (R 3.6.2)   
 reshape2       * 1.4.3     2017-12-11 [1] CRAN (R 3.6.0)   
 rlang            0.4.6     2020-05-02 [1] CRAN (R 3.6.3)   
 rpf            * 1.0.2     2020-02-19 [1] CRAN (R 3.6.2)   
 rprojroot        1.3-2     2018-01-03 [1] CRAN (R 3.6.0)   
 rsconnect        0.8.16    2019-12-13 [1] CRAN (R 3.6.2)   
 rstudioapi       0.11      2020-02-07 [1] CRAN (R 3.6.2)   
 sandwich         2.5-1     2019-04-06 [1] CRAN (R 3.6.0)   
 scales           1.1.0     2019-11-18 [1] CRAN (R 3.6.2)   
 sessioninfo      1.1.1     2018-11-05 [1] CRAN (R 3.6.0)   
 shiny          * 1.4.0     2019-10-10 [1] CRAN (R 3.6.2)   
 stringi          1.4.6     2020-02-17 [1] CRAN (R 3.6.2)   
 stringr          1.4.0     2019-02-10 [1] CRAN (R 3.6.0)   
 survival         3.1-8     2019-12-03 [2] CRAN (R 3.6.2)   
 testthat         2.3.2     2020-03-02 [1] CRAN (R 3.6.3)   
 TH.data          1.0-10    2019-01-21 [1] CRAN (R 3.6.2)   
 tibble           2.1.3     2019-06-06 [1] CRAN (R 3.6.0)   
 tidyr            1.0.2     2020-01-24 [1] CRAN (R 3.6.0)   
 tidyselect       1.0.0     2020-01-27 [1] CRAN (R 3.6.2)   
 usethis          1.6.1     2020-04-29 [1] CRAN (R 3.6.3)   
 vctrs            0.2.3     2020-02-20 [1] CRAN (R 3.6.2)   
 withr            2.1.2     2018-03-15 [1] CRAN (R 3.6.0)   
 xtable           1.8-4     2019-04-21 [1] CRAN (R 3.6.0)   
 zoo              1.8-7     2020-01-10 [1] CRAN (R 3.6.2)
File attachments
mpirt_example_newR.pdf
Log in or register to post comments
Replied on Fri, 05/01/2026 - 12:02
Picture of user. AdminRobK Joined: Jan 24, 2014

I am attempting to attach an RMarkdown's pdf output (.rmd is not allows for attachments?).

Sorry about that.  We should tweak the set of allowable file extensions for attachments.  Although, I bet it would work if you changed the extension from '.rmd' to '.txt' or '.rmd.txt'.

I have not actually run your script yet, but I did look through all the release announcements for OpenMx versions 2.17.2 to 2.22.10, and I did not see anything that might explain the performance regression.  Sorry!  I might have some ideas later, once I grok what your code does.

 

Log in or register to post comments