You are here

Status of parallelization on Windows

9 posts / 0 new
Last post
bwiernik's picture
Offline
Joined: 01/30/2014 - 19:39
Status of parallelization on Windows

I was wondering what the status of supporting multicore systems on Windows. I thought that several compilers were available that supported OpenMP on Windows, and I was wondering if OpenMP-supporting builds of the OpenMX Windows binary were on a development timeline. (Note that this area is outside my wheelhouse, so I might be mistaken about the obstacle to Windows multicore support.)

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
OpenMP

Good question. For as long as I have been involved in OpenMx development, we have built OpenMx binaries with CRAN's official Windows toolchain, which has included gcc v4.6.3. We have never been able to build thread-safe OpenMP-enabled Windows binaries--they invariably crash. I have personally built a multihreaded Windows binary, run a parallelized loop under gdb, and watched as the current thread overran its array limit. Our consensus is that there is a compiler bug.

I have several times tentatively raised the possibility of trying a newer version of gcc, or perhaps a different optimizer (maybe Intel's), but have done so with more than a little trepidation. Per the R Installation and Administration Manual (emphasis in original):

We have found that the build process for R is quite sensitive to the choice of tools: please follow our instructions exactly, even to the choice of particular versions of the tools. The build process for add-on packages is somewhat more forgiving, but we recommend using the exact toolset at first, and only substituting other tools once you are familiar with the process.

This appendix contains a lot of prescriptive comments. They are here as a result of bitter experience. Please do not report problems to the R mailing lists unless you have followed all the prescriptions.

In other words, it may take quite a bit of trial-and-error to get a different compiler to work (if it ever does). OpenMP-enabled Windows binaries are not currently a development priority, though we would be eager to hear from anyone who can set up a build environment that can produce thread-safe such builds.

mhunter's picture
Offline
Joined: 07/31/2009 - 15:26
Newer gcc

Rob,

Note that the latest toolchains use a newer gcc. From the official site you linked.

R 3.3.0 and later use a toolchain based on gcc 4.9.3 and mingw-w64 v3, put together by Jeroen Ooms and others. See the project page for details.

They also use a new CXX standard. Maybe this is worth trying again?

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
Oops

Oops. "v4.6.3" was a typo on my part. "v4.9.3" is correct.

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
gcc
bwiernik's picture
Offline
Joined: 01/30/2014 - 19:39
That's really exciting news!

That's really exciting news!

bwiernik's picture
Offline
Joined: 01/30/2014 - 19:39
The Rtools 40 Windows

The Rtools 40 Windows toolchain now uses gcc 8.30, so I was wondering if there was any developer interest in trying to compile a multithread build of Windows OpenMx again?

AdminRobK's picture
Offline
Joined: 01/24/2014 - 12:15
already tried, and failed

See my post in another thread. In summary: I already tried to build a multithreaded OpenMx Windows binary with the current toolchain, and the binary is still not thread-safe. Disappointing!

Per this comment on github, OpenMP appears not to be a good choice in the first place for multithreading an R package under Windows.

bwiernik's picture
Offline
Joined: 01/30/2014 - 19:39
Ah, that's unfortunate.

Ah, that's unfortunate. Thanks for the update!