Fix: T71040 / T58983 Ocean Modifier crashes at high resolutions

This is not as much a fix as a work around, but given the real
involves replacing how we build fftw, it is not eligible for 2.83
which is in BCON3 already.

The root of the issue lies with (how we build) fftw3

The first issue is: fftw does not build with MSVC, there are other
dependencies that are not compatible with MSVC and for those we
build the libraries required with mingw64, same for fftw

The second issue is: for reasons unknown we really really really
liked all deps to link statically so wherever possible we did so.

Now during the building of the fftw it linked a few symbols from
libgcc (which we do not ship) like __chkstk_ms, for which we passed
some flags to stop generating calls to it. Problem solved! There
is no way this could possibly turn around and bite us in the rear.

fast forward to today mystery crashes that look like a race condition.

What is happening is, we tell the linker that each thread will require
a 2-megabyte stack, now if every thread immediately allocated 2 megs,
that be 'rough' on the memory usage. So, what happens is (for all apps
not just blender), 2 megs are reserved but not backed by any real memory
and the first page is allocated for use by the stack, now as the stack
grows, it will eventually grow out of that first page, and end up in
an area that has not been allocated yet, to deal with that the allocated
page is followed by a guard page, someone touches the guard page it's
time to grow the stack!

Meanwhile in FFTW is it's doing substantial allocation using alloca
(up to 64 kb) on the stack, jumping over the guard page, and ending
up in reserved but not yet committed memory, causing an access violation.

Now if you think, that doesn't sound right! something should have
protected us from that! You are correct! That thing was __chkstk_ms
which we disabled.

Given we do not want a dependency on libgcc while building with MSVC
the proper solution is to build fftw as a shared library which will
statically link any bits and pieces it needs, however that change
is a little bit too big to be doing in BCON3.

So as a work around, we change the size the stack grows from 8k to
68k which gives fftw a little bit more wiggle room to keep it out
of trouble most of the time.

Note this only sidesteps the issue, this may come up again if the
conditions are just right, and a proper solution will need to be
implemented for 2.90.

1 files changed, 1 insertions, 1 deletions

diff --git a/build_files/cmake/platform/platform_win32.cmake b/build_files/cmake/platform/platform_win32.cmake
index 1ecde8635c8..77652511416 100644
--- a/build_files/cmake/platform/platform_win32.cmake
+++ b/build_files/cmake/platform/platform_win32.cmake
@@ -166,7 +166,7 @@ if(MSVC_VERSION GREATER 1914 AND NOT MSVC_CLANG)
   set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /JMC")
 endif()
 
-set(PLATFORM_LINKFLAGS "${PLATFORM_LINKFLAGS} /SUBSYSTEM:CONSOLE /STACK:2097152 /INCREMENTAL:NO ")
+set(PLATFORM_LINKFLAGS "${PLATFORM_LINKFLAGS} /SUBSYSTEM:CONSOLE /STACK:2097152,70656 /INCREMENTAL:NO ")
 set(PLATFORM_LINKFLAGS_RELEASE "/NODEFAULTLIB:libcmt.lib /NODEFAULTLIB:libcmtd.lib /NODEFAULTLIB:msvcrtd.lib")
 set(PLATFORM_LINKFLAGS_DEBUG "${PLATFORM_LINKFLAGS_DEBUG} /IGNORE:4099 /NODEFAULTLIB:libcmt.lib /NODEFAULTLIB:msvcrt.lib /NODEFAULTLIB:libcmtd.lib")