CMake: Refactor external dependencies handling

This is a more correct fix to the issue Brecht was fixing in D6600.

While the fix in that patch worked fine for linking it broke ASAN
runtime under some circumstances.
For example, `make full debug developer` would compile, but trying
to start blender will cause assert failure in ASAN (related on check
that ASAN is not running already).

Top-level idea: leave it to CMake to keep track of dependency graph.

The root of the issue comes to the fact that target like "blender" is
configured to use a lot of static libraries coming from Blender sources
and to use external static libraries. There is nothing which ensures
order between blender's and external libraries. Only order of blender
libraries is guaranteed.

It was possible that due to a cycle or other circumstances some of
blender libraries would have been passed to linker after libraries
it uses, causing linker errors.

For example, this order will likely fail:

  libbf_blenfont.a libfreetype6.a libbf_blenfont.a

This change makes it so blender libraries are explicitly provided
their dependencies to an external libraries, which allows CMake to
ensure they are always linked against them.

General rule here: if bf_foo depends on an external library it is
to be provided to LIBS for bf_foo.
For example, if bf_blenkernel depends on opensubdiv then LIBS in
blenkernel's CMakeLists.txt is to include OPENSUBDIB_LIBRARIES.

The change is made based on searching for used include folders
such as OPENSUBDIV_INCLUDE_DIRS and adding corresponding libraries
to LIBS ion that CMakeLists.txt. Transitive dependencies are not
simplified by this approach, but I am not aware of any downside of
this: CMake should be smart enough to simplify them on its side.
And even if not, this shouldn't affect linking time.

Benefit of not relying on transitive dependencies is that build
system is more robust towards future changes. For example, if
bf_intern_opensubiv is no longer depends on OPENSUBDIV_LIBRARIES
and all such code is moved to bf_blenkernel this will not break
linking.

The not-so-trivial part is change to blender_add_lib (and its
version in Cycles). The complexity is caused by libraries being
provided as a single list argument which doesn't allow to use
different release and debug libraries on Windows. The idea is:

- Have every library prefixed as "optimized" or "debug" if
  separation is needed (non-prefixed libraries will be considered
  "generic").

- Loop through libraries passed to function and do simple parsing
  which will look for "optimized" and "debug" words and specify
  following library to corresponding category.

This isn't something particularly great. Alternative would be to
use target_link_libraries() directly, which sounds like more code
but which is more explicit and allows to have more flexibility
and control comparing to wrapper approach.

Tested the following configurations on Linux, macOS and Windows:

- make full debug developer
- make full release developer
- make lite debug developer
- make lite release developer

NOTE: Linux libraries needs to be compiled with D6641 applied,
otherwise, depending on configuration, it's possible to run into
duplicated zlib symbols error.

Differential Revision: https://developer.blender.org/D6642

18 files changed, 151 insertions, 7 deletions

diff --git a/intern/audaspace/CMakeLists.txt b/intern/audaspace/CMakeLists.txt
index 1972a5fc94c..254fb5ecf19 100644
--- a/intern/audaspace/CMakeLists.txt
+++ b/intern/audaspace/CMakeLists.txt
@@ -50,6 +50,11 @@ if(NOT WITH_SYSTEM_AUDASPACE)
       extern_sdlew
     )
   endif()
+else()
+  list(APPEND LIB
+    ${AUDASPACE_C_LIBRARIES}
+    ${AUDASPACE_PY_LIBRARIES}
+  )
 endif()
 
 if(WITH_PYTHON)
@@ -60,6 +65,10 @@ if(WITH_PYTHON)
     intern/AUD_PyInit.cpp
     intern/AUD_PyInit.h
   )
+  list(APPEND LIB
+      ${PYTHON_LINKFLAGS}
+      ${PYTHON_LIBRARIES}
+  )
   if(NOT WITH_SYSTEM_AUDASPACE)
     list(APPEND LIB
       audaspace-py
diff --git a/intern/cycles/blender/CMakeLists.txt b/intern/cycles/blender/CMakeLists.txt
index 09379bf2017..b6ecab28555 100644
--- a/intern/cycles/blender/CMakeLists.txt
+++ b/intern/cycles/blender/CMakeLists.txt
@@ -49,11 +49,15 @@ set(LIB
   cycles_render
   cycles_subd
   cycles_util
+
+  ${PYTHON_LINKFLAGS}
+  ${PYTHON_LIBRARIES}
 )
 
 if(WITH_CYCLES_LOGGING)
   list(APPEND LIB
-    extern_glog
+    ${GLOG_LIBRARIES}
+    ${GFLAGS_LIBRARIES}
   )
 endif()
 
diff --git a/intern/cycles/bvh/CMakeLists.txt b/intern/cycles/bvh/CMakeLists.txt
index 27a7f604e1c..fb724704a84 100644
--- a/intern/cycles/bvh/CMakeLists.txt
+++ b/intern/cycles/bvh/CMakeLists.txt
@@ -39,9 +39,16 @@ set(SRC_HEADERS
 
 set(LIB
   cycles_render
+  cycles_util
 )
 
 include_directories(${INC})
 include_directories(SYSTEM ${INC_SYS})
 
+if(WITH_CYCLES_EMBREE)
+  list(APPEND LIB
+    ${EMBREE_LIBRARIES}
+  )
+endif()
+
 cycles_add_library(cycles_bvh "${LIB}" ${SRC} ${SRC_HEADERS})
diff --git a/intern/cycles/cmake/macros.cmake b/intern/cycles/cmake/macros.cmake
index 0efd8bb7ea8..13328a8b6bf 100644
--- a/intern/cycles/cmake/macros.cmake
+++ b/intern/cycles/cmake/macros.cmake
@@ -8,8 +8,64 @@ endfunction()
 
 macro(cycles_add_library target library_deps)
   add_library(${target} ${ARGN})
-  if(NOT ("${library_deps}" STREQUAL ""))
-    target_link_libraries(${target} "${library_deps}")
+
+  # On Windows certain libraries have two sets of binaries: one for debug builds and one for
+  # release builds. The root of this requirement goes into ABI, I believe, but that's outside
+  # of a scope of this comment.
+  #
+  # CMake have a native way of dealing with this, which is specifying what build type the
+  # libraries are provided for:
+  #
+  #   target_link_libraries(tagret optimized|debug|general <libraries>)
+  #
+  # The build type is to be provided as a separate argument to the function.
+  #
+  # CMake's variables for libraries will contain build type in such cases. For example:
+  #
+  #   set(FOO_LIBRARIES optimized libfoo.lib debug libfoo_d.lib)
+  #
+  # Complications starts with a single argument for library_deps: all the elements are being
+  # put to a list: "${FOO_LIBRARIES}" will become "optimized;libfoo.lib;debug;libfoo_d.lib".
+  # This makes it impossible to pass it as-is to target_link_libraries sine it will treat
+  # this argument as a list of libraries to be linked against, causing missing libraries
+  # for optimized.lib.
+  #
+  # What this code does it traverses library_deps and extracts information about whether
+  # library is to provided as general, debug or optimized. This is a little state machine which
+  # keeps track of whiuch build type library is to provided for:
+  #
+  # - If "debug" or "optimized" word is found, the next element in the list is expected to be
+  #   a library which will be passed to target_link_libraries() under corresponding build type.
+  #
+  # - If there is no "debug" or "optimized" used library is specified for all build types.
+  #
+  # NOTE: If separated libraries for debug and release ar eneeded every library is the list are
+  # to be prefixed explicitly.
+  #
+  #  Use: "optimized libfoo optimized libbar debug libfoo_d debug libbar_d"
+  #  NOT: "optimized libfoo libbar debug libfoo_d libbar_d"
+  #
+  # TODO(sergey): This is the same as Blender's side CMake. Find a way to avoid duplication
+  # somehow in a way which allows to have Cycles standalone.
+  if(NOT "${library_deps}" STREQUAL "")
+    set(next_library_mode "")
+    foreach(library ${library_deps})
+      string(TOLOWER "${library}" library_lower)
+      if(("${library_lower}" STREQUAL "optimized") OR
+         ("${library_lower}" STREQUAL "debug"))
+        set(next_library_mode "${library_lower}")
+      else()
+        if("${next_library_mode}" STREQUAL "optimized")
+          target_link_libraries(${target} optimized ${library})
+        elseif("${next_library_mode}" STREQUAL "debug")
+          target_link_libraries(${target} debug ${library})
+        else()
+          target_link_libraries(${target} ${library})
+        endif()
+        set(next_library_mode "")
+      endif()
+    endforeach()
   endif()
+
   cycles_set_solution_folder(${target})
 endmacro()
diff --git a/intern/cycles/device/CMakeLists.txt b/intern/cycles/device/CMakeLists.txt
index a8c4949ad07..35a79356957 100644
--- a/intern/cycles/device/CMakeLists.txt
+++ b/intern/cycles/device/CMakeLists.txt
@@ -60,7 +60,9 @@ set(SRC_HEADERS
 )
 
 set(LIB
-
+  cycles_render
+  cycles_kernel
+  cycles_util
 )
 
 if(WITH_CUDA_DYNLOAD)
diff --git a/intern/cycles/graph/CMakeLists.txt b/intern/cycles/graph/CMakeLists.txt
index c6c46941598..9ff1c5b98c6 100644
--- a/intern/cycles/graph/CMakeLists.txt
+++ b/intern/cycles/graph/CMakeLists.txt
@@ -17,7 +17,7 @@ set(SRC_HEADERS
 )
 
 set(LIB
-
+  cycles_util
 )
 
 include_directories(${INC})
diff --git a/intern/cycles/kernel/osl/CMakeLists.txt b/intern/cycles/kernel/osl/CMakeLists.txt
index 35cca2da8ad..5be5bd181ec 100644
--- a/intern/cycles/kernel/osl/CMakeLists.txt
+++ b/intern/cycles/kernel/osl/CMakeLists.txt
@@ -27,6 +27,10 @@ set(HEADER_SRC
 
 set(LIB
   cycles_render
+
+  ${OSL_LIBRARIES}
+  ${OPENIMAGEIO_LIBRARIES}
+  ${LLVM_LIBRARY}
 )
 
 include_directories(${INC})
diff --git a/intern/cycles/render/CMakeLists.txt b/intern/cycles/render/CMakeLists.txt
index 53196b013f6..92578b888a6 100644
--- a/intern/cycles/render/CMakeLists.txt
+++ b/intern/cycles/render/CMakeLists.txt
@@ -77,6 +77,9 @@ set(SRC_HEADERS
 
 set(LIB
   cycles_bvh
+  cycles_device
+  cycles_subd
+  cycles_util
 )
 
 if(WITH_CYCLES_OSL)
diff --git a/intern/ghost/CMakeLists.txt b/intern/ghost/CMakeLists.txt
index ec2e84268be..cc671e31f92 100644
--- a/intern/ghost/CMakeLists.txt
+++ b/intern/ghost/CMakeLists.txt
@@ -112,6 +112,9 @@ if(WITH_INPUT_NDOF)
   list(APPEND INC_SYS
     ${NDOF_INCLUDE_DIRS}
   )
+  list(APPEND LIB
+    ${NDOF_LIBRARIES}
+  )
 endif()
 
 if(WITH_HEADLESS OR WITH_GHOST_SDL)
@@ -141,6 +144,11 @@ if(WITH_HEADLESS OR WITH_GHOST_SDL)
     list(APPEND INC_SYS
       ${SDL_INCLUDE_DIR}
     )
+    if(NOT WITH_SDL_DYNLOAD)
+      list(APPEND LIB
+        ${SDL_LIBRARY}
+      )
+    endif()
   endif()
 
 elseif(APPLE AND NOT WITH_X11)
diff --git a/intern/libmv/CMakeLists.txt b/intern/libmv/CMakeLists.txt
index e16e27368d0..f587aee615b 100644
--- a/intern/libmv/CMakeLists.txt
+++ b/intern/libmv/CMakeLists.txt
@@ -38,6 +38,8 @@ set(LIB
 )
 
 if(WITH_LIBMV)
+  setup_libdirs()
+
   if(WIN32)
     add_definitions(-D_USE_MATH_DEFINES)
   endif()
@@ -62,7 +64,10 @@ if(WITH_LIBMV)
 
   list(APPEND LIB
     extern_ceres
-    extern_glog
+
+    ${GLOG_LIBRARIES}
+    ${GFLAGS_LIBRARIES}
+    ${PNG_LIBRARIES}
   )
 
   add_definitions(
diff --git a/intern/libmv/bundle.sh b/intern/libmv/bundle.sh
index 08bdf491d3b..2601a2d6754 100755
--- a/intern/libmv/bundle.sh
+++ b/intern/libmv/bundle.sh
@@ -117,6 +117,8 @@ set(LIB
 )
 
 if(WITH_LIBMV)
+  setup_libdirs()
+
   add_definitions(\${GFLAGS_DEFINES})
   add_definitions(\${GLOG_DEFINES})
   add_definitions(\${CERES_DEFINES})
@@ -138,7 +140,10 @@ if(WITH_LIBMV)
 
   list(APPEND LIB
     extern_ceres
-    extern_glog
+
+    \${GLOG_LIBRARIES}
+    \${GFLAGS_LIBRARIES}
+    \${PNG_LIBRARIES}
   )
 
   add_definitions(
diff --git a/intern/locale/CMakeLists.txt b/intern/locale/CMakeLists.txt
index 5c5da31482d..732fa1e4d11 100644
--- a/intern/locale/CMakeLists.txt
+++ b/intern/locale/CMakeLists.txt
@@ -53,6 +53,9 @@ if(WITH_INTERNATIONAL)
   list(APPEND INC_SYS
     ${BOOST_INCLUDE_DIR}
   )
+  list(APPEND LIB
+    ${BOOST_LIBRARIES}
+  )
   add_definitions(-DWITH_INTERNATIONAL)
   add_definitions(${BOOST_DEFINITIONS})
 endif()
diff --git a/intern/mantaflow/CMakeLists.txt b/intern/mantaflow/CMakeLists.txt
index 9fdd8b59aca..c7b3c56c3c2 100644
--- a/intern/mantaflow/CMakeLists.txt
+++ b/intern/mantaflow/CMakeLists.txt
@@ -62,6 +62,10 @@ set(SRC
 
 set(LIB
   extern_mantaflow
+
+  ${PYTHON_LINKFLAGS}
+  ${PYTHON_LIBRARIES}
+  ${ZLIB_LIBRARIES}
 )
 
 blender_add_lib(bf_intern_mantaflow "${SRC}" "${INC}" "${INC_SYS}" "${LIB}")
diff --git a/intern/opencolorio/CMakeLists.txt b/intern/opencolorio/CMakeLists.txt
index 4d95c1b701b..d2336692d22 100644
--- a/intern/opencolorio/CMakeLists.txt
+++ b/intern/opencolorio/CMakeLists.txt
@@ -59,6 +59,10 @@ if(WITH_OPENCOLORIO)
     ocio_impl_glsl.cc
   )
 
+  list(APPEND LIB
+    ${OPENCOLORIO_LIBRARIES}
+  )
+
   if(WIN32)
     list(APPEND INC_SYS
       ${BOOST_INCLUDE_DIR}
@@ -66,6 +70,9 @@ if(WITH_OPENCOLORIO)
     add_definitions(
       -DOpenColorIO_STATIC
     )
+    list(APPEND LIB
+      ${BOOST_LIBRARIES}
+    )
   endif()
 
   data_to_c_simple(gpu_shader_display_transform.glsl SRC)
diff --git a/intern/opensubdiv/CMakeLists.txt b/intern/opensubdiv/CMakeLists.txt
index 69f4bad62b2..268b4e31d7d 100644
--- a/intern/opensubdiv/CMakeLists.txt
+++ b/intern/opensubdiv/CMakeLists.txt
@@ -83,6 +83,16 @@ if(WITH_OPENSUBDIV)
     internal/opensubdiv_util.h
   )
 
+  list(APPEND LIB
+    ${OPENSUBDIV_LIBRARIES}
+  )
+
+  if(WITH_OPENMP_STATIC)
+    list(APPEND LIB
+      ${OpenMP_LIBRARIES}
+    )
+  endif()
+
   OPENSUBDIV_DEFINE_COMPONENT(OPENSUBDIV_HAS_OPENMP)
   # TODO(sergey): OpenCL is not tested and totally unstable atm.
   # OPENSUBDIV_DEFINE_COMPONENT(OPENSUBDIV_HAS_OPENCL)
diff --git a/intern/openvdb/CMakeLists.txt b/intern/openvdb/CMakeLists.txt
index fdc57d1a282..a681d723b9c 100644
--- a/intern/openvdb/CMakeLists.txt
+++ b/intern/openvdb/CMakeLists.txt
@@ -77,11 +77,26 @@ if(WITH_OPENVDB)
     openvdb_util.h
   )
 
+  list(APPEND LIB
+    ${OPENVDB_LIBRARIES}
+    ${OPENEXR_LIBRARIES}
+    ${ZLIB_LIBRARIES}
+  )
+
   if(WITH_OPENVDB_BLOSC)
     add_definitions(
       -DWITH_OPENVDB_BLOSC
     )
+    list(APPEND LIB
+      ${BLOSC_LIBRARIES}
+      ${ZLIB_LIBRARIES}
+    )
   endif()
+
+  list(APPEND LIB
+    ${BOOST_LIBRARIES}
+    ${TBB_LIBRARIES}
+  )
 endif()
 
 blender_add_lib(bf_intern_openvdb "${SRC}" "${INC}" "${INC_SYS}" "${LIB}")
diff --git a/intern/quadriflow/CMakeLists.txt b/intern/quadriflow/CMakeLists.txt
index 35cfe22a018..5fe60f3eda2 100644
--- a/intern/quadriflow/CMakeLists.txt
+++ b/intern/quadriflow/CMakeLists.txt
@@ -36,6 +36,7 @@ set(SRC
 
 set(LIB
   extern_quadriflow
+  ${BOOST_LIBRARIES}
 )
 
 if(WIN32)
diff --git a/intern/rigidbody/CMakeLists.txt b/intern/rigidbody/CMakeLists.txt
index cf9b70448e0..77d88548e1b 100644
--- a/intern/rigidbody/CMakeLists.txt
+++ b/intern/rigidbody/CMakeLists.txt
@@ -33,6 +33,7 @@ set(SRC
 )
 
 set(LIB
+  ${BULLET_LIBRARIES}
 )
 
 blender_add_lib(bf_intern_rigidbody "${SRC}" "${INC}" "${INC_SYS}" "${LIB}")