[GitHub] [incubator-mxnet] leezu commented on issue #19610: [RFC] MXNet: Improve the build with DNNL, MKL, and OpenMP.

[GitBox <gi...@apache.org>]

leezu commented on issue #19610:
URL: https://github.com/apache/incubator-mxnet/issues/19610#issuecomment-736695392


   Thank you @akarbown for opening this RFC. It LGTM overall. Some minor comments / questions below:
   
   I think MKL_USE_ILP64 does not need to be exposed to the user and we may re-use the MXNet USE_INT64_TENSOR_SIZE flag. We should switch to USE_DNNL sooner rather than later.
   
   If ILP64 is used, we may be forced to static link MKL, to avoid symbol conflicts
   with LP64 Blas libraries that could be loaded into the same process if users
   import MXNet and NumPy in the same Python process. cc @access2rohit @Zha0q1 
   
   > To use Intel MKL BLAS, one can install it through apt or yum following the instructions from Intel: Installing Intel® Performance Libraries and Intel® Distribution for Python* Using APT Repository.
   
   We also need to support the upstream Ubuntu MKL packages which are available from Ubuntu 20.04 https://packages.ubuntu.com/focal/intel-mkl In my preliminary trial, the following patch works to find Ubuntu intel-mkl package (and may allow us to delete MXNet's FindMKL.cmake):
   
   
   ``` diff
   modified   cmake/ChooseBlas.cmake
   @@ -47,16 +47,15 @@ elseif(BLAS STREQUAL "Open" OR BLAS STREQUAL "open")
      add_definitions(-DMXNET_USE_BLAS_OPEN=1)
    elseif(BLAS STREQUAL "MKL" OR BLAS STREQUAL "mkl")
      if (USE_INT64_TENSOR_SIZE)
   -    set(MKL_USE_ILP64 ON CACHE BOOL "enable using ILP64 in MKL" FORCE)
   +    set(BLA_VENDOR Intel10_64ilp)
   +    find_package(BLAS)
      else()
   -    if(MKL_USE_ILP64)
   -      message(FATAL_ERROR "MKL_USE_ILP64 cannot be set without USE_INT64_TENSOR_SIZE; "
   -              "Please set USE_INT64_TENSOR_SIZE instead of MKL_USE_ILP64.")
   -    endif()
   +    set(BLA_VENDOR Intel10_64lp)
   +    find_package(BLAS)
      endif()
   -  find_package(MKL REQUIRED)
   +  find_path(MKL_INCLUDE_DIR mkl.h PATHS ENV MKLROOT PATH_SUFFIXES include mkl REQUIRED)
      include_directories(SYSTEM ${MKL_INCLUDE_DIR})
   -  list(APPEND mshadow_LINKER_LIBS ${MKL_LIBRARIES})
   +  list(APPEND mshadow_LINKER_LIBS ${BLAS_LIBRARIES})
      add_definitions(-DMSHADOW_USE_CBLAS=0)
      add_definitions(-DMSHADOW_USE_MKL=1)
      add_definitions(-DMXNET_USE_BLAS_MKL=1)
   ```
   
   > need to enable the Intel compiler build process which seems to be broken at this moment [6].
   
   Is there any update on the ICC upstream compiler fix?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org