You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/04/29 21:42:59 UTC
[GitHub] [incubator-mxnet] hasanmainul opened a new issue #18196: Building from source on Ubuntu for gpu environment
hasanmainul opened a new issue #18196:
URL: https://github.com/apache/incubator-mxnet/issues/18196
## Description
I tried building from source for a while now. Followed the official [link](https://mxnet.apache.org/get_started/ubuntu_setup) for building on Ubuntu. I wanted to experiment with GPU setup. My instance has two Tesla P100 in it.
used `cp config/linux_gpu.cmake config.cmake` instruction
### Error Message
```
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_CROSSCOMPILING FALSE
-- CMAKE_HOST_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_NAME Linux
-- CMake version '3.17.2' using generator 'Ninja'
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda-10.1/bin/nvcc
-- The CUDA compiler identification is NVIDIA 10.1.168
-- Check for working CUDA compiler: /usr/local/cuda-10.1/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda-10.1/bin/nvcc - works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- Intel MKL-DNN compat: set DNNL_BUILD_EXAMPLES to MKLDNN_BUILD_EXAMPLES with value `OFF`
-- Intel MKL-DNN compat: set DNNL_BUILD_TESTS to MKLDNN_BUILD_TESTS with value `OFF`
-- Intel MKL-DNN compat: set DNNL_ENABLE_JIT_PROFILING to MKLDNN_ENABLE_JIT_PROFILING with value `OFF`
-- Intel MKL-DNN compat: set DNNL_LIBRARY_TYPE to MKLDNN_LIBRARY_TYPE with value `STATIC`
-- Intel MKL-DNN compat: set DNNL_ARCH_OPT_FLAGS to MKLDNN_ARCH_OPT_FLAGS with value ``
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.0")
-- Found OpenMP_CXX: -fopenmp (found version "4.0")
-- Found OpenMP: TRUE (found version "4.0")
-- GPU support is disabled
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.11") found components: doxygen missing components: dot
-- Found Git: /usr/bin/git (found version "2.7.4")
-- Intel(R) VTune(TM) Amplifier JIT profiling disabled
-- Found OpenBLAS libraries: /usr/lib/libopenblas.so
-- Found OpenBLAS include: /usr/include
CMake Warning at /usr/share/OpenCV/OpenCVConfig.cmake:185 (message):
opencv_imgcodecs is required but was not found
Call Stack (most recent call first):
CMakeLists.txt:355 (find_package)
-- OpenCV 2.4.9.1 found (/usr/share/OpenCV)
-- OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc
-- Performing Test OPENMP_HAVE_WERROR_FLAG
-- Performing Test OPENMP_HAVE_WERROR_FLAG - Success
-- Performing Test OPENMP_HAVE_STD_GNUPP11_FLAG
-- Performing Test OPENMP_HAVE_STD_GNUPP11_FLAG - Success
-- Performing Test OPENMP_HAVE_STD_CPP11_FLAG
-- Performing Test OPENMP_HAVE_STD_CPP11_FLAG - Success
-- Found PythonInterp: /usr/bin/python (found version "3.5.2")
-- Cannot find llvm-lit.
-- Please put llvm-lit in your PATH, set OPENMP_LLVM_LIT_EXECUTABLE to its full path, or point OPENMP_LLVM_TOOLS_DIR to its directory.
CMake Warning at 3rdparty/openmp/cmake/OpenMPTesting.cmake:22 (message):
The check targets will not be available!
Call Stack (most recent call first):
3rdparty/openmp/cmake/OpenMPTesting.cmake:40 (find_standalone_test_dependencies)
3rdparty/openmp/CMakeLists.txt:49 (include)
-- Performing Test LIBOMP_HAVE_FNO_EXCEPTIONS_FLAG
-- Performing Test LIBOMP_HAVE_FNO_EXCEPTIONS_FLAG - Success
-- Performing Test LIBOMP_HAVE_FNO_RTTI_FLAG
-- Performing Test LIBOMP_HAVE_FNO_RTTI_FLAG - Success
-- Performing Test LIBOMP_HAVE_X_CPP_FLAG
-- Performing Test LIBOMP_HAVE_X_CPP_FLAG - Success
-- Performing Test LIBOMP_HAVE_WCAST_QUAL_FLAG
-- Performing Test LIBOMP_HAVE_WCAST_QUAL_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_FUNCTION_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_FUNCTION_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_LOCAL_TYPEDEF_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_LOCAL_TYPEDEF_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VALUE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VALUE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VARIABLE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNUSED_VARIABLE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_SWITCH_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SWITCH_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_COVERED_SWITCH_DEFAULT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_COVERED_SWITCH_DEFAULT_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_DEPRECATED_REGISTER_FLAG
-- Performing Test LIBOMP_HAVE_WNO_DEPRECATED_REGISTER_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_SIGN_COMPARE_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SIGN_COMPARE_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_GNU_ANONYMOUS_STRUCT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_GNU_ANONYMOUS_STRUCT_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_UNKNOWN_PRAGMAS_FLAG
-- Performing Test LIBOMP_HAVE_WNO_UNKNOWN_PRAGMAS_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_MISSING_FIELD_INITIALIZERS_FLAG
-- Performing Test LIBOMP_HAVE_WNO_MISSING_FIELD_INITIALIZERS_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_MISSING_BRACES_FLAG
-- Performing Test LIBOMP_HAVE_WNO_MISSING_BRACES_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_COMMENT_FLAG
-- Performing Test LIBOMP_HAVE_WNO_COMMENT_FLAG - Success
-- Performing Test LIBOMP_HAVE_WNO_SELF_ASSIGN_FLAG
-- Performing Test LIBOMP_HAVE_WNO_SELF_ASSIGN_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_VLA_EXTENSION_FLAG
-- Performing Test LIBOMP_HAVE_WNO_VLA_EXTENSION_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WNO_FORMAT_PEDANTIC_FLAG
-- Performing Test LIBOMP_HAVE_WNO_FORMAT_PEDANTIC_FLAG - Failed
-- Performing Test LIBOMP_HAVE_WSTRINGOP_OVERFLOW_FLAG
-- Performing Test LIBOMP_HAVE_WSTRINGOP_OVERFLOW_FLAG - Failed
-- Performing Test LIBOMP_HAVE_MSSE2_FLAG
-- Performing Test LIBOMP_HAVE_MSSE2_FLAG - Success
-- Performing Test LIBOMP_HAVE_FTLS_MODEL_FLAG
-- Performing Test LIBOMP_HAVE_FTLS_MODEL_FLAG - Success
-- Performing Test LIBOMP_HAVE_MMIC_FLAG
-- Performing Test LIBOMP_HAVE_MMIC_FLAG - Failed
-- Performing Test LIBOMP_HAVE_M32_FLAG
-- Performing Test LIBOMP_HAVE_M32_FLAG - Failed
-- Performing Test LIBOMP_HAVE_X_FLAG
-- Performing Test LIBOMP_HAVE_X_FLAG - Success
-- Performing Test LIBOMP_HAVE_WARN_SHARED_TEXTREL_FLAG
-- Performing Test LIBOMP_HAVE_WARN_SHARED_TEXTREL_FLAG - Success
-- Performing Test LIBOMP_HAVE_AS_NEEDED_FLAG
-- Performing Test LIBOMP_HAVE_AS_NEEDED_FLAG - Success
-- Performing Test LIBOMP_HAVE_VERSION_SCRIPT_FLAG
-- Performing Test LIBOMP_HAVE_VERSION_SCRIPT_FLAG - Success
-- Performing Test LIBOMP_HAVE_STATIC_LIBGCC_FLAG
-- Performing Test LIBOMP_HAVE_STATIC_LIBGCC_FLAG - Success
-- Performing Test LIBOMP_HAVE_Z_NOEXECSTACK_FLAG
-- Performing Test LIBOMP_HAVE_Z_NOEXECSTACK_FLAG - Success
-- Performing Test LIBOMP_HAVE_FINI_FLAG
-- Performing Test LIBOMP_HAVE_FINI_FLAG - Success
-- Found Perl: /usr/bin/perl (found version "5.22.1")
-- Performing Test LIBOMP_HAVE_VERSION_SYMBOLS
-- Performing Test LIBOMP_HAVE_VERSION_SYMBOLS - Success
-- Performing Test LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS
-- Performing Test LIBOMP_HAVE___BUILTIN_FRAME_ADDRESS - Success
-- Performing Test LIBOMP_HAVE_WEAK_ATTRIBUTE
-- Performing Test LIBOMP_HAVE_WEAK_ATTRIBUTE - Success
-- Looking for include files windows.h, psapi.h
-- Looking for include files windows.h, psapi.h - not found
-- Looking for EnumProcessModules in psapi
-- Looking for EnumProcessModules in psapi - not found
-- LIBOMP: Operating System -- Linux
-- LIBOMP: Target Architecture -- x86_64
-- LIBOMP: Build Type -- Release
-- LIBOMP: Library Kind -- SHARED
-- LIBOMP: Library Type -- normal
-- LIBOMP: Fortran Modules -- FALSE
-- LIBOMP: Build -- 20140926
-- LIBOMP: Use Stats-gathering -- FALSE
-- LIBOMP: Use Debugger-support -- FALSE
-- LIBOMP: Use ITT notify -- TRUE
-- LIBOMP: Use OMPT-support -- TRUE
-- LIBOMP: Use OMPT-optional -- TRUE
-- LIBOMP: Use Adaptive locks -- TRUE
-- LIBOMP: Use quad precision -- TRUE
-- LIBOMP: Use TSAN-support -- FALSE
-- LIBOMP: Use Hwloc library -- FALSE
-- Looking for sqrt in m
-- Looking for sqrt in m - found
-- Looking for __atomic_load_1
-- Looking for __atomic_load_1 - not found
-- Looking for __atomic_load_1 in atomic
-- Looking for __atomic_load_1 in atomic - found
-- check-libomp does nothing.
-- check-ompt does nothing.
-- check-openmp does nothing.
USE_LAPACK is ON
CMake Warning at 3rdparty/googletest/googletest/CMakeLists.txt:47 (project):
VERSION keyword not followed by a value or was followed by a value that
expanded to nothing.
-- Found GTest: gtest
-- Found CUDNN: /usr/local/cuda/lib64/libcudnn.so
-- Found OpenMP_C: -fopenmp
-- Found OpenMP_CXX: -fopenmp
-- Found OpenMP: TRUE
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for fopen64
-- Looking for fopen64 - not found
-- Looking for C++ include cxxabi.h
-- Looking for C++ include cxxabi.h - found
-- Looking for nanosleep
-- Looking for nanosleep - found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /usr/include
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Searching 16 bit integer - Using unsigned short
-- Check if the system is big endian - little endian
-- /home/cc/mxnet/3rdparty/dmlc-core/cmake/build_config.h.in -> include/dmlc/build_config.h
-- Performing Test SUPPORT_MSSE2
-- Performing Test SUPPORT_MSSE2 - Success
-- Autodetected CUDA architecture(s): 6.0 6.0
-- CUDA: Using the following NVCC architecture flags -gencode;arch=compute_60,code=sm_60
-- Found CUDAToolkit: /usr/local/cuda-10.1/include (found version "10.1.168")
-- Found NVML: /usr/local/cuda/include
-- Found NVML (include: /usr/local/cuda/include, library: /usr/lib/x86_64-linux-gnu/libnvidia-ml.so)
-- Found NCCL: /usr/include
-- Found NCCL (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnccl.so)
-- Performing Test SUPPORT_MSSE3
-- Performing Test SUPPORT_MSSE3 - Success
-- Determining F16C support
-- Performing Test COMPILER_SUPPORT_MF16C
-- Performing Test COMPILER_SUPPORT_MF16C - Success
-- Found Python3: /usr/bin/python3.5 (found version "3.5.2") found components: Interpreter
CMake Warning at CMakeLists.txt:832 (message):
OpenCV_VERSION_MAJOR: 2, version 3 with imgcodecs is required for im2rec,
im2rec will not be available
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cc/mxnet/build
```
## To Reproduce
Just followed the instruction from the building page.
`cmake -GNinja ..`
is failing
## What have you tried to solve it?
1. Installed `nccl`, `cuDNN`
2. Also Followed [old](https://mxnet.apache.org/get_started/build_from_source) build instruction using `make`
## Environment
```
curl --retry 10 -s https://raw.githubusercontent.com/dmlc/gluon-nlp/master/tools/diagnose.py | python
----------Python Info----------
Version : 3.5.2
Compiler : GCC 5.4.0 20160609
Build : ('default', 'Oct 8 2019 13:06:37')
Arch : ('64bit', 'ELF')
------------Pip Info-----------
Version : 8.1.1
Directory : /usr/lib/python3/dist-packages/pip
----------MXNet Info-----------
No MXNet installed.
----------System Info----------
Platform : Linux-4.4.0-177-generic-x86_64-with-Ubuntu-16.04-xenial
system : Linux
node : bytescheduler
release : 4.4.0-177-generic
version : #207-Ubuntu SMP Mon Mar 16 01:16:10 UTC 2020
----------Hardware Info----------
machine : x86_64
processor : x86_64
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 48
On-line CPU(s) list: 0-47
Thread(s) per core: 2
Core(s) per socket: 12
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
Stepping: 2
CPU MHz: 1463.015
CPU max MHz: 3100.0000
CPU min MHz: 1200.0000
BogoMIPS: 4601.07
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 30720K
NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46
NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear flush_l1d
----------Network Test----------
Setting timeout: 10
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.026647567749023438 sec.
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0013 sec, LOAD: 0.5594 sec.
Timing for GluonNLP: http://gluon-nlp.mxnet.io, DNS: 0.1320 sec, LOAD: 0.4132 sec.
Timing for D2L (zh-cn): http://zh.d2l.ai, DNS: 0.0350 sec, LOAD: 0.1409 sec.
Timing for GluonNLP GitHub: https://github.com/dmlc/gluon-nlp, DNS: 0.0000 sec, LOAD: 0.5460 sec.
Timing for FashionMNIST: https://repo.mxnet.io/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0922 sec, LOAD: 0.3971 sec.
Timing for D2L: http://d2l.ai, DNS: 0.0331 sec, LOAD: 0.4027 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0118 sec, LOAD: 0.2965 sec.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on issue #18196: Building from source on Ubuntu for gpu environment
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #18196:
URL: https://github.com/apache/incubator-mxnet/issues/18196#issuecomment-621485869
You need c++17 compiler for compiling latest master version. Doc will be updated by https://github.com/apache/incubator-mxnet/pull/18161/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu edited a comment on issue #18196: Building from source on Ubuntu for gpu environment
Posted by GitBox <gi...@apache.org>.
leezu edited a comment on issue #18196:
URL: https://github.com/apache/incubator-mxnet/issues/18196#issuecomment-621485869
You need c++17 compiler for compiling latest master version. For example gcc7. You're on gcc5. Doc will be updated by https://github.com/apache/incubator-mxnet/pull/18161/
Let's close this issue now, but you can reopen it if you face more issues
You can also compile the 1.x branch which still works with gcc5 and c++11
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu edited a comment on issue #18196: Building from source on Ubuntu for gpu environment
Posted by GitBox <gi...@apache.org>.
leezu edited a comment on issue #18196:
URL: https://github.com/apache/incubator-mxnet/issues/18196#issuecomment-621485869
You need c++17 compiler for compiling latest master version. For example gcc7. You're on gcc5. Doc will be updated by https://github.com/apache/incubator-mxnet/pull/18161/
Let's close this issue now, but you can reopen it if you face more issues
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org