[GitHub] LakeCarrot opened a new issue #7370: Error when trying to build docker image with GPU and S3 support

[gi...@git.apache.org]

LakeCarrot opened a new issue #7370: Error when trying to build docker image with GPU and S3 support
URL: https://github.com/apache/incubator-mxnet/issues/7370
 
 
   Hi all,
   
   I tried to customize the official MXNet docker image to get the Amazon S3 and GPU support. The dockerfile I use to create such image is as follows:
   ```
   # -*- mode: dockerfile -*-
   # dockerfile to build libmxnet.so on GPU
   FROM nvidia/cuda:8.0-cudnn5-devel
   
   COPY install/cpp.sh install/
   RUN install/cpp.sh
   
   COPY hadoop-3.0.0-alpha4 /tmp/hadoop-3.0.0-alpha4/
   
   ENV HADOOP_HOME /tmp/hadoop-3.0.0-alpha4/
   
   RUN apt-get update && \
       apt-get install -y libopenblas-dev liblapack-dev && \
       apt-get install -y libopencv-dev && \
       apt-get install -y default-jre && \
       apt-get install -y default-jdk 
   ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
   RUN export LD_LIBRARY_PATH=/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server:$LD_LIBRARY_PATH && \
       ls /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server
   
   ENV BUILD_OPTS "USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1 USE_OPENCV=1 USE_BLAS=openblas USE_DIST_KVSTORE=1 USE_S3=1"
   RUN git clone -b v0.10.0-support --recursive https://github.com/apache/incubator-mxnet.git && cd incubator-mxnet && \
       make clean && \
       git submodule update --init --recursive && \
       make -j$(nproc) $BUILD_OPTS 
       
   # -*- mode: dockerfile -*-
   # part of the dockerfile to install the python binding
   
   COPY install/python.sh install/
   RUN install/python.sh
   ENV PYTHONPATH=/incubator-mxnet/python
   CMD sh -c 'ln -s /dev/null /dev/raw1394'; bash
   ```
   And I get the following error message when trying to build the mxnet
   ```
   g++: error: /incubator-mxnet/dmlc-core/libdmlc.a: No such file or directory
   make: *** [lib/libmxnet.so] Error 1
   make: *** Waiting for unfinished jobs....
   ```
   I wonder does anyone have any idea of this problem? I googled the problem. it seems all of them are related to the Windows installation. But I made all of this under ubuntu16.04. 
   To give you more context, I attached more log below.
   ```
   build/src/operator/rnn_gpu.o build/src/operator/convolution_gpu.o -pthread -lm -lcudart -lcublas -lcurand -L/usr/local/cuda/lib64 -L/usr/local/cuda/lib -lopenblas -fopenmp -lrt -lcurl -lssl -lcrypto /usr/lib/x86_64-linux-gnu/libopencv_calib3d.so -lopencv_calib3d /usr/lib/x86_64-linux-gnu/libopencv_contrib.so -lopencv_contrib /usr/lib/x86_64-linux-gnu/libopencv_core.so -lopencv_core /usr/lib/x86_64-linux-gnu/libopencv_features2d.so -lopencv_features2d /usr/lib/x86_64-linux-gnu/libopencv_flann.so -lopencv_flann /usr/lib/x86_64-linux-gnu/libopencv_gpu.so -lopencv_gpu /usr/lib/x86_64-linux-gnu/libopencv_highgui.so -lopencv_highgui /usr/lib/x86_64-linux-gnu/libopencv_imgproc.so -lopencv_imgproc /usr/lib/x86_64-linux-gnu/libopencv_legacy.so -lopencv_legacy /usr/lib/x86_64-linux-gnu/libopencv_ml.so -lopencv_ml /usr/lib/x86_64-linux-gnu/libopencv_objdetect.so -lopencv_objdetect /usr/lib/x86_64-linux-gnu/libopencv_ocl.so -lopencv_ocl /usr/lib/x86_64-linux-gnu/libopencv_photo.so -lopencv_ph
 oto /usr/lib/x86_64-linux-gnu/libopencv_stitching.so -lopencv_stitching /usr/lib/x86_64-linux-gnu/libopencv_superres.so -lopencv_superres /usr/lib/x86_64-linux-gnu/libopencv_ts.so /usr/lib/x86_64-linux-gnu/libopencv_video.so -lopencv_video /usr/lib/x86_64-linux-gnu/libopencv_videostab.so -lopencv_videostab -lcudnn  /incubator-mxnet/deps/lib/libprotobuf-lite.a /incubator-mxnet/deps/lib/libzmq.a -lcuda -lcufft \
    -Wl,--whole-archive /incubator-mxnet/nnvm/lib/libnnvm.a -Wl,--no-whole-archive
   g++: error: /incubator-mxnet/dmlc-core/libdmlc.a: No such file or directory
   make: *** [lib/libmxnet.so] Error 1
   make: *** Waiting for unfinished jobs....
   Makefile:242: recipe for target 'lib/libmxnet.so' failed
   c/engine/engine.o
   a - build/src/engine/threaded_engine_perdevice.o
   a - build/src/storage/storage.o
   a - build/src/c_api/c_api_executor.o
   a - build/src/c_api/c_api_symbolic.o
   a - build/src/c_api/c_api_ndarray.o
   a - build/src/c_api/c_predict_api.o
   a - build/src/c_api/c_api.o
   ...
   The command '/bin/sh -c git clone -b v0.10.0-support --recursive https://github.com/apache/incubator-mxnet.git && cd incubator-mxnet &&     make clean &&     git submodule update --init --recursive &&     make -j$(nproc) $BUILD_OPTS' returned a non-zero code: 2
   ```
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services