You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/09/21 02:07:37 UTC
[GitHub] [tvm] mshr-h opened a new pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
mshr-h opened a new pull request #9060:
URL: https://github.com/apache/tvm/pull/9060
This an update for #6023.
@fernchen @zhiics @tqchen @FrozenGene
Can you take a look at this PR, please? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] fernchen commented on a change in pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
fernchen commented on a change in pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#discussion_r719199217
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
Review comment:
Why needs this command?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mshr-h commented on a change in pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
mshr-h commented on a change in pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#discussion_r746330230
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
Review comment:
That's just a best practice for Dockerfiles.
[Best practices for writing Dockerfiles | Docker Documentation](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#run)
It might be unnecessary for this script. I'll remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mshr-h commented on a change in pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
mshr-h commented on a change in pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#discussion_r746332294
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
-apt-get install caffe-cpu -y
+# install python packages
+pip install "numpy>=1.7.1"
+pip install "protobuf>=2.5.0"
+pip install "scikit-image>=0.9.3"
+pip install "six>=1.1.0"
-pip3 install --upgrade scikit-image
+# Build the Caffe and the python wrapper
+echo "Downloading Caffe"
+CAFFE_HOME="/opt/caffe"
+git clone --branch=ssd --depth 1 https://github.com/weiliu89/caffe /caffe_src
+cd /caffe_src
+
+echo "Building Caffe"
+mkdir /caffe_src/build && cd /caffe_src/build
+cmake .. -DCMAKE_INSTALL_PREFIX=${CAFFE_HOME} -DCMAKE_BUILD_TYPE=Release -DCPU_ONLY=1 \
+ -Dpython_version=3 -DUSE_OPENCV=OFF -DUSE_LEVELDB=OFF -DUSE_LMDB=OFF -DBUILD_docs=OFF -DBLAS=open
+make all -j`nproc`
Review comment:
Ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mshr-h commented on a change in pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
mshr-h commented on a change in pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#discussion_r719886573
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
Review comment:
Those packages are the minimum requirement for building caffe from source.
So I put them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mshr-h commented on pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
mshr-h commented on pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#issuecomment-951625409
Hi @leandron @areusch @jroesch @masahi , could you take a review for this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#discussion_r746161072
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
-apt-get install caffe-cpu -y
+# install python packages
+pip install "numpy>=1.7.1"
Review comment:
can you add these to `python/gen_requirements.py`? also, can you use just one pip command to do this?
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
-apt-get install caffe-cpu -y
+# install python packages
+pip install "numpy>=1.7.1"
+pip install "protobuf>=2.5.0"
+pip install "scikit-image>=0.9.3"
+pip install "six>=1.1.0"
-pip3 install --upgrade scikit-image
+# Build the Caffe and the python wrapper
+echo "Downloading Caffe"
+CAFFE_HOME="/opt/caffe"
+git clone --branch=ssd --depth 1 https://github.com/weiliu89/caffe /caffe_src
+cd /caffe_src
+
+echo "Building Caffe"
+mkdir /caffe_src/build && cd /caffe_src/build
+cmake .. -DCMAKE_INSTALL_PREFIX=${CAFFE_HOME} -DCMAKE_BUILD_TYPE=Release -DCPU_ONLY=1 \
+ -Dpython_version=3 -DUSE_OPENCV=OFF -DUSE_LEVELDB=OFF -DUSE_LMDB=OFF -DBUILD_docs=OFF -DBLAS=open
+make all -j`nproc`
Review comment:
should this `$(expr $(nproc) - 1)` to avoid overloading systems when running docker/build.sh?
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
Review comment:
the `rm -rf /var/lib/apt/lists/*` part seems a little weird. can you elaborate/link to these instructions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#discussion_r777749933
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
Review comment:
ah, didn't know this but it definitely makes sense. it wasn't the convention we were using before, but we should adopt it. i've filed #9825 to track this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch merged pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
areusch merged pull request #9060:
URL: https://github.com/apache/tvm/pull/9060
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mshr-h commented on a change in pull request #9060: [CI][Caffe Frontend] Change the caffe deps into SSD distribution
Posted by GitBox <gi...@apache.org>.
mshr-h commented on a change in pull request #9060:
URL: https://github.com/apache/tvm/pull/9060#discussion_r746332294
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
-apt-get install caffe-cpu -y
+# install python packages
+pip install "numpy>=1.7.1"
+pip install "protobuf>=2.5.0"
+pip install "scikit-image>=0.9.3"
+pip install "six>=1.1.0"
-pip3 install --upgrade scikit-image
+# Build the Caffe and the python wrapper
+echo "Downloading Caffe"
+CAFFE_HOME="/opt/caffe"
+git clone --branch=ssd --depth 1 https://github.com/weiliu89/caffe /caffe_src
+cd /caffe_src
+
+echo "Building Caffe"
+mkdir /caffe_src/build && cd /caffe_src/build
+cmake .. -DCMAKE_INSTALL_PREFIX=${CAFFE_HOME} -DCMAKE_BUILD_TYPE=Release -DCPU_ONLY=1 \
+ -Dpython_version=3 -DUSE_OPENCV=OFF -DUSE_LEVELDB=OFF -DUSE_LMDB=OFF -DBUILD_docs=OFF -DBLAS=open
+make all -j`nproc`
Review comment:
That was just for testing on my local machine.
I'll remove it.
##########
File path: docker/install/ubuntu_install_caffe.sh
##########
@@ -22,13 +22,40 @@ set -o pipefail
apt-get update --fix-missing
-# The precompiled caffe dependents on tzdata.
-# While installing tzdata in docker, we need set the time zone manually,
-# which will cause the container to hang during installation.
-# So in order to avoid manually selecting the time zone, set as following:
-export DEBIAN_FRONTEND=noninteractive
-apt-get install -y tzdata
+# Install dependencies
+apt-get install -y --no-install-recommends libboost-filesystem-dev libboost-python-dev \
+ libboost-system-dev libboost-thread-dev libboost-regex-dev protobuf-compiler \
+ libprotobuf-dev libhdf5-serial-dev libopenblas-dev libgflags-dev libgoogle-glog-dev
+rm -rf /var/lib/apt/lists/*
-apt-get install caffe-cpu -y
+# install python packages
+pip install "numpy>=1.7.1"
Review comment:
Ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org