You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/01/19 00:31:03 UTC
[GitHub] [tvm] driazati opened a new pull request #9971: Use ci.py explicitly in docs building instructions
driazati opened a new pull request #9971:
URL: https://github.com/apache/tvm/pull/9971
This adds `ci.py` to the docs to make it more clear how to easily build the docs locally. This also re-arranges CI following the merging of all CI steps to run concurrently since there's no need to run the Sphinx precheck during GPU unit tests. This still preserves it though in the docs step as a way to quickly bail out if there are formatting errors so the full tutorials don't get built.
Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius merged pull request #9971: Use ci.py explicitly in docs building instructions
Posted by GitBox <gi...@apache.org>.
Mousius merged pull request #9971:
URL: https://github.com/apache/tvm/pull/9971
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] driazati commented on a change in pull request #9971: Use ci.py explicitly in docs building instructions
Posted by GitBox <gi...@apache.org>.
driazati commented on a change in pull request #9971:
URL: https://github.com/apache/tvm/pull/9971#discussion_r793945463
##########
File path: tests/scripts/task_python_docs.sh
##########
@@ -34,14 +32,78 @@ cleanup()
}
trap cleanup 0
-# cleanup old states
-rm -rf docs/_build
-rm -rf docs/_staging
-mkdir -p docs/_build/html
-mkdir -p docs/_staging/html
-rm -rf docs/gen_modules
-rm -rf docs/doxygen
+clean_files() {
+ # cleanup old states
+ rm -rf docs/_build
+ rm -rf docs/_staging
+ mkdir -p docs/_build/html
+ mkdir -p docs/_staging/html
+ rm -rf docs/gen_modules
+ rm -rf docs/doxygen
+ find . -type f -path "*.pyc" | xargs rm -f
+}
+
+sphinx_precheck() {
+ clean_files
+ echo "PreCheck sphinx doc generation WARNINGS.."
+ make cython3
+
+ pushd docs
+ make clean
+ TVM_TUTORIAL_EXEC_PATTERN=none make html 2>&1 | tee /tmp/$$.log.txt
+ check_sphinx_warnings "docs"
+ popd
+}
+
+
+function join_by { local IFS="$1"; shift; echo "$*"; }
+
+IGNORED_WARNINGS=(
+ '__mro__'
Review comment:
Added a comment, generally people shouldn't be adding new warnings here unless there are infra problems (i.e. we had to ignore some stuff when bumping up the PyTorch version). Most of these aren't new and were being ignored before, just not in the actual docs build (where they weren't checked at all) and only on the precheck.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #9971: Use ci.py explicitly in docs building instructions
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #9971:
URL: https://github.com/apache/tvm/pull/9971#discussion_r793165293
##########
File path: docs/README.md
##########
@@ -16,52 +16,61 @@
<!--- under the License. -->
# TVM Documentation
+
This folder contains the source of TVM's documentation, hosted at https://tvm.apache.org/docs
## Build Locally
-See also the instructions below to run a specific tutorial. Note that some of the tutorials need GPU support. Once build, either of these can be served using Python's build in HTTP server:
+### With Docker (recommended)
-```bash
-# Run this and then visit http://localhost:8000 in your browser
-cd docs/_build/html && python3 -m http.server
-```
+1. Build TVM and the docs inside the [tlcpack/ci-gpu image](https://hub.docker.com/r/tlcpack/ci-gpu) using the [`ci.py`](../tests/scripts/ci.py) script.
-### With Docker (recommended)
+ ```bash
+ # If this runs into errors, try cleaning your 'build' directory
+ python tests/scripts/ci.py docs
-1. Build TVM and the docs inside the [tlcpack/ci-gpu image](https://hub.docker.com/r/tlcpack/ci-gpu)
+ # See other doc building options
+ python tests/scripts/ci.py docs --help
+ ```
- ```bash
- # If this runs into errors, try cleaning your 'build' directory
- make docs
- ```
+2. Serve the docs and visit http://localhost:8000 in your browser
+ ```bash
+ # Run an HTTP server you can visit to view the docs in your browser
+ python tests/scripts/ci.py serve-docs
+ ```
### Native
1. [Build TVM](https://tvm.apache.org/docs/install/from_source.html) first in the repo root folder
2. Install dependencies
- ```bash
- # Pillow on Ubuntu may require libjpeg-dev from apt
- docker run tlcpack/ci-gpu:v0.78 bash -c \
- 'python3 -m pip install --quiet tlcpack-sphinx-addon==0.2.1 synr==0.5.0 && python3 -m pip freeze' > frozen-requirements.txt
+ ```bash
+ # Pillow on Ubuntu may require libjpeg-dev from apt
+ docker run tlcpack/ci-gpu:v0.78 bash -c \
Review comment:
should this be docker/bash.sh ci_gpu to avoid the version?
##########
File path: tests/scripts/task_python_docs.sh
##########
@@ -34,14 +32,78 @@ cleanup()
}
trap cleanup 0
-# cleanup old states
-rm -rf docs/_build
-rm -rf docs/_staging
-mkdir -p docs/_build/html
-mkdir -p docs/_staging/html
-rm -rf docs/gen_modules
-rm -rf docs/doxygen
+clean_files() {
+ # cleanup old states
+ rm -rf docs/_build
+ rm -rf docs/_staging
+ mkdir -p docs/_build/html
+ mkdir -p docs/_staging/html
+ rm -rf docs/gen_modules
+ rm -rf docs/doxygen
+ find . -type f -path "*.pyc" | xargs rm -f
+}
+
+sphinx_precheck() {
+ clean_files
+ echo "PreCheck sphinx doc generation WARNINGS.."
+ make cython3
+
+ pushd docs
+ make clean
+ TVM_TUTORIAL_EXEC_PATTERN=none make html 2>&1 | tee /tmp/$$.log.txt
+ check_sphinx_warnings "docs"
+ popd
+}
+
+
+function join_by { local IFS="$1"; shift; echo "$*"; }
+
+IGNORED_WARNINGS=(
+ '__mro__'
Review comment:
could you add a comment explaining why these are ignored, and which piece of the warning string these match? i.e if I want to add one here, what do i do?
##########
File path: tests/scripts/task_python_docs.sh
##########
@@ -34,14 +32,78 @@ cleanup()
}
trap cleanup 0
-# cleanup old states
-rm -rf docs/_build
-rm -rf docs/_staging
-mkdir -p docs/_build/html
-mkdir -p docs/_staging/html
-rm -rf docs/gen_modules
-rm -rf docs/doxygen
+clean_files() {
+ # cleanup old states
+ rm -rf docs/_build
+ rm -rf docs/_staging
+ mkdir -p docs/_build/html
+ mkdir -p docs/_staging/html
+ rm -rf docs/gen_modules
+ rm -rf docs/doxygen
+ find . -type f -path "*.pyc" | xargs rm -f
+}
+
+sphinx_precheck() {
+ clean_files
+ echo "PreCheck sphinx doc generation WARNINGS.."
+ make cython3
+
+ pushd docs
+ make clean
+ TVM_TUTORIAL_EXEC_PATTERN=none make html 2>&1 | tee /tmp/$$.log.txt
+ check_sphinx_warnings "docs"
+ popd
+}
+
+
+function join_by { local IFS="$1"; shift; echo "$*"; }
+
+IGNORED_WARNINGS=(
+ '__mro__'
+ 'UserWarning'
+ 'FutureWarning'
+ 'tensorflow'
+ 'Keras'
+ 'pytorch'
+ 'TensorFlow'
+ 'coremltools'
+ '403'
+ 'git describe'
+ 'scikit-learn version'
+ 'doing serial write'
+ 'gen_gallery extension is not safe for parallel'
+ 'strategy:conv2d NHWC layout is not optimized for x86 with autotvm.'
+ 'strategy:depthwise_conv2d NHWC layout is not optimized for x86 with autotvm.'
+ 'autotvm:Cannot find config for target=llvm -keys=cpu -link-params=0'
+ 'autotvm:One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.'
+ 'autotvm:Cannot find config for target=cuda -keys=cuda,gpu'
+)
+
+JOINED_WARNINGS=$(join_by '|' "${IGNORED_WARNINGS[@]}")
Review comment:
this is a little perilous as they aren't escaped. wdyt about doing this in python and then we could add tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on pull request #9971: Use ci.py explicitly in docs building instructions
Posted by GitBox <gi...@apache.org>.
Mousius commented on pull request #9971:
URL: https://github.com/apache/tvm/pull/9971#issuecomment-1025547644
Thanks @driazati :smile_cat:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] driazati commented on a change in pull request #9971: Use ci.py explicitly in docs building instructions
Posted by GitBox <gi...@apache.org>.
driazati commented on a change in pull request #9971:
URL: https://github.com/apache/tvm/pull/9971#discussion_r793943633
##########
File path: tests/scripts/task_python_docs.sh
##########
@@ -34,14 +32,78 @@ cleanup()
}
trap cleanup 0
-# cleanup old states
-rm -rf docs/_build
-rm -rf docs/_staging
-mkdir -p docs/_build/html
-mkdir -p docs/_staging/html
-rm -rf docs/gen_modules
-rm -rf docs/doxygen
+clean_files() {
+ # cleanup old states
+ rm -rf docs/_build
+ rm -rf docs/_staging
+ mkdir -p docs/_build/html
+ mkdir -p docs/_staging/html
+ rm -rf docs/gen_modules
+ rm -rf docs/doxygen
+ find . -type f -path "*.pyc" | xargs rm -f
+}
+
+sphinx_precheck() {
+ clean_files
+ echo "PreCheck sphinx doc generation WARNINGS.."
+ make cython3
+
+ pushd docs
+ make clean
+ TVM_TUTORIAL_EXEC_PATTERN=none make html 2>&1 | tee /tmp/$$.log.txt
+ check_sphinx_warnings "docs"
+ popd
+}
+
+
+function join_by { local IFS="$1"; shift; echo "$*"; }
+
+IGNORED_WARNINGS=(
+ '__mro__'
+ 'UserWarning'
+ 'FutureWarning'
+ 'tensorflow'
+ 'Keras'
+ 'pytorch'
+ 'TensorFlow'
+ 'coremltools'
+ '403'
+ 'git describe'
+ 'scikit-learn version'
+ 'doing serial write'
+ 'gen_gallery extension is not safe for parallel'
+ 'strategy:conv2d NHWC layout is not optimized for x86 with autotvm.'
+ 'strategy:depthwise_conv2d NHWC layout is not optimized for x86 with autotvm.'
+ 'autotvm:Cannot find config for target=llvm -keys=cpu -link-params=0'
+ 'autotvm:One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.'
+ 'autotvm:Cannot find config for target=cuda -keys=cuda,gpu'
+)
+
+JOINED_WARNINGS=$(join_by '|' "${IGNORED_WARNINGS[@]}")
Review comment:
ideally the entire CI would be mostly Python and minimal bash instead of the opposite which we have right now. but to keep this PR simple since it does re-arrange a bunch of stuff I think we should leave it as is for now and leave migrating to Python for a follow up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org