You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/07/28 19:30:05 UTC
[GitHub] [tvm] Lunderberg commented on a change in pull request #8576: Enable pytest-xdist for TVM CI
Lunderberg commented on a change in pull request #8576:
URL: https://github.com/apache/tvm/pull/8576#discussion_r678589727
##########
File path: tests/scripts/setup-pytest-env.sh
##########
@@ -32,7 +32,33 @@ export PYTHONPATH="${TVM_PATH}/python"
export TVM_PYTEST_RESULT_DIR="${TVM_PATH}/build/pytest-results"
mkdir -p "${TVM_PYTEST_RESULT_DIR}"
+if [ -n "${CI_PYTEST_NUM_CPUS-}" ]; then
+ PYTEST_NUM_CPUS=${CI_PYTEST_NUM_CPUS}
+else
+ PYTEST_NUM_CPUS=$(nproc)
+ if [ -z "${PYTEST_NUM_CPUS}" ]; then
+ PYTEST_NUM_CPUS=1
+ elif [ ${PYTEST_NUM_CPUS} -gt 1 ]; then
+ PYTEST_NUM_CPUS=$(expr ${PYTEST_NUM_CPUS} - 1) # Don't nuke interactive work.
+ fi
+fi
+
+if [ ${PYTEST_NUM_CPUS} -gt 8 ]; then
+ PYTEST_NUM_CPUS=8 # It usually doesn't make sense to launch > 8 workers
+fi
+
+# DNS: remove after we actually fix up CI_PYTEST_NUM_CPUS in Jenkinsfile
+if [ ${PYTEST_NUM_CPUS} -gt 2 ]; then
+ PYTEST_NUM_CPUS=2 # Fix to 2 CPUs for Jenkins
+fi
+
+
function run_pytest() {
+ local extra_args=( )
+ if [ "$1" == "--parallel" ]; then
Review comment:
Since the documentation [currently recommends](https://github.com/apache/tvm/blob/main/docs/contribute/pull_request.rst#L121) using `task_python_unittest.sh` to run the unit tests, we should either make sure that location also has `pytest-xdist` in the list of packages to install in order to run the tests, or check whether `pytest-xdist` is installed by wrapping this in `if python3 -c "import xdist" > /dev/null 2>&1; then`.
##########
File path: tests/scripts/task_python_unittest.sh
##########
@@ -31,9 +31,9 @@ if [ -z "${TVM_UNITTEST_TESTSUITE_NAME:-}" ]; then
fi
# First run minimal test on both ctypes and cython.
-run_pytest ctypes ${TVM_UNITTEST_TESTSUITE_NAME}-platform-minimal-test tests/python/all-platform-minimal-test
-run_pytest cython ${TVM_UNITTEST_TESTSUITE_NAME}-platform-minimal-test tests/python/all-platform-minimal-test
+run_pytest --parallel ctypes ${TVM_UNITTEST_TESTSUITE_NAME}-platform-minimal-test tests/python/all-platform-minimal-test
Review comment:
Looks like this applies to everything in the unittest directory, including `test_tvm_testing_features.py`. I don't see the [explicit ordering](https://github.com/apache/tvm/pull/8343#discussion_r660038825) that will be required for some of those tests, so we should add it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org