You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/06/15 15:33:43 UTC
[GitHub] [incubator-tvm] tqchen opened a new pull request #5815: [CI] Limit number of threads in all jobs
tqchen opened a new pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815
To reduce pressure on a crowded CI setting.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#discussion_r440269311
##########
File path: tests/scripts/task_python_vta_fsim.sh
##########
@@ -20,6 +20,10 @@ set -e
set -u
source tests/scripts/setup-pytest-env.sh
+# to avoid CI thread throttling.
+export TVM_BIND_THREADS=0
+export OMP_NUM_THREADS=1
+
Review comment:
I have thought about that. However we will also need to setup the same env for rust and cpp tests, and different test might have a different threading setting (we want to have one multi-threaded case).
So i end up not doing so. An alternative might be try to directly limit the resources of the docker container. Will look into it and report back
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
u99127 commented on a change in pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#discussion_r440267908
##########
File path: tests/scripts/task_python_vta_fsim.sh
##########
@@ -20,6 +20,10 @@ set -e
set -u
source tests/scripts/setup-pytest-env.sh
+# to avoid CI thread throttling.
+export TVM_BIND_THREADS=0
+export OMP_NUM_THREADS=1
+
Review comment:
Why not move this into setup-pytest-env.sh and maybe rename it to setup-test-env.sh if you think it's too pytest centric ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#discussion_r440269311
##########
File path: tests/scripts/task_python_vta_fsim.sh
##########
@@ -20,6 +20,10 @@ set -e
set -u
source tests/scripts/setup-pytest-env.sh
+# to avoid CI thread throttling.
+export TVM_BIND_THREADS=0
+export OMP_NUM_THREADS=1
+
Review comment:
I have thought about that. However we will also need to setup the same env for rust and cpp tests, and different specific test might have a different threading setting. So i end up not doing so. An alternative might be try to directly limit the resources of the docker container. Will look into it and report back
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#issuecomment-644207809
cc @yzhliu @zhiics @icemelon9 @tmoreau89
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#discussion_r440328308
##########
File path: tests/scripts/task_python_vta_fsim.sh
##########
@@ -20,6 +20,10 @@ set -e
set -u
source tests/scripts/setup-pytest-env.sh
+# to avoid CI thread throttling.
+export TVM_BIND_THREADS=0
+export OMP_NUM_THREADS=1
+
Review comment:
I meant https://docs.docker.com/config/containers/resource_constraints/, to directly restrict the CPUs that a container can access physically. although we might need some experimentation to see which option is the best, and how are they presented to the runtime.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#discussion_r440269311
##########
File path: tests/scripts/task_python_vta_fsim.sh
##########
@@ -20,6 +20,10 @@ set -e
set -u
source tests/scripts/setup-pytest-env.sh
+# to avoid CI thread throttling.
+export TVM_BIND_THREADS=0
+export OMP_NUM_THREADS=1
+
Review comment:
I have thought about that. However we will also need to setup the same env for rust and cpp tests, and different test might have a different threading setting (we want to have one multi-threaded case).
So i end up concluded that it is worth the duplication and not doing so. An alternative might be try to directly limit the resources of the docker container. Will look into it and report back
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
u99127 commented on a change in pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#discussion_r440319264
##########
File path: tests/scripts/task_python_vta_fsim.sh
##########
@@ -20,6 +20,10 @@ set -e
set -u
source tests/scripts/setup-pytest-env.sh
+# to avoid CI thread throttling.
+export TVM_BIND_THREADS=0
+export OMP_NUM_THREADS=1
+
Review comment:
Thanks for the reply. Agreed on the rust and cpp environments which is why I suggested setup-test-env.sh :). On the second point I would have thought the lower level driver script be able to override the environment variable for the test ?
I'd like to understand more about what you mean by directly limiting the resources of the container.
regards
Ramana
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815#issuecomment-644268424
Merging for now as the current loc changes is small, we can do more delibration about option consolidation as follow ups, Thanks @u99127 @zhiics !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen merged pull request #5815: [CI] Limit number of threads in all jobs
Posted by GitBox <gi...@apache.org>.
tqchen merged pull request #5815:
URL: https://github.com/apache/incubator-tvm/pull/5815
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org