You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/04/23 14:08:17 UTC
[GitHub] [incubator-tvm] u99127 opened a new pull request #5421: [RFC] Pytest environment improvements
u99127 opened a new pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421
In many places having a global pytest flag is useful . For me with the
build and test of tvm , I would like to be able to globally pass in
pytest options as part of development flow or CI flows where one would
like to measure other things regularly that need measurements including
pytest coverage data that I would like to experiment with across the stack.
This has been achieved with an additional setup-pytest-env.sh file in
tests/scripts rather than putting in something in every single task test
script and something I would like to avoid.
This now means the -v option to pytest is superfluous. I did consider
having a pytest.ini file but that doesn't allow me to pass any old
environment variable in and this seems to be the best compromise.
I have also updated the pull request documentation to reflect
the presence of the -k pytest option at the same time though
technically this could be a separate pull request.
@tqchen
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#discussion_r413990735
##########
File path: docs/contribute/pull_request.rst
##########
@@ -118,3 +118,6 @@ If you want to run a single test:
rm -rf python/tvm/*.pyc python/tvm/*/*.pyc python/tvm/*/*/*.pyc
TVM_FFI=ctypes python -m pytest -v tests/python/unittest/test_pass_storage_rewrite.py
+
+ #Additionally if you want to run a single test, for example test_all_elemwise inside a file.
Review comment:
nit : space between `#` and `Additionally`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on a change in pull request #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
u99127 commented on a change in pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#discussion_r414036151
##########
File path: docs/contribute/pull_request.rst
##########
@@ -118,3 +118,6 @@ If you want to run a single test:
rm -rf python/tvm/*.pyc python/tvm/*/*.pyc python/tvm/*/*/*.pyc
TVM_FFI=ctypes python -m pytest -v tests/python/unittest/test_pass_storage_rewrite.py
+
+ #Additionally if you want to run a single test, for example test_all_elemwise inside a file.
Review comment:
Ah, thanks - fixed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on pull request #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-618695859
Thanks @u99127 @tom-gall
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] u99127 edited a comment on issue #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
u99127 edited a comment on issue #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-618416038
I don't like my current hack of overloading "with_same_user" for sourcing this global environment but it seemed like the simplest hack and worked in my environment. Obviously I don't have cuda testing in my CI or my regular test environment which is why I didn't see it. I'll wait for some comments before I work on this again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on issue #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
u99127 commented on issue #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-618416038
I don't like my current hack of overloading "with_same_user" for sourcing this global environment but it seemed like the simplest hack and worked in my environment. Obviously I don't have cuda testing in my CI or my regular test environment, so this isn't fully clear.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on issue #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
u99127 commented on issue #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-618493747
>
>
Thanks for the quick review.
> It would be great if we can avoid the hack into the `with_same_user`. One alternative would be still pass in the `PYTEST_ADDOPTS` env variable from the docker env(for development purposes) but source the setup-pytest-env within each of the script.
>
Indeed , that seems a simple enough solution, here's a rework.
> This also makes the intention of the CI scripts clear, since sometimes we could directly use these scripts. For example, I sometimes would use `./tests/scripts/task_python_unittest.sh` without the docker image to run the testcases
Yeah our wrapping on top uses docker most of the time and that's why I went with the "with_same_user" hack for a prototype but I can see that others may not choose to develop in the same way as us.
R
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-618455854
It would be great if we can avoid the hack into the `with_same_user`. One alternative would be still pass in the `PYTEST_ADDOPTS` env variable from the docker env(for development purposes) but source the setup-pytest-env within each of the script.
This also makes the intention of the CI scripts clear, since sometimes we could directly use these scripts. For example, I sometimes would use `./tests/scripts/task_python_unittest.sh` without the docker image to run the testcases
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tom-gall commented on issue #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
tom-gall commented on issue #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-618499668
> It would be great if we can avoid the hack into the `with_same_user`. One alternative would be still pass in the `PYTEST_ADDOPTS` env variable from the docker env(for development purposes) but source the setup-pytest-env within each of the script.
>
> This also makes the intention of the CI scripts clear, since sometimes we could directly use these scripts. For example, I sometimes would use `./tests/scripts/task_python_unittest.sh` without the docker image to run the testcases
Exactly. For my modest arm dev board environments, where i want/need to run tests, docker just isn't viable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#discussion_r413990735
##########
File path: docs/contribute/pull_request.rst
##########
@@ -118,3 +118,6 @@ If you want to run a single test:
rm -rf python/tvm/*.pyc python/tvm/*/*.pyc python/tvm/*/*/*.pyc
TVM_FFI=ctypes python -m pytest -v tests/python/unittest/test_pass_storage_rewrite.py
+
+ #Additionally if you want to run a single test, for example test_all_elemwise inside a file.
Review comment:
not : space between `#` and `Additionally`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] roastduck commented on pull request #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
roastduck commented on pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-619507703
I have a question. Actually I cannot directly run `./tests/scripts/task_python_unittest.sh` without Docker after merging this PR.
```
tests/scripts/setup-pytest-env.sh: line 23: CI_PYTEST_ADD_OPTIONS: unbound variable
```
What is `CI_PYTEST_ADD_OPTIONS`? Should I set it to an empty string?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on a change in pull request #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#discussion_r413991965
##########
File path: docs/contribute/pull_request.rst
##########
@@ -118,3 +118,6 @@ If you want to run a single test:
rm -rf python/tvm/*.pyc python/tvm/*/*.pyc python/tvm/*/*/*.pyc
TVM_FFI=ctypes python -m pytest -v tests/python/unittest/test_pass_storage_rewrite.py
+
+ #Additionally if you want to run a single test, for example test_all_elemwise inside a file.
Review comment:
` # Additionally if`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] u99127 commented on issue #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
u99127 commented on issue #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-618504136
>
>
> > It would be great if we can avoid the hack into the `with_same_user`. One alternative would be still pass in the `PYTEST_ADDOPTS` env variable from the docker env(for development purposes) but source the setup-pytest-env within each of the script.
> > This also makes the intention of the CI scripts clear, since sometimes we could directly use these scripts. For example, I sometimes would use `./tests/scripts/task_python_unittest.sh` without the docker image to run the testcases
>
> Exactly. For my modest arm dev board environments, where i want/need to run tests, docker just isn't viable.
That is certainly a very interesting orthogonal question as we would need to figure out how best to run things cross as not every single python package tvm depends on is available natively on AArch32 or AArch64.
regards
Ramana
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on pull request #5421: [RFC] Pytest environment improvements
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #5421:
URL: https://github.com/apache/incubator-tvm/pull/5421#issuecomment-619645809
hmm, @u99127 do you mind send a patch? we should detect if CI_PYTEST_ADD_OPTIONS is available and set it to none
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org