You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2021/06/15 13:10:32 UTC
[GitHub] [incubator-mxnet] akarbown opened a new pull request #20352: [WIP] This fix enables running tests on the CI with pytest-parallel
akarbown opened a new pull request #20352:
URL: https://github.com/apache/incubator-mxnet/pull/20352
## Description ##
This proposal of using the pytest-parallel plugin could be the temporary WA for the slowdown in the tests (#20092) after removal of LLVM OpenMP.
Using pytest-parallel plugin (instead of pytest-xdist):
- needs the WA connected with python described [here](https://bugs.python.org/issue38904). Thus, the '--timeout-method=thread' option appeared in the command line and the fixes in the dataloader.py file.
- might result in the deadlocks, as some of the tests may not be thread safe. The one I've encountered is i.e test_ndarray_saveload and this is why I've mark that test as serial. There might be some others but it needs definitely longer testing.
- make the tests run faster than when using pytest-xdist.
For now, it's only the draft that will check/test that fix in the production. In the next steps I'll try to find the root-cause of the slowdown when MxNET is compiled with GNU OpenMP.
## Comments ##
Where can I find any installation guide or other documentation that needs mentioning that change? I'm asking as I couldn't find any.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] Zha0q1 commented on pull request #20352: [WIP] This fix enables running tests on the CI with pytest-parallel
Posted by GitBox <gi...@apache.org>.
Zha0q1 commented on pull request #20352:
URL: https://github.com/apache/incubator-mxnet/pull/20352#issuecomment-865404663
hi @akarbown , what's the status of this pr? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #20352: [WIP] This fix enables running tests on the CI with pytest-parallel
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #20352:
URL: https://github.com/apache/incubator-mxnet/pull/20352#issuecomment-861484549
Hey @akarbown , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [windows-cpu, miscellaneous, sanity, centos-gpu, unix-cpu, edge, clang, unix-gpu, centos-cpu, windows-gpu, website]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] Zha0q1 commented on pull request #20352: [WIP] This fix enables running tests on the CI with pytest-parallel
Posted by GitBox <gi...@apache.org>.
Zha0q1 commented on pull request #20352:
URL: https://github.com/apache/incubator-mxnet/pull/20352#issuecomment-866242249
> I guess that @bgawrych fixed the issue in the following PR: #20367. However, there might be a potential to speed up the CI, but it will require more time and effort to figure out the details. Thus, I'm thinking if this might be useful and dig into it more (here the GPU cases), or withdraw it. @Zha0q1, what do you think?
Thanks for the reply! Do we know if #20367 is the only cause of slowdown? I think faster ci will be favorable but I can monitor the CI runtime dash board for a few days and update back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] Zha0q1 commented on pull request #20352: [WIP] This fix enables running tests on the CI with pytest-parallel
Posted by GitBox <gi...@apache.org>.
Zha0q1 commented on pull request #20352:
URL: https://github.com/apache/incubator-mxnet/pull/20352#issuecomment-865404663
hi @akarbown , what's the status of this pr? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] akarbown commented on pull request #20352: [WIP] This fix enables running tests on the CI with pytest-parallel
Posted by GitBox <gi...@apache.org>.
akarbown commented on pull request #20352:
URL: https://github.com/apache/incubator-mxnet/pull/20352#issuecomment-865680113
I guess that @bgawrych fixed the issue in the following PR: #20367. However, there might be a potential to speed up the CI, but it will require more time and effort to figure out the details. Thus, I'm thinking if this might be useful and dig into it more (here the GPU cases), or withdraw it. @Zha0q1, what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org