You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/03/30 17:35:54 UTC
[GitHub] [incubator-mxnet] ptrendx opened a new pull request #17937: Fix for
handling negative indices in the fusion of slice
ptrendx opened a new pull request #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937
## Description ##
Fixes #17914
@leezu @vafl
## Checklist ##
### Essentials ###
Please feel free to remove inapplicable items for your PR.
- [x] Changes are complete (i.e. I finished coding on this PR)
- [x] All changes have test coverage:
- Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
- [x] To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
### Changes ###
- [x] Handling of the negative axis in the `build_tuple` helper function and added tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-610454793
@szha @ChaiBapchya Is anybody looking at this hang after e.g. `test_np_empty` tests? It would be really useful for the CI admins to hop on the node as it hangs and at least get a stacktrace or something to start figuring out where exactly the failure happens. From my experience those hangs are pretty common.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-612234648
@mxnet-bot run ci [centos-gpu, unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606139184
Hey @ptrendx , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [miscellaneous, website, centos-cpu, unix-cpu, windows-cpu, unix-gpu, centos-gpu, windows-gpu, sanity, edge, clang]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] TaoLv commented on issue #17937: Fix for handling
negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
TaoLv commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606662435
> @leezu Having a patch release would require following a release process again and my understanding is that there is going to be 1.7 release soon. @TaoLv I believe you guys wanted to do 1.7, right? Could you comment on the timeline?
@ptrendx Thank you for the ping. @ciyongch will help to manage the release of 1.7 and will send out the plan and timeline to dev@ soon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-612307273
Yes, I'm going to backport my prs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606176543
@leezu Having a patch release would require following a release process again and my understanding is that there is going to be 1.7 release soon. @TaoLv I believe you guys wanted to do 1.7, right? Could you comment on the timeline?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-612234689
Jenkins CI successfully triggered : [unix-gpu, centos-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-608983400
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu commented on issue #17937: Fix for handling
negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-610528888
@ChaiBapchya the hang occurs after ~10 minutes but the instance will remain available until timeout happens (3 hours). So @ptrendx's suggestion should also work.
I do remember that someone attempted this for the same or a similar issue before? I think there are logs in the internal issue tracker?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx edited a comment on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx edited a comment on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606177797
unix-gpu and unix-cpu failed with gcc segfaulting - @ChaiBapchya have you seen this before?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ciyongch commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ciyongch commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606973304
@ptrendx It's really a great job to complete the journey of MXNet 1.6.0 release! I'm going to manage the release of 1.7 with the support from @TaoLv, and will update the plan later in dev.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-612236018
Added the v1.x label even though this PR goes to master in order to track that we want this included in 1.7 @ciyongch FYI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606231134
Let's test the bot :-)
@mxnet-bot run ci [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606177797
unix-gpu failed with gcc segfaulting - @ChaiBapchya have you seen this before?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya edited a comment on issue #17937: Fix
for handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ChaiBapchya edited a comment on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-610474354
Yes. Getting a stacktrace as it hangs needs to be done before the instance is shut down by the autoscaler [as it is not feasible to disable autoscaler]. Right now only 1 unix-gpu instance is running and the python3 tests have passed. So we really have to time it such that the gpu test fail and the node is alive so that we get the stacktrace.
@leezu found that this might be operator related bug that can be tested locally.
To check the hypothesis, I am trying out testing these files using nosetest in following order:
```
test_contrib_amp test_deferred_compute_gpu test_device test_extensions_gpu test_forward test_fusion test_gluon_gpu test_operator_gpu
```
If I can reproduce it locally then we can ID the issue with the operator test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606187667
Nope.. does it have something to do with your change?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-609941428
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-608983379
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu merged pull request #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
leezu merged pull request #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-611750804
@mxnet-bot run ci [centos-gpu, unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-611750847
Jenkins CI successfully triggered : [unix-gpu, centos-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606189459
I don't believe so - my change does not even touch files that are compiled in CPU only build.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ciyongch commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ciyongch commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-612300778
@ptrendx thanks for point it out, I'm adding this to 1.7.0 roadmap https://github.com/apache/incubator-mxnet/issues/16864.
BTW, are you going to backport this PR to v1.x after it's merged into master?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] leezu commented on issue #17937: Fix for handling
negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606154779
Thanks @ptrendx. Do you plan to backport this change to 1.6 and perhaps have a 1.6.1 release?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-609941366
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] ChaiBapchya commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ChaiBapchya commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-610474354
Yes. Getting a stacktrace as it hangs needs to be done before the instance is shut down by the autoscaler [as it is not feasible to disable autoscaler]. Right now only 1 unix-gpu instance is running and the python3 tests have passed. So we really have to time it such that the gpu test fail and the node is alive so that we get the stacktrace.
@leezu found that this might be operator related bug that can be tested locally.
To check the hypothesis, I am trying out testing these files in following order:
test_contrib_amp test_deferred_compute_gpu test_device test_extensions_gpu test_forward test_fusion test_gluon_gpu test_operator_gpu
If I can reproduce it locally then we can ID the issue with the operator test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #17937: Fix for
handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #17937: Fix for handling negative indices in the fusion of slice
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-606231234
Jenkins CI successfully triggered : [unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] vafl commented on issue #17937: Fix for handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
vafl commented on issue #17937:
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-616788157
> Yes, I'm going to backport my prs.
Thanks for the fix. I'm wondering when are you planning to release a fix for 1.6?
This is a somewhat urgent issue for gluonts.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937:
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-616789589
@vafl There is pretty much 0 chance that something like 1.6.1 would happen before 1.7.0 (at least that 's my understanding, @szha please correct me if I'm wrong), is there anything preventing GluonTS from jumping straight to 1.7?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] ptrendx commented on issue #17937: Fix for handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
ptrendx commented on issue #17937:
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-616808445
I see. Ok, will push it there.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] leezu commented on issue #17937: Fix for handling negative indices in the fusion of slice
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #17937:
URL: https://github.com/apache/incubator-mxnet/pull/17937#issuecomment-616802685
@ptrendx not everyone is using the binary wheels from pypi. Having the fix in the 1.6 branch can still be useful despite not having a 1.6.1 release in the short term
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org