You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mxnet.apache.org by Chris Olivier <cj...@gmail.com> on 2018/03/29 17:44:31 UTC
Killed builds
I killed several builds which were > 11 hours old -- all stuck at this
python3 GPU hang problem
Re: Killed builds
Posted by Chris Olivier <cj...@gmail.com>.
Kellen got the call stacks. I reproduced once so far locally.
On Thu, Mar 29, 2018 at 2:55 PM, Marco de Abreu <
marco.g.abreu@googlemail.com> wrote:
> Thank you, Chris!
>
> What's interesting here (e.g. at [1]) is the matter of the fact that all
> tests are actually finishing, but the process does not terminate. I have
> experienced such a behaviour in my past C# and Java projects. In these
> cases, it was related to threads being created as non-foreground-threads or
> threadpools that have not been disposed and thus causing the process to
> stay alive until explicitly terminated. Does anybody remember a change to
> the threading in the last days or has a better idea what this could be
> related to?
>
> -Marco
>
> [1]:
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/
> incubator-mxnet/detail/PR-10308/5/pipeline/586
>
> On Thu, Mar 29, 2018 at 7:44 PM, Chris Olivier <cj...@gmail.com>
> wrote:
>
> > I killed several builds which were > 11 hours old -- all stuck at this
> > python3 GPU hang problem
> >
>
Re: Killed builds
Posted by Marco de Abreu <ma...@googlemail.com>.
Thank you, Chris!
What's interesting here (e.g. at [1]) is the matter of the fact that all
tests are actually finishing, but the process does not terminate. I have
experienced such a behaviour in my past C# and Java projects. In these
cases, it was related to threads being created as non-foreground-threads or
threadpools that have not been disposed and thus causing the process to
stay alive until explicitly terminated. Does anybody remember a change to
the threading in the last days or has a better idea what this could be
related to?
-Marco
[1]:
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-10308/5/pipeline/586
On Thu, Mar 29, 2018 at 7:44 PM, Chris Olivier <cj...@gmail.com>
wrote:
> I killed several builds which were > 11 hours old -- all stuck at this
> python3 GPU hang problem
>