You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Steve Loughran <st...@hortonworks.com> on 2013/09/06 18:02:45 UTC
which notification signasl container termination?
I'm trying to understand when I should be listening to NM vs RM callbacks
to be notified of container failures, and while I could corner colleagues
direct, I'm doing this over email for a searchable result.
my question is simple: how should I be reacting
to NMClientAsync.CallbackHandler.onStopContainerError() and
NMClientAsync.CallbackHandler .onContainerStopped() given there's also
AMRMClientAsync.onContainersCompleted() to listen to?
Can I just ignore the NM events and hook straight into the
AMRMClientAsync.onContainersCompleted() callbacks, which should be telling
me of all problems and anything triggered by NM failures/partitions -or are
AM's expected to be listening for NM events too?
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: which notification signasl container termination?
Posted by Steve Loughran <st...@hortonworks.com>.
thanks. you were the person I was going to ask about this anyway...
I'm just going to log the NM stop events, even the failure ones (start
failed, stop failed)
-Steve
On 7 September 2013 08:42, Vinod Kumar Vavilapalli
<vi...@hortonworks.com>wrote:
>
> NMClientAsync will only do a call-back when it is explicitly asked to stop
> a container. It doesn't track exit of all running containers that it knows
> of. It could, logically, but doesn't because of Hadoop RPC limitations.
>
> You should in most cases be relying on the notification from
> AMRMClientAsync.
>
> Thanks,
> +Vinod
>
> On Sep 6, 2013, at 9:02 AM, Steve Loughran wrote:
>
> > I'm trying to understand when I should be listening to NM vs RM callbacks
> > to be notified of container failures, and while I could corner colleagues
> > direct, I'm doing this over email for a searchable result.
> >
> > my question is simple: how should I be reacting
> > to NMClientAsync.CallbackHandler.onStopContainerError() and
> > NMClientAsync.CallbackHandler .onContainerStopped() given there's also
> > AMRMClientAsync.onContainersCompleted() to listen to?
> >
> > Can I just ignore the NM events and hook straight into the
> > AMRMClientAsync.onContainersCompleted() callbacks, which should be
> telling
> > me of all problems and anything triggered by NM failures/partitions -or
> are
> > AM's expected to be listening for NM events too?
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: which notification signasl container termination?
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
NMClientAsync will only do a call-back when it is explicitly asked to stop a container. It doesn't track exit of all running containers that it knows of. It could, logically, but doesn't because of Hadoop RPC limitations.
You should in most cases be relying on the notification from AMRMClientAsync.
Thanks,
+Vinod
On Sep 6, 2013, at 9:02 AM, Steve Loughran wrote:
> I'm trying to understand when I should be listening to NM vs RM callbacks
> to be notified of container failures, and while I could corner colleagues
> direct, I'm doing this over email for a searchable result.
>
> my question is simple: how should I be reacting
> to NMClientAsync.CallbackHandler.onStopContainerError() and
> NMClientAsync.CallbackHandler .onContainerStopped() given there's also
> AMRMClientAsync.onContainersCompleted() to listen to?
>
> Can I just ignore the NM events and hook straight into the
> AMRMClientAsync.onContainersCompleted() callbacks, which should be telling
> me of all problems and anything triggered by NM failures/partitions -or are
> AM's expected to be listening for NM events too?
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.