You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Vinod Kumar Vavilapalli <vi...@hortonworks.com> on 2014/01/21 19:16:15 UTC
Re: Container's completion issue
It means that the first process in the container is either crashing due to
some reason or explicitly killed by an external entity. You can look at the
logs for the container on the web-UI. Also look at ResourceManager logs to
trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom
application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com>wrote:
> Hi there,
> I was using hadoop-2.2.0 to run an application that only lunch 3
> containers the application master container and 2 more containers that run
> the job. one of the 2 containers is returning as completed from the
> Resource Manager's logs as soon as it is launched. The problem is the
> process that is running with that container continue to run while the
> container's local directory is already cleaned up. I investigated my code
> to understand the reason this happens and could not figure out. Anyone ever
> experience this ? If yes please share. Also please give the details about
> how the containers completion event is triggered and what make it possible
> for a process to continue running after it is marked as completed.
>
>
> Thanks
>
>
> Reyane OUKPEDJO
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: Container's completion issue
Posted by John Lilley <jo...@redpoint.net>.
Are you creating any children? Changing the process group? If so, you may fool the NM into not tracking your child process.
john
From: Vinod Kumar Vavilapalli [mailto:vinodkv@hortonworks.com]
Sent: Tuesday, January 21, 2014 4:22 PM
To: user@hadoop.apache.org; REYANE OUKPEDJO
Subject: Re: Container's completion issue
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com>> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
Hi there,
I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
Thanks
Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: Container's completion issue
Posted by John Lilley <jo...@redpoint.net>.
Are you creating any children? Changing the process group? If so, you may fool the NM into not tracking your child process.
john
From: Vinod Kumar Vavilapalli [mailto:vinodkv@hortonworks.com]
Sent: Tuesday, January 21, 2014 4:22 PM
To: user@hadoop.apache.org; REYANE OUKPEDJO
Subject: Re: Container's completion issue
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com>> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
Hi there,
I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
Thanks
Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: Container's completion issue
Posted by John Lilley <jo...@redpoint.net>.
Are you creating any children? Changing the process group? If so, you may fool the NM into not tracking your child process.
john
From: Vinod Kumar Vavilapalli [mailto:vinodkv@hortonworks.com]
Sent: Tuesday, January 21, 2014 4:22 PM
To: user@hadoop.apache.org; REYANE OUKPEDJO
Subject: Re: Container's completion issue
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com>> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
Hi there,
I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
Thanks
Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: Container's completion issue
Posted by John Lilley <jo...@redpoint.net>.
Are you creating any children? Changing the process group? If so, you may fool the NM into not tracking your child process.
john
From: Vinod Kumar Vavilapalli [mailto:vinodkv@hortonworks.com]
Sent: Tuesday, January 21, 2014 4:22 PM
To: user@hadoop.apache.org; REYANE OUKPEDJO
Subject: Re: Container's completion issue
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com>> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com>> wrote:
Hi there,
I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
Thanks
Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Container's completion issue
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
>
>
>
>
> On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
> It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
>
> Which application is this? MapReduce? Distributed Shell? Or your own custom application?
>
> +Vinod
> Hortonworks Inc.
> http://hortonworks.com/
>
>
> On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> Hi there,
> I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
> Thanks
>
>
> Reyane OUKPEDJO
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Container's completion issue
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
>
>
>
>
> On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
> It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
>
> Which application is this? MapReduce? Distributed Shell? Or your own custom application?
>
> +Vinod
> Hortonworks Inc.
> http://hortonworks.com/
>
>
> On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> Hi there,
> I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
> Thanks
>
>
> Reyane OUKPEDJO
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Container's completion issue
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
>
>
>
>
> On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
> It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
>
> Which application is this? MapReduce? Distributed Shell? Or your own custom application?
>
> +Vinod
> Hortonworks Inc.
> http://hortonworks.com/
>
>
> On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> Hi there,
> I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
> Thanks
>
>
> Reyane OUKPEDJO
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Container's completion issue
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Can you paste the trace for this container on the NodeManager? Specifically any exception messages.
+Vinod
On Jan 21, 2014, at 12:00 PM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
>
>
>
>
> On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
> It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
>
> Which application is this? MapReduce? Distributed Shell? Or your own custom application?
>
> +Vinod
> Hortonworks Inc.
> http://hortonworks.com/
>
>
> On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
> Hi there,
> I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
> Thanks
>
>
> Reyane OUKPEDJO
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Container's completion issue
Posted by REYANE OUKPEDJO <r....@yahoo.com>.
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
Hi there,
>I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
>
>
>Thanks
>
>
>
>
>Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Container's completion issue
Posted by REYANE OUKPEDJO <r....@yahoo.com>.
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
Hi there,
>I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
>
>
>Thanks
>
>
>
>
>Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Container's completion issue
Posted by REYANE OUKPEDJO <r....@yahoo.com>.
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
Hi there,
>I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
>
>
>Thanks
>
>
>
>
>Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Container's completion issue
Posted by REYANE OUKPEDJO <r....@yahoo.com>.
It is my own custom application. But looking at the Resource manager's logs , the container completed as normal with exit code of 0 . This is really weird to me.
On Tuesday, January 21, 2014 1:17 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
It means that the first process in the container is either crashing due to some reason or explicitly killed by an external entity. You can look at the logs for the container on the web-UI. Also look at ResourceManager logs to trace what is happening with this container.
Which application is this? MapReduce? Distributed Shell? Or your own custom application?
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Mon, Jan 20, 2014 at 6:38 AM, REYANE OUKPEDJO <r....@yahoo.com> wrote:
Hi there,
>I was using hadoop-2.2.0 to run an application that only lunch 3 containers the application master container and 2 more containers that run the job. one of the 2 containers is returning as completed from the Resource Manager's logs as soon as it is launched. The problem is the process that is running with that container continue to run while the container's local directory is already cleaned up. I investigated my code to understand the reason this happens and could not figure out. Anyone ever experience this ? If yes please share. Also please give the details about how the containers completion event is triggered and what make it possible for a process to continue running after it is marked as completed.
>
>
>
>
>Thanks
>
>
>
>
>Reyane OUKPEDJO
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.