You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Kapoor <ka...@capecode.in> on 2013/02/27 11:29:31 UTC

DistributedShell and YARN application not finishing

Hi,

I am trying to run the DistributedShell exmaple on my hadoop with YARN

Below are the details:

Command used

./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
/home/impadmin/opensource/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
-shell_command ls --num_containers 5 --debug

I can see the application is submitted successfully but the final status is
UNDEFINED also application is not get into finished state.

Screenshot attached

Please assist

Re: DistributedShell and YARN application not finishing

Posted by Sandy Ryza <sa...@cloudera.com>.
Thanks, looks like the silent failure is being addressed in
https://issues.apache.org/jira/browse/YARN-193

On Thu, Feb 28, 2013 at 2:05 AM, <ka...@capecode.in> wrote:

>
> It was just 2 but I did changed to 500 then also it didn't worked.
>
> .
> Thanks and Regards,
> Kapoor
> Phone: 9971061114
>
> -------
> Sent from Blackberry
>
> -----Original Message-----
> From: Sandy Ryza <sa...@cloudera.com>
> Date: Thu, 28 Feb 2013 01:57:49
> To: <ya...@hadoop.apache.org>
> Reply-To: yarn-dev@hadoop.apache.org
> Subject: Re: DistributedShell and YARN application not finishing
>
> Out of curiosity - what was it before?
>
> -Sandy
>
> On Thu, Feb 28, 2013 at 1:31 AM, Kapoor <ka...@capecode.in> wrote:
>
> > It worked now.
> >
> > The changes I did was, increased the value to 8 GB
> >
> >   <property>
> >     <name>yarn.nodemanager.resource.memory-mb</name>
> >     <value>8192</value>
> >     <description>the amount of memory on the NodeManager in
> > GB</description>
> >   </property>
> >
> >
> > On Thu, Feb 28, 2013 at 10:08 AM, Kapoor <ka...@capecode.in> wrote:
> >
> > > Please find the link attached with this email.
> > >
> > > Link for screenshot :
> > > http://picpaste.com/Screenshot_from_2013-02-28_10_03_05-9H0PfXWs.png
> > >
> > > Though I am also looking into this(code debug) and found out that
> > > Application master is never called by RM for the respective jobs.
> > >
> > > Since I am relatively new, it would kind of you,  if you suggest am I
> > > missing something in configuration wise.
> > >
> > >
> > >
> > >
> > >
> > > On Thu, Feb 28, 2013 at 4:51 AM, Vinod Kumar Vavilapalli <
> > > vinodkv@hortonworks.com> wrote:
> > >
> > >>
> > >> Attachments won't go through, please host it somewhere and post a
> link.
> > >>
> > >> Does the total memory on RM webui show a non-zero number?
> > >>
> > >> Thanks,
> > >> +Vinod
> > >>
> > >> On Feb 27, 2013, at 2:29 AM, Kapoor wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > I am trying to run the DistributedShell exmaple on my hadoop with
> YARN
> > >> >
> > >> > Below are the details:
> > >> >
> > >> > Command used
> > >> >
> > >> > ./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client
> > >> -jar
> > >>
> >
> /home/impadmin/opensource/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> > >> -shell_command ls --num_containers 5 --debug
> > >> >
> > >> > I can see the application is submitted successfully but the final
> > >> status is UNDEFINED also application is not get into finished state.
> > >> >
> > >> > Screenshot attached
> > >> >
> > >> > Please assist
> > >>
> > >>
> > >
> >
>
>

Re: DistributedShell and YARN application not finishing

Posted by ka...@capecode.in.
It was just 2 but I did changed to 500 then also it didn't worked.

.
Thanks and Regards,
Kapoor
Phone: 9971061114

-------
Sent from Blackberry

-----Original Message-----
From: Sandy Ryza <sa...@cloudera.com>
Date: Thu, 28 Feb 2013 01:57:49 
To: <ya...@hadoop.apache.org>
Reply-To: yarn-dev@hadoop.apache.org
Subject: Re: DistributedShell and YARN application not finishing

Out of curiosity - what was it before?

-Sandy

On Thu, Feb 28, 2013 at 1:31 AM, Kapoor <ka...@capecode.in> wrote:

> It worked now.
>
> The changes I did was, increased the value to 8 GB
>
>   <property>
>     <name>yarn.nodemanager.resource.memory-mb</name>
>     <value>8192</value>
>     <description>the amount of memory on the NodeManager in
> GB</description>
>   </property>
>
>
> On Thu, Feb 28, 2013 at 10:08 AM, Kapoor <ka...@capecode.in> wrote:
>
> > Please find the link attached with this email.
> >
> > Link for screenshot :
> > http://picpaste.com/Screenshot_from_2013-02-28_10_03_05-9H0PfXWs.png
> >
> > Though I am also looking into this(code debug) and found out that
> > Application master is never called by RM for the respective jobs.
> >
> > Since I am relatively new, it would kind of you,  if you suggest am I
> > missing something in configuration wise.
> >
> >
> >
> >
> >
> > On Thu, Feb 28, 2013 at 4:51 AM, Vinod Kumar Vavilapalli <
> > vinodkv@hortonworks.com> wrote:
> >
> >>
> >> Attachments won't go through, please host it somewhere and post a link.
> >>
> >> Does the total memory on RM webui show a non-zero number?
> >>
> >> Thanks,
> >> +Vinod
> >>
> >> On Feb 27, 2013, at 2:29 AM, Kapoor wrote:
> >>
> >> > Hi,
> >> >
> >> > I am trying to run the DistributedShell exmaple on my hadoop with YARN
> >> >
> >> > Below are the details:
> >> >
> >> > Command used
> >> >
> >> > ./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client
> >> -jar
> >>
> /home/impadmin/opensource/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> >> -shell_command ls --num_containers 5 --debug
> >> >
> >> > I can see the application is submitted successfully but the final
> >> status is UNDEFINED also application is not get into finished state.
> >> >
> >> > Screenshot attached
> >> >
> >> > Please assist
> >>
> >>
> >
>


Re: DistributedShell and YARN application not finishing

Posted by Sandy Ryza <sa...@cloudera.com>.
Out of curiosity - what was it before?

-Sandy

On Thu, Feb 28, 2013 at 1:31 AM, Kapoor <ka...@capecode.in> wrote:

> It worked now.
>
> The changes I did was, increased the value to 8 GB
>
>   <property>
>     <name>yarn.nodemanager.resource.memory-mb</name>
>     <value>8192</value>
>     <description>the amount of memory on the NodeManager in
> GB</description>
>   </property>
>
>
> On Thu, Feb 28, 2013 at 10:08 AM, Kapoor <ka...@capecode.in> wrote:
>
> > Please find the link attached with this email.
> >
> > Link for screenshot :
> > http://picpaste.com/Screenshot_from_2013-02-28_10_03_05-9H0PfXWs.png
> >
> > Though I am also looking into this(code debug) and found out that
> > Application master is never called by RM for the respective jobs.
> >
> > Since I am relatively new, it would kind of you,  if you suggest am I
> > missing something in configuration wise.
> >
> >
> >
> >
> >
> > On Thu, Feb 28, 2013 at 4:51 AM, Vinod Kumar Vavilapalli <
> > vinodkv@hortonworks.com> wrote:
> >
> >>
> >> Attachments won't go through, please host it somewhere and post a link.
> >>
> >> Does the total memory on RM webui show a non-zero number?
> >>
> >> Thanks,
> >> +Vinod
> >>
> >> On Feb 27, 2013, at 2:29 AM, Kapoor wrote:
> >>
> >> > Hi,
> >> >
> >> > I am trying to run the DistributedShell exmaple on my hadoop with YARN
> >> >
> >> > Below are the details:
> >> >
> >> > Command used
> >> >
> >> > ./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client
> >> -jar
> >>
> /home/impadmin/opensource/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> >> -shell_command ls --num_containers 5 --debug
> >> >
> >> > I can see the application is submitted successfully but the final
> >> status is UNDEFINED also application is not get into finished state.
> >> >
> >> > Screenshot attached
> >> >
> >> > Please assist
> >>
> >>
> >
>

Re: DistributedShell and YARN application not finishing

Posted by Kapoor <ka...@capecode.in>.
It worked now.

The changes I did was, increased the value to 8 GB

  <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>8192</value>
    <description>the amount of memory on the NodeManager in GB</description>
  </property>


On Thu, Feb 28, 2013 at 10:08 AM, Kapoor <ka...@capecode.in> wrote:

> Please find the link attached with this email.
>
> Link for screenshot :
> http://picpaste.com/Screenshot_from_2013-02-28_10_03_05-9H0PfXWs.png
>
> Though I am also looking into this(code debug) and found out that
> Application master is never called by RM for the respective jobs.
>
> Since I am relatively new, it would kind of you,  if you suggest am I
> missing something in configuration wise.
>
>
>
>
>
> On Thu, Feb 28, 2013 at 4:51 AM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
>>
>> Attachments won't go through, please host it somewhere and post a link.
>>
>> Does the total memory on RM webui show a non-zero number?
>>
>> Thanks,
>> +Vinod
>>
>> On Feb 27, 2013, at 2:29 AM, Kapoor wrote:
>>
>> > Hi,
>> >
>> > I am trying to run the DistributedShell exmaple on my hadoop with YARN
>> >
>> > Below are the details:
>> >
>> > Command used
>> >
>> > ./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client
>> -jar
>> /home/impadmin/opensource/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command ls --num_containers 5 --debug
>> >
>> > I can see the application is submitted successfully but the final
>> status is UNDEFINED also application is not get into finished state.
>> >
>> > Screenshot attached
>> >
>> > Please assist
>>
>>
>

Re: DistributedShell and YARN application not finishing

Posted by Kapoor <ka...@capecode.in>.
Please find the link attached with this email.

Link for screenshot :
http://picpaste.com/Screenshot_from_2013-02-28_10_03_05-9H0PfXWs.png

Though I am also looking into this(code debug) and found out that
Application master is never called by RM for the respective jobs.

Since I am relatively new, it would kind of you,  if you suggest am I
missing something in configuration wise.





On Thu, Feb 28, 2013 at 4:51 AM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:

>
> Attachments won't go through, please host it somewhere and post a link.
>
> Does the total memory on RM webui show a non-zero number?
>
> Thanks,
> +Vinod
>
> On Feb 27, 2013, at 2:29 AM, Kapoor wrote:
>
> > Hi,
> >
> > I am trying to run the DistributedShell exmaple on my hadoop with YARN
> >
> > Below are the details:
> >
> > Command used
> >
> > ./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client
> -jar
> /home/impadmin/opensource/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
> -shell_command ls --num_containers 5 --debug
> >
> > I can see the application is submitted successfully but the final status
> is UNDEFINED also application is not get into finished state.
> >
> > Screenshot attached
> >
> > Please assist
>
>

Re: DistributedShell and YARN application not finishing

Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Attachments won't go through, please host it somewhere and post a link.

Does the total memory on RM webui show a non-zero number?

Thanks,
+Vinod

On Feb 27, 2013, at 2:29 AM, Kapoor wrote:

> Hi,
> 
> I am trying to run the DistributedShell exmaple on my hadoop with YARN
> 
> Below are the details:
> 
> Command used 
> 
> ./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar /home/impadmin/opensource/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command ls --num_containers 5 --debug
> 
> I can see the application is submitted successfully but the final status is UNDEFINED also application is not get into finished state.
> 
> Screenshot attached
> 
> Please assist