You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Dhanasekaran Anbalagan <bu...@gmail.com> on 2014/01/02 23:03:44 UTC
Need YARN - Test job.
Hi Guys,
we recently installed CDH5 in our test cluster. we need any test job for
YARN framework.
we are able to run mapreduce job successfully on YARN framework without
code change.
But we need to test yarn job functionality. can you please guide me.
we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
com.hortonworks.simpleyarnapp.Client /bin/date 2
/apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
dvcloudlab231/192.168.70.231:8032
Submitting application application_1388687890867_0007
14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
application_1388687890867_0007 to ResourceManager at dvcloudlab231/
192.168.70.231:8032
Application application_1388687890867_0007 finished with state FAILED at
1388699348344
Note:
In Resource manager node. I don't see any container logs.
-Dhanasekaran.
Did I learn something today? If not, I wasted it.
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Dhahasekaran,
You can use the Hadoop workload to do this testing. The key is to set the
HADOOP_CONF_DIR envar to the right location:
*Yarn*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
*MapReduce*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.mapreduce1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
I hope this helps.
Thanks,
Gunnar
On Thu, Jan 2, 2014 at 3:03 PM, Dhanasekaran Anbalagan
<bu...@gmail.com>wrote:
> Hi Guys,
>
> we recently installed CDH5 in our test cluster. we need any test job for
> YARN framework.
>
> we are able to run mapreduce job successfully on YARN framework without
> code change.
>
> But we need to test yarn job functionality. can you please guide me.
>
> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
> com.hortonworks.simpleyarnapp.Client /bin/date 2
> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
> dvcloudlab231/192.168.70.231:8032
> Submitting application application_1388687890867_0007
> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
> 192.168.70.231:8032
> Application application_1388687890867_0007 finished with state FAILED at
> 1388699348344
>
> Note:
> In Resource manager node. I don't see any container logs.
>
> -Dhanasekaran.
>
> Did I learn something today? If not, I wasted it.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Dhahasekaran,
You can use the Hadoop workload to do this testing. The key is to set the
HADOOP_CONF_DIR envar to the right location:
*Yarn*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
*MapReduce*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.mapreduce1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
I hope this helps.
Thanks,
Gunnar
On Thu, Jan 2, 2014 at 3:03 PM, Dhanasekaran Anbalagan
<bu...@gmail.com>wrote:
> Hi Guys,
>
> we recently installed CDH5 in our test cluster. we need any test job for
> YARN framework.
>
> we are able to run mapreduce job successfully on YARN framework without
> code change.
>
> But we need to test yarn job functionality. can you please guide me.
>
> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
> com.hortonworks.simpleyarnapp.Client /bin/date 2
> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
> dvcloudlab231/192.168.70.231:8032
> Submitting application application_1388687890867_0007
> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
> 192.168.70.231:8032
> Application application_1388687890867_0007 finished with state FAILED at
> 1388699348344
>
> Note:
> In Resource manager node. I don't see any container logs.
>
> -Dhanasekaran.
>
> Did I learn something today? If not, I wasted it.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Dhahasekaran,
You can use the Hadoop workload to do this testing. The key is to set the
HADOOP_CONF_DIR envar to the right location:
*Yarn*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
*MapReduce*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.mapreduce1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
I hope this helps.
Thanks,
Gunnar
On Thu, Jan 2, 2014 at 3:03 PM, Dhanasekaran Anbalagan
<bu...@gmail.com>wrote:
> Hi Guys,
>
> we recently installed CDH5 in our test cluster. we need any test job for
> YARN framework.
>
> we are able to run mapreduce job successfully on YARN framework without
> code change.
>
> But we need to test yarn job functionality. can you please guide me.
>
> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
> com.hortonworks.simpleyarnapp.Client /bin/date 2
> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
> dvcloudlab231/192.168.70.231:8032
> Submitting application application_1388687890867_0007
> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
> 192.168.70.231:8032
> Application application_1388687890867_0007 finished with state FAILED at
> 1388699348344
>
> Note:
> In Resource manager node. I don't see any container logs.
>
> -Dhanasekaran.
>
> Did I learn something today? If not, I wasted it.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Dhahasekaran,
You can use the Hadoop workload to do this testing. The key is to set the
HADOOP_CONF_DIR envar to the right location:
*Yarn*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
*MapReduce*
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.mapreduce1
hadoop fs -rm -r /terasort/TS_input2
yarn jar
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
teragen -D dfs.replication=1 -Dmapred.map.tasks=144 10000
/terasort/TS_input2
I hope this helps.
Thanks,
Gunnar
On Thu, Jan 2, 2014 at 3:03 PM, Dhanasekaran Anbalagan
<bu...@gmail.com>wrote:
> Hi Guys,
>
> we recently installed CDH5 in our test cluster. we need any test job for
> YARN framework.
>
> we are able to run mapreduce job successfully on YARN framework without
> code change.
>
> But we need to test yarn job functionality. can you please guide me.
>
> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
> com.hortonworks.simpleyarnapp.Client /bin/date 2
> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
> dvcloudlab231/192.168.70.231:8032
> Submitting application application_1388687890867_0007
> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
> 192.168.70.231:8032
> Application application_1388687890867_0007 finished with state FAILED at
> 1388699348344
>
> Note:
> In Resource manager node. I don't see any container logs.
>
> -Dhanasekaran.
>
> Did I learn something today? If not, I wasted it.
>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Need YARN - Test job.
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Gaurav,
[BCCing user@h.a.o to move the thread out of it]
It seems you are missing some step when reconfiguring your cluster in
Cloudera Manager, you should have to modify things by hand in your setup.
Adding the Cloudera Manager user alias.
Thanks.
On Fri, Jan 3, 2014 at 7:11 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> yes, i am setting it but still still hangs there..
> Also there are no failure logs, it just hangs without erroring out.
>
> Any log location you want me to look at?
>
>
> On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
>> Hi Guarav,
>>
>> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
>> don't set it as:
>>
>> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>>
>> Gunnar
>>
>>
>> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <sh...@gmail.com>wrote:
>>
>>> Folks,
>>>
>>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>>> no luck.
>>> What i have figured out till now:
>>>
>>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and
>>> TaskTracker.
>>> 2. You can either run MRv1 or MRv2 but not both.
>>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>>> priority for YARN, deployed the Client Configuration (Using Cloudera
>>> Manager).
>>> 4. Submitted the teragen program and can see the YARN framework is in
>>> action from Web UI's.
>>> 5. But it hangs at Map 0% and Reduce %.
>>>
>>> Tried multiple option to no avail.
>>>
>>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in
>>> CDH4.5?
>>>
>>> Attached doc with screen shots.
>>>
>>> Reference:
>>> http://www.cloudera.com/content/cloudera-content/cloudera-
>>> docs/CM4Ent/latest/Cloudera-Manager-Managing-Clusters/cmmc_adding_YARN_
>>> MRv2.html
>>>
>>> Regards
>>> Gaurav
>>>
>>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>>> wrote:
>>>>
>>>> Hi Guys,
>>>>
>>>> we recently installed CDH5 in our test cluster. we need any test job
>>>> for YARN framework.
>>>>
>>>> we are able to run mapreduce job successfully on YARN framework without
>>>> code change.
>>>>
>>>> But we need to test yarn job functionality. can you please guide me.
>>>>
>>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>>> us.
>>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>>> dvcloudlab231/192.168.70.231:8032
>>>> Submitting application application_1388687890867_0007
>>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>>> 192.168.70.231:8032
>>>> Application application_1388687890867_0007 finished with state FAILED
>>>> at 1388699348344
>>>>
>>>> Note:
>>>> In Resource manager node. I don't see any container logs.
>>>>
>>>> -Dhanasekaran.
>>>>
>>>> Did I learn something today? If not, I wasted it.
>>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to cdh-user+u...@cloudera.org.
>>>
>>> For more options, visit https://groups.google.com/a/
>>> cloudera.org/groups/opt_out.
>>>
>>
>>
>>
>> --
>> Thanks,
>>
>> Gunnar
>> *If you think you can you can, if you think you can't you're right.*
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Alejandro
Re: Need YARN - Test job.
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Gaurav,
[BCCing user@h.a.o to move the thread out of it]
It seems you are missing some step when reconfiguring your cluster in
Cloudera Manager, you should have to modify things by hand in your setup.
Adding the Cloudera Manager user alias.
Thanks.
On Fri, Jan 3, 2014 at 7:11 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> yes, i am setting it but still still hangs there..
> Also there are no failure logs, it just hangs without erroring out.
>
> Any log location you want me to look at?
>
>
> On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
>> Hi Guarav,
>>
>> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
>> don't set it as:
>>
>> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>>
>> Gunnar
>>
>>
>> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <sh...@gmail.com>wrote:
>>
>>> Folks,
>>>
>>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>>> no luck.
>>> What i have figured out till now:
>>>
>>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and
>>> TaskTracker.
>>> 2. You can either run MRv1 or MRv2 but not both.
>>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>>> priority for YARN, deployed the Client Configuration (Using Cloudera
>>> Manager).
>>> 4. Submitted the teragen program and can see the YARN framework is in
>>> action from Web UI's.
>>> 5. But it hangs at Map 0% and Reduce %.
>>>
>>> Tried multiple option to no avail.
>>>
>>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in
>>> CDH4.5?
>>>
>>> Attached doc with screen shots.
>>>
>>> Reference:
>>> http://www.cloudera.com/content/cloudera-content/cloudera-
>>> docs/CM4Ent/latest/Cloudera-Manager-Managing-Clusters/cmmc_adding_YARN_
>>> MRv2.html
>>>
>>> Regards
>>> Gaurav
>>>
>>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>>> wrote:
>>>>
>>>> Hi Guys,
>>>>
>>>> we recently installed CDH5 in our test cluster. we need any test job
>>>> for YARN framework.
>>>>
>>>> we are able to run mapreduce job successfully on YARN framework without
>>>> code change.
>>>>
>>>> But we need to test yarn job functionality. can you please guide me.
>>>>
>>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>>> us.
>>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>>> dvcloudlab231/192.168.70.231:8032
>>>> Submitting application application_1388687890867_0007
>>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>>> 192.168.70.231:8032
>>>> Application application_1388687890867_0007 finished with state FAILED
>>>> at 1388699348344
>>>>
>>>> Note:
>>>> In Resource manager node. I don't see any container logs.
>>>>
>>>> -Dhanasekaran.
>>>>
>>>> Did I learn something today? If not, I wasted it.
>>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to cdh-user+u...@cloudera.org.
>>>
>>> For more options, visit https://groups.google.com/a/
>>> cloudera.org/groups/opt_out.
>>>
>>
>>
>>
>> --
>> Thanks,
>>
>> Gunnar
>> *If you think you can you can, if you think you can't you're right.*
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Alejandro
Re: Need YARN - Test job.
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Gaurav,
[BCCing user@h.a.o to move the thread out of it]
It seems you are missing some step when reconfiguring your cluster in
Cloudera Manager, you should have to modify things by hand in your setup.
Adding the Cloudera Manager user alias.
Thanks.
On Fri, Jan 3, 2014 at 7:11 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> yes, i am setting it but still still hangs there..
> Also there are no failure logs, it just hangs without erroring out.
>
> Any log location you want me to look at?
>
>
> On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
>> Hi Guarav,
>>
>> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
>> don't set it as:
>>
>> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>>
>> Gunnar
>>
>>
>> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <sh...@gmail.com>wrote:
>>
>>> Folks,
>>>
>>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>>> no luck.
>>> What i have figured out till now:
>>>
>>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and
>>> TaskTracker.
>>> 2. You can either run MRv1 or MRv2 but not both.
>>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>>> priority for YARN, deployed the Client Configuration (Using Cloudera
>>> Manager).
>>> 4. Submitted the teragen program and can see the YARN framework is in
>>> action from Web UI's.
>>> 5. But it hangs at Map 0% and Reduce %.
>>>
>>> Tried multiple option to no avail.
>>>
>>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in
>>> CDH4.5?
>>>
>>> Attached doc with screen shots.
>>>
>>> Reference:
>>> http://www.cloudera.com/content/cloudera-content/cloudera-
>>> docs/CM4Ent/latest/Cloudera-Manager-Managing-Clusters/cmmc_adding_YARN_
>>> MRv2.html
>>>
>>> Regards
>>> Gaurav
>>>
>>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>>> wrote:
>>>>
>>>> Hi Guys,
>>>>
>>>> we recently installed CDH5 in our test cluster. we need any test job
>>>> for YARN framework.
>>>>
>>>> we are able to run mapreduce job successfully on YARN framework without
>>>> code change.
>>>>
>>>> But we need to test yarn job functionality. can you please guide me.
>>>>
>>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>>> us.
>>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>>> dvcloudlab231/192.168.70.231:8032
>>>> Submitting application application_1388687890867_0007
>>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>>> 192.168.70.231:8032
>>>> Application application_1388687890867_0007 finished with state FAILED
>>>> at 1388699348344
>>>>
>>>> Note:
>>>> In Resource manager node. I don't see any container logs.
>>>>
>>>> -Dhanasekaran.
>>>>
>>>> Did I learn something today? If not, I wasted it.
>>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to cdh-user+u...@cloudera.org.
>>>
>>> For more options, visit https://groups.google.com/a/
>>> cloudera.org/groups/opt_out.
>>>
>>
>>
>>
>> --
>> Thanks,
>>
>> Gunnar
>> *If you think you can you can, if you think you can't you're right.*
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Alejandro
Re: Need YARN - Test job.
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Gaurav,
[BCCing user@h.a.o to move the thread out of it]
It seems you are missing some step when reconfiguring your cluster in
Cloudera Manager, you should have to modify things by hand in your setup.
Adding the Cloudera Manager user alias.
Thanks.
On Fri, Jan 3, 2014 at 7:11 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> yes, i am setting it but still still hangs there..
> Also there are no failure logs, it just hangs without erroring out.
>
> Any log location you want me to look at?
>
>
> On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
>> Hi Guarav,
>>
>> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
>> don't set it as:
>>
>> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>>
>> Gunnar
>>
>>
>> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <sh...@gmail.com>wrote:
>>
>>> Folks,
>>>
>>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>>> no luck.
>>> What i have figured out till now:
>>>
>>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and
>>> TaskTracker.
>>> 2. You can either run MRv1 or MRv2 but not both.
>>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>>> priority for YARN, deployed the Client Configuration (Using Cloudera
>>> Manager).
>>> 4. Submitted the teragen program and can see the YARN framework is in
>>> action from Web UI's.
>>> 5. But it hangs at Map 0% and Reduce %.
>>>
>>> Tried multiple option to no avail.
>>>
>>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in
>>> CDH4.5?
>>>
>>> Attached doc with screen shots.
>>>
>>> Reference:
>>> http://www.cloudera.com/content/cloudera-content/cloudera-
>>> docs/CM4Ent/latest/Cloudera-Manager-Managing-Clusters/cmmc_adding_YARN_
>>> MRv2.html
>>>
>>> Regards
>>> Gaurav
>>>
>>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>>> wrote:
>>>>
>>>> Hi Guys,
>>>>
>>>> we recently installed CDH5 in our test cluster. we need any test job
>>>> for YARN framework.
>>>>
>>>> we are able to run mapreduce job successfully on YARN framework without
>>>> code change.
>>>>
>>>> But we need to test yarn job functionality. can you please guide me.
>>>>
>>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>>> us.
>>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>>> dvcloudlab231/192.168.70.231:8032
>>>> Submitting application application_1388687890867_0007
>>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>>> 192.168.70.231:8032
>>>> Application application_1388687890867_0007 finished with state FAILED
>>>> at 1388699348344
>>>>
>>>> Note:
>>>> In Resource manager node. I don't see any container logs.
>>>>
>>>> -Dhanasekaran.
>>>>
>>>> Did I learn something today? If not, I wasted it.
>>>>
>>> --
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "CDH Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to cdh-user+u...@cloudera.org.
>>>
>>> For more options, visit https://groups.google.com/a/
>>> cloudera.org/groups/opt_out.
>>>
>>
>>
>>
>> --
>> Thanks,
>>
>> Gunnar
>> *If you think you can you can, if you think you can't you're right.*
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Alejandro
Re: Need YARN - Test job.
Posted by Gaurav Shankhdhar <sh...@gmail.com>.
yes, i am setting it but still still hangs there..
Also there are no failure logs, it just hangs without erroring out.
Any log location you want me to look at?
On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
> Hi Guarav,
>
> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
> don't set it as:
>
> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>
> Gunnar
>
>
> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <shankhdh...@gmail.com<javascript:>
> > wrote:
>
>> Folks,
>>
>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>> no luck.
>> What i have figured out till now:
>>
>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
>> 2. You can either run MRv1 or MRv2 but not both.
>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>> priority for YARN, deployed the Client Configuration (Using Cloudera
>> Manager).
>> 4. Submitted the teragen program and can see the YARN framework is in
>> action from Web UI's.
>> 5. But it hangs at Map 0% and Reduce %.
>>
>> Tried multiple option to no avail.
>>
>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>>
>> Attached doc with screen shots.
>>
>> Reference:
>> http://www.cloudera.com/content/cloudera-content/
>> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
>> Clusters/cmmc_adding_YARN_MRv2.html
>>
>> Regards
>> Gaurav
>>
>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>> wrote:
>>>
>>> Hi Guys,
>>>
>>> we recently installed CDH5 in our test cluster. we need any test job for
>>> YARN framework.
>>>
>>> we are able to run mapreduce job successfully on YARN framework without
>>> code change.
>>>
>>> But we need to test yarn job functionality. can you please guide me.
>>>
>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>> us.
>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>> dvcloudlab231/192.168.70.231:8032
>>> Submitting application application_1388687890867_0007
>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>> 192.168.70.231:8032
>>> Application application_1388687890867_0007 finished with state FAILED at
>>> 1388699348344
>>>
>>> Note:
>>> In Resource manager node. I don't see any container logs.
>>>
>>> -Dhanasekaran.
>>>
>>> Did I learn something today? If not, I wasted it.
>>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CDH Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cdh-user+u...@cloudera.org <javascript:>.
>> For more options, visit
>> https://groups.google.com/a/cloudera.org/groups/opt_out.
>>
>
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
Re: Need YARN - Test job.
Posted by Gaurav Shankhdhar <sh...@gmail.com>.
yes, i am setting it but still still hangs there..
Also there are no failure logs, it just hangs without erroring out.
Any log location you want me to look at?
On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
> Hi Guarav,
>
> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
> don't set it as:
>
> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>
> Gunnar
>
>
> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <shankhdh...@gmail.com<javascript:>
> > wrote:
>
>> Folks,
>>
>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>> no luck.
>> What i have figured out till now:
>>
>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
>> 2. You can either run MRv1 or MRv2 but not both.
>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>> priority for YARN, deployed the Client Configuration (Using Cloudera
>> Manager).
>> 4. Submitted the teragen program and can see the YARN framework is in
>> action from Web UI's.
>> 5. But it hangs at Map 0% and Reduce %.
>>
>> Tried multiple option to no avail.
>>
>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>>
>> Attached doc with screen shots.
>>
>> Reference:
>> http://www.cloudera.com/content/cloudera-content/
>> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
>> Clusters/cmmc_adding_YARN_MRv2.html
>>
>> Regards
>> Gaurav
>>
>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>> wrote:
>>>
>>> Hi Guys,
>>>
>>> we recently installed CDH5 in our test cluster. we need any test job for
>>> YARN framework.
>>>
>>> we are able to run mapreduce job successfully on YARN framework without
>>> code change.
>>>
>>> But we need to test yarn job functionality. can you please guide me.
>>>
>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>> us.
>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>> dvcloudlab231/192.168.70.231:8032
>>> Submitting application application_1388687890867_0007
>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>> 192.168.70.231:8032
>>> Application application_1388687890867_0007 finished with state FAILED at
>>> 1388699348344
>>>
>>> Note:
>>> In Resource manager node. I don't see any container logs.
>>>
>>> -Dhanasekaran.
>>>
>>> Did I learn something today? If not, I wasted it.
>>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CDH Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cdh-user+u...@cloudera.org <javascript:>.
>> For more options, visit
>> https://groups.google.com/a/cloudera.org/groups/opt_out.
>>
>
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
Re: Need YARN - Test job.
Posted by Gaurav Shankhdhar <sh...@gmail.com>.
yes, i am setting it but still still hangs there..
Also there are no failure logs, it just hangs without erroring out.
Any log location you want me to look at?
On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
> Hi Guarav,
>
> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
> don't set it as:
>
> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>
> Gunnar
>
>
> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <shankhdh...@gmail.com<javascript:>
> > wrote:
>
>> Folks,
>>
>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>> no luck.
>> What i have figured out till now:
>>
>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
>> 2. You can either run MRv1 or MRv2 but not both.
>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>> priority for YARN, deployed the Client Configuration (Using Cloudera
>> Manager).
>> 4. Submitted the teragen program and can see the YARN framework is in
>> action from Web UI's.
>> 5. But it hangs at Map 0% and Reduce %.
>>
>> Tried multiple option to no avail.
>>
>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>>
>> Attached doc with screen shots.
>>
>> Reference:
>> http://www.cloudera.com/content/cloudera-content/
>> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
>> Clusters/cmmc_adding_YARN_MRv2.html
>>
>> Regards
>> Gaurav
>>
>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>> wrote:
>>>
>>> Hi Guys,
>>>
>>> we recently installed CDH5 in our test cluster. we need any test job for
>>> YARN framework.
>>>
>>> we are able to run mapreduce job successfully on YARN framework without
>>> code change.
>>>
>>> But we need to test yarn job functionality. can you please guide me.
>>>
>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>> us.
>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>> dvcloudlab231/192.168.70.231:8032
>>> Submitting application application_1388687890867_0007
>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>> 192.168.70.231:8032
>>> Application application_1388687890867_0007 finished with state FAILED at
>>> 1388699348344
>>>
>>> Note:
>>> In Resource manager node. I don't see any container logs.
>>>
>>> -Dhanasekaran.
>>>
>>> Did I learn something today? If not, I wasted it.
>>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CDH Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cdh-user+u...@cloudera.org <javascript:>.
>> For more options, visit
>> https://groups.google.com/a/cloudera.org/groups/opt_out.
>>
>
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
Re: Need YARN - Test job.
Posted by Gaurav Shankhdhar <sh...@gmail.com>.
yes, i am setting it but still still hangs there..
Also there are no failure logs, it just hangs without erroring out.
Any log location you want me to look at?
On Friday, January 3, 2014 8:38:16 PM UTC+5:30, Gunnar Tapper wrote:
>
> Hi Guarav,
>
> What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you
> don't set it as:
>
> export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
>
> Gunnar
>
>
> On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <shankhdh...@gmail.com<javascript:>
> > wrote:
>
>> Folks,
>>
>> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
>> no luck.
>> What i have figured out till now:
>>
>> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
>> 2. You can either run MRv1 or MRv2 but not both.
>> 3. I disabled MRv1 and configured YARN by increasing the alternatives
>> priority for YARN, deployed the Client Configuration (Using Cloudera
>> Manager).
>> 4. Submitted the teragen program and can see the YARN framework is in
>> action from Web UI's.
>> 5. But it hangs at Map 0% and Reduce %.
>>
>> Tried multiple option to no avail.
>>
>> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>>
>> Attached doc with screen shots.
>>
>> Reference:
>> http://www.cloudera.com/content/cloudera-content/
>> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
>> Clusters/cmmc_adding_YARN_MRv2.html
>>
>> Regards
>> Gaurav
>>
>> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
>> wrote:
>>>
>>> Hi Guys,
>>>
>>> we recently installed CDH5 in our test cluster. we need any test job for
>>> YARN framework.
>>>
>>> we are able to run mapreduce job successfully on YARN framework without
>>> code change.
>>>
>>> But we need to test yarn job functionality. can you please guide me.
>>>
>>> we tired https://github.com/hortonworks/simple-yarn-app it's not help
>>> us.
>>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>>> dvcloudlab231/192.168.70.231:8032
>>> Submitting application application_1388687890867_0007
>>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>>> 192.168.70.231:8032
>>> Application application_1388687890867_0007 finished with state FAILED at
>>> 1388699348344
>>>
>>> Note:
>>> In Resource manager node. I don't see any container logs.
>>>
>>> -Dhanasekaran.
>>>
>>> Did I learn something today? If not, I wasted it.
>>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CDH Users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cdh-user+u...@cloudera.org <javascript:>.
>> For more options, visit
>> https://groups.google.com/a/cloudera.org/groups/opt_out.
>>
>
>
>
> --
> Thanks,
>
> Gunnar
> *If you think you can you can, if you think you can't you're right.*
>
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Guarav,
What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you don't
set it as:
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
Gunnar
On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> Folks,
>
> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
> no luck.
> What i have figured out till now:
>
> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
> 2. You can either run MRv1 or MRv2 but not both.
> 3. I disabled MRv1 and configured YARN by increasing the alternatives
> priority for YARN, deployed the Client Configuration (Using Cloudera
> Manager).
> 4. Submitted the teragen program and can see the YARN framework is in
> action from Web UI's.
> 5. But it hangs at Map 0% and Reduce %.
>
> Tried multiple option to no avail.
>
> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>
> Attached doc with screen shots.
>
> Reference:
> http://www.cloudera.com/content/cloudera-content/
> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
> Clusters/cmmc_adding_YARN_MRv2.html
>
> Regards
> Gaurav
>
> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
> wrote:
>>
>> Hi Guys,
>>
>> we recently installed CDH5 in our test cluster. we need any test job for
>> YARN framework.
>>
>> we are able to run mapreduce job successfully on YARN framework without
>> code change.
>>
>> But we need to test yarn job functionality. can you please guide me.
>>
>> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>> dvcloudlab231/192.168.70.231:8032
>> Submitting application application_1388687890867_0007
>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>> 192.168.70.231:8032
>> Application application_1388687890867_0007 finished with state FAILED at
>> 1388699348344
>>
>> Note:
>> In Resource manager node. I don't see any container logs.
>>
>> -Dhanasekaran.
>>
>> Did I learn something today? If not, I wasted it.
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Guarav,
What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you don't
set it as:
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
Gunnar
On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> Folks,
>
> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
> no luck.
> What i have figured out till now:
>
> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
> 2. You can either run MRv1 or MRv2 but not both.
> 3. I disabled MRv1 and configured YARN by increasing the alternatives
> priority for YARN, deployed the Client Configuration (Using Cloudera
> Manager).
> 4. Submitted the teragen program and can see the YARN framework is in
> action from Web UI's.
> 5. But it hangs at Map 0% and Reduce %.
>
> Tried multiple option to no avail.
>
> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>
> Attached doc with screen shots.
>
> Reference:
> http://www.cloudera.com/content/cloudera-content/
> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
> Clusters/cmmc_adding_YARN_MRv2.html
>
> Regards
> Gaurav
>
> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
> wrote:
>>
>> Hi Guys,
>>
>> we recently installed CDH5 in our test cluster. we need any test job for
>> YARN framework.
>>
>> we are able to run mapreduce job successfully on YARN framework without
>> code change.
>>
>> But we need to test yarn job functionality. can you please guide me.
>>
>> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>> dvcloudlab231/192.168.70.231:8032
>> Submitting application application_1388687890867_0007
>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>> 192.168.70.231:8032
>> Application application_1388687890867_0007 finished with state FAILED at
>> 1388699348344
>>
>> Note:
>> In Resource manager node. I don't see any container logs.
>>
>> -Dhanasekaran.
>>
>> Did I learn something today? If not, I wasted it.
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Guarav,
What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you don't
set it as:
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
Gunnar
On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> Folks,
>
> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
> no luck.
> What i have figured out till now:
>
> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
> 2. You can either run MRv1 or MRv2 but not both.
> 3. I disabled MRv1 and configured YARN by increasing the alternatives
> priority for YARN, deployed the Client Configuration (Using Cloudera
> Manager).
> 4. Submitted the teragen program and can see the YARN framework is in
> action from Web UI's.
> 5. But it hangs at Map 0% and Reduce %.
>
> Tried multiple option to no avail.
>
> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>
> Attached doc with screen shots.
>
> Reference:
> http://www.cloudera.com/content/cloudera-content/
> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
> Clusters/cmmc_adding_YARN_MRv2.html
>
> Regards
> Gaurav
>
> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
> wrote:
>>
>> Hi Guys,
>>
>> we recently installed CDH5 in our test cluster. we need any test job for
>> YARN framework.
>>
>> we are able to run mapreduce job successfully on YARN framework without
>> code change.
>>
>> But we need to test yarn job functionality. can you please guide me.
>>
>> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>> dvcloudlab231/192.168.70.231:8032
>> Submitting application application_1388687890867_0007
>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>> 192.168.70.231:8032
>> Application application_1388687890867_0007 finished with state FAILED at
>> 1388699348344
>>
>> Note:
>> In Resource manager node. I don't see any container logs.
>>
>> -Dhanasekaran.
>>
>> Did I learn something today? If not, I wasted it.
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Need YARN - Test job.
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Guarav,
What are you setting HADOOP_CONF_DIR to? IME, you get the hang if you don't
set it as:
export HADOOP_CONF_DIR=/etc/hadoop/conf.cloudera.yarn1
Gunnar
On Fri, Jan 3, 2014 at 6:47 AM, Gaurav Shankhdhar <
shankhdhar.gaurav@gmail.com> wrote:
> Folks,
>
> I am trying to run "teragen" program using YARN framework in CDH 4.5 but
> no luck.
> What i have figured out till now:
>
> 1. By default Demo VM run the MR in MRv1 uses JobTracker and TaskTracker.
> 2. You can either run MRv1 or MRv2 but not both.
> 3. I disabled MRv1 and configured YARN by increasing the alternatives
> priority for YARN, deployed the Client Configuration (Using Cloudera
> Manager).
> 4. Submitted the teragen program and can see the YARN framework is in
> action from Web UI's.
> 5. But it hangs at Map 0% and Reduce %.
>
> Tried multiple option to no avail.
>
> Any step by Step guide or doc to execute MR in MRv2 (YARN) mode in CDH4.5?
>
> Attached doc with screen shots.
>
> Reference:
> http://www.cloudera.com/content/cloudera-content/
> cloudera-docs/CM4Ent/latest/Cloudera-Manager-Managing-
> Clusters/cmmc_adding_YARN_MRv2.html
>
> Regards
> Gaurav
>
> On Friday, January 3, 2014 3:33:44 AM UTC+5:30, Dhanasekaran Anbalagan
> wrote:
>>
>> Hi Guys,
>>
>> we recently installed CDH5 in our test cluster. we need any test job for
>> YARN framework.
>>
>> we are able to run mapreduce job successfully on YARN framework without
>> code change.
>>
>> But we need to test yarn job functionality. can you please guide me.
>>
>> we tired https://github.com/hortonworks/simple-yarn-app it's not help us.
>> tech@dvcloudlab231:~$ *hadoop jar simple-yarn-app-1.0-SNAPSHOT.jar
>> com.hortonworks.simpleyarnapp.Client /bin/date 2
>> /apps/simple/simple-yarn-app-1.0-SNAPSHOT.jar*
>> 14/01/02 16:49:05 INFO client.RMProxy: Connecting to ResourceManager at
>> dvcloudlab231/192.168.70.231:8032
>> Submitting application application_1388687890867_0007
>> 14/01/02 16:49:05 INFO impl.YarnClientImpl: Submitted application
>> application_1388687890867_0007 to ResourceManager at dvcloudlab231/
>> 192.168.70.231:8032
>> Application application_1388687890867_0007 finished with state FAILED at
>> 1388699348344
>>
>> Note:
>> In Resource manager node. I don't see any container logs.
>>
>> -Dhanasekaran.
>>
>> Did I learn something today? If not, I wasted it.
>>
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "CDH Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cdh-user+unsubscribe@cloudera.org.
> For more options, visit
> https://groups.google.com/a/cloudera.org/groups/opt_out.
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*