You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jens Scheidtmann <je...@gmail.com> on 2013/03/19 09:33:29 UTC
2.0.3-alpha in pseudodristributed mode: How to force examples to run
on yarn?
Dear all,
I finally have set up 2.0.3-alpha in pseudo distributed mode as per
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html
This is evidenced by having yarn resourcemanager and nodemanager running,
no exceptions in the logs and I am able to access their web interfaces. The
node manager has registered in the resourcemanager.
Instead of setting all the listed environment variables, I opted for
setting HADOOP_HOME correctly, and used the supplied start-yarn.sh script.
When running randomwriter as indicated on the above website, I notice that
the local job runner is used, which means only one map task is ever used.
Am I missing a configuration setting to have examples run on yarn?
Is there a way to fail Jobs, when they are using LocalJobRunner?
Thanks in advance,
Jens
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Harsh J <ha...@cloudera.com>.
Do you have a mapred-site.xml configured with mapreduce.framework.name
set to yarn?
On Wed, Mar 20, 2013 at 12:28 AM, Jens Scheidtmann
<je...@gmail.com> wrote:
> Further notes:
>
> I may be mistaken here:
>
> 2013/3/19 Jens Scheidtmann <je...@gmail.com>
>>
>>
>> When running randomwriter as indicated on the above website, I notice that
>> the local job runner is used, which means only one map task is ever used.
>
>
> ...but examples do not register as applications in the resourcetracker and
> randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
> map and 1 GB.
>
> Any hints on how to debug this?
>
> Jens
--
Harsh J
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Harsh J <ha...@cloudera.com>.
Do you have a mapred-site.xml configured with mapreduce.framework.name
set to yarn?
On Wed, Mar 20, 2013 at 12:28 AM, Jens Scheidtmann
<je...@gmail.com> wrote:
> Further notes:
>
> I may be mistaken here:
>
> 2013/3/19 Jens Scheidtmann <je...@gmail.com>
>>
>>
>> When running randomwriter as indicated on the above website, I notice that
>> the local job runner is used, which means only one map task is ever used.
>
>
> ...but examples do not register as applications in the resourcetracker and
> randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
> map and 1 GB.
>
> Any hints on how to debug this?
>
> Jens
--
Harsh J
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Harsh J <ha...@cloudera.com>.
Do you have a mapred-site.xml configured with mapreduce.framework.name
set to yarn?
On Wed, Mar 20, 2013 at 12:28 AM, Jens Scheidtmann
<je...@gmail.com> wrote:
> Further notes:
>
> I may be mistaken here:
>
> 2013/3/19 Jens Scheidtmann <je...@gmail.com>
>>
>>
>> When running randomwriter as indicated on the above website, I notice that
>> the local job runner is used, which means only one map task is ever used.
>
>
> ...but examples do not register as applications in the resourcetracker and
> randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
> map and 1 GB.
>
> Any hints on how to debug this?
>
> Jens
--
Harsh J
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Harsh J <ha...@cloudera.com>.
Do you have a mapred-site.xml configured with mapreduce.framework.name
set to yarn?
On Wed, Mar 20, 2013 at 12:28 AM, Jens Scheidtmann
<je...@gmail.com> wrote:
> Further notes:
>
> I may be mistaken here:
>
> 2013/3/19 Jens Scheidtmann <je...@gmail.com>
>>
>>
>> When running randomwriter as indicated on the above website, I notice that
>> the local job runner is used, which means only one map task is ever used.
>
>
> ...but examples do not register as applications in the resourcetracker and
> randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
> map and 1 GB.
>
> Any hints on how to debug this?
>
> Jens
--
Harsh J
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Jens Scheidtmann <je...@gmail.com>.
Further notes:
I may be mistaken here:
2013/3/19 Jens Scheidtmann <je...@gmail.com>
>
> When running randomwriter as indicated on the above website, I notice that
> the local job runner is used, which means only one map task is ever used.
>
...but examples do not register as applications in the resourcetracker and
randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
map and 1 GB.
Any hints on how to debug this?
Jens
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Jens Scheidtmann <je...@gmail.com>.
Further notes:
I may be mistaken here:
2013/3/19 Jens Scheidtmann <je...@gmail.com>
>
> When running randomwriter as indicated on the above website, I notice that
> the local job runner is used, which means only one map task is ever used.
>
...but examples do not register as applications in the resourcetracker and
randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
map and 1 GB.
Any hints on how to debug this?
Jens
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Jens Scheidtmann <je...@gmail.com>.
Further notes:
I may be mistaken here:
2013/3/19 Jens Scheidtmann <je...@gmail.com>
>
> When running randomwriter as indicated on the above website, I notice that
> the local job runner is used, which means only one map task is ever used.
>
...but examples do not register as applications in the resourcetracker and
randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
map and 1 GB.
Any hints on how to debug this?
Jens
Re: 2.0.3-alpha in pseudodristributed mode: How to force examples to
run on yarn?
Posted by Jens Scheidtmann <je...@gmail.com>.
Further notes:
I may be mistaken here:
2013/3/19 Jens Scheidtmann <je...@gmail.com>
>
> When running randomwriter as indicated on the above website, I notice that
> the local job runner is used, which means only one map task is ever used.
>
...but examples do not register as applications in the resourcetracker and
randomwriter does not write out 10 maps/node x 1 GB but instead does only 1
map and 1 GB.
Any hints on how to debug this?
Jens