Config setting for staging.dir?

[Dolan Antenucci <an...@gmail.com>]

Is there a way to change the staging directory in the config files?  Mine
currently goes to "/<tmp-folder>/staging".

I read someone else set mapreduce.jobtracker.staging.root.dir, but I have
had no luck with this (or slightly different versions of it) in my config
files.  I'm guessing this other user set it in their code?   How would I do
that if I run a command like, "hadoop jar
/usr/lib/hadoop/hadoop-*-examples.jar pi 2 100000"?  My preference though
would be to overwrite this setting in the config files so that every user
wouldn't have to specify it (i.e. assuming I don't let them use the temp
folder).

Thanks

-Dolan Antenucci

Re: Config setting for staging.dir?

[Dolan Antenucci <an...@gmail.com>]

After retrying that setting in mapred-site.xml, that is now working.  I last
had "mapred.jobtracker..." in the file, so I may have just had the setting
misspelled, or failed to restart when I had it spelled correctly. Either
way, all good now.  Thanks for the quick follow-up!

Sincerely,

Dolan Antenucci


On Fri, Sep 9, 2011 at 3:21 PM, Jeffrey Naisbitt <jn...@yahoo-inc.com>wrote:

>  Which version of hadoop are you using?  Changing
> ‘mapreduce.jobtracker.staging.root.dir’ in mapred-site.xml should work.  Did
> you restart the jobtracker after making the change?
> With the latest version of hadoop on trunk, there is also
> ‘yarn.app.mapreduce.staging-dir’ in yarn-site.xml
>
> -Jeff
>
>
> On 9/9/11 1:38 PM, "Dolan Antenucci" <an...@gmail.com> wrote:
>
> Is there a way to change the staging directory in the config files?  Mine
> currently goes to "/<tmp-folder>/staging".
>
> I read someone else set mapreduce.jobtracker.staging.root.dir, but I have
> had no luck with this (or slightly different versions of it) in my config
> files.  I'm guessing this other user set it in their code?   How would I do
> that if I run a command like, "hadoop jar
> /usr/lib/hadoop/hadoop-*-examples.jar pi 2 100000"?  My preference though
> would be to overwrite this setting in the config files so that every user
> wouldn't have to specify it (i.e. assuming I don't let them use the temp
> folder).
>
> Thanks
>
> -Dolan Antenucci
>
>
>

Re: Config setting for staging.dir?

[Jeffrey Naisbitt <jn...@yahoo-inc.com>]

Which version of hadoop are you using?  Changing ‘mapreduce.jobtracker.staging.root.dir’ in mapred-site.xml should work.  Did you restart the jobtracker after making the change?
With the latest version of hadoop on trunk, there is also ‘yarn.app.mapreduce.staging-dir’ in yarn-site.xml

-Jeff

On 9/9/11 1:38 PM, "Dolan Antenucci" <an...@gmail.com> wrote:

Is there a way to change the staging directory in the config files?  Mine currently goes to "/<tmp-folder>/staging".

I read someone else set mapreduce.jobtracker.staging.root.dir, but I have had no luck with this (or slightly different versions of it) in my config files.  I'm guessing this other user set it in their code?   How would I do that if I run a command like, "hadoop jar /usr/lib/hadoop/hadoop-*-examples.jar pi 2 100000"?  My preference though would be to overwrite this setting in the config files so that every user wouldn't have to specify it (i.e. assuming I don't let them use the temp folder).

Thanks

-Dolan Antenucci