You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by Ferdy Galema <fe...@kalooga.com> on 2011/03/01 18:24:16 UTC

different hadoop.tmp.dir for master and slaves (CDH3B4)

The master only has a single disk (RAID for mirroring and striping), but 
the slaves are JBOD. Let's say I want to specify /raid/tmp for the 
master, but /disk1/tmp,/disk2/tmp,/disk3/tmp,/disk4/tmp for the slaves. 
How do I do this cleanly?

The fact that I need all 4 disks for temp-space is because the 
tasktrackers can spread the intermediate output on these disks.

Hadoop is version CDH3B4.

Re: different hadoop.tmp.dir for master and slaves (CDH3B4)

Posted by Harsh J <qw...@gmail.com>.

Hello,

On Tue, Mar 1, 2011 at 10:54 PM, Ferdy Galema <fe...@kalooga.com> wrote:
> The master only has a single disk (RAID for mirroring and striping), but the
> slaves are JBOD. Let's say I want to specify /raid/tmp for the master, but
> /disk1/tmp,/disk2/tmp,/disk3/tmp,/disk4/tmp for the slaves. How do I do this
> cleanly?

Use the `mapred.local.dir` property for this. Give it a
comma-separated list of paths to use as local directories (for
intermediate outputs, cached items, and Task local writes).

-- 
Harsh J
www.harshj.com