You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by Steve Loughran <st...@hortonworks.com> on 2017/05/12 09:51:55 UTC

understanding hadoop.tmp.dir

I've been looking for a location on HDFS for temp data as part of HADOOP-13786, which has had me looking at hadoop.tmp.dir


  1.  There's no actual java constant for it
  2.  We don't spell out that it is in the local FS only...there's a risk that it is picked up and used in the cluster path, which is wrong, especially if a config changes that temp dir
  3.  And as Milind & AW have noted in the past, it's a single dir, not a list of directories

Putting aside this confusion: we don't have any equivalent for the cluster fs, like "hdfs.tmp.dir"

what should I be using here? And if there isn't a good answer other than "use hadoop.tmp.dir and hope the permissions are consistent", maybe its time to add one —even if the default value for it is just hadoop.tmp.dir

-steve