You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by stephen mulcahy <st...@deri.org> on 2009/06/10 14:56:07 UTC

maybe a bug in hadoop?

Hi,

I've been running some tests with hadoop in pseudo-distributed mode. My 
config includes the following in conf/hdfs-site.xml

    <property>
      <name>dfs.data.dir</name>
      <value>/hdfs/disk1, /hdfs/disk2, /hdfs/disk3, /hdfs/disk4</value>
    </property>

When I started running hadoop I was disappointed to see that only 
/hdfs/disk1 was created. It was only later I noticed that HADOOP_HOME 
now contained the following new directories

HADOOP_HOME/ /hdfs/disk2
HADOOP_HOME/ /hdfs/disk3
HADOOP_HOME/ /hdfs/disk4

I guess any leading spaces should be stripped out of the data dir names? 
Or maybe there is a reason for this behaviour. I thought I should 
mention it just in case.

Thanks,

-stephen

-- 
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Re: maybe a bug in hadoop?

Posted by Michele Catasta <mi...@deri.org>.
Hi,

> This seems to already be addressed by
> https://issues.apache.org/jira/browse/HADOOP-2366

Thanks for the pointer, I just submitted a trivial patch for it.


Best,
  Michele

Re: maybe a bug in hadoop?

Posted by stephen mulcahy <st...@deri.org>.
Hi,

This seems to already be addressed by 
https://issues.apache.org/jira/browse/HADOOP-2366

-stephen

Brian Bockelman wrote:
> Hey Stephen,
> 
> I've hit this "bug" before (rather, our admins did...).
> 
> I would be happy to see you file it  - after checking for duplicates - 
> so I no longer have to warn people about it.
> 
> Brian
> 
> On Jun 10, 2009, at 6:29 AM, stephen mulcahy wrote:
> 
>> Tim Wintle wrote:
>>> I thought I'd double check...
>>> $touch \ file\ with\ leading\ space
>>> $touch file\ without\ leading\ space
>>> $ ls
>>> file with leading space
>>> file without leading space
>>
>> Oh sure, Linux will certainly allow you to create files/dirs with such 
>> names (and better .. hows about
>>
>> touch \.\.\.
>>
>> or similar (which used to be a favourite of guys who had managed to 
>> exploit bad permissions on ftp sites to upload material of dubious 
>> copyright). But is that the intention in the Hadoop config or is it a 
>> usability oversight? Hence my mail here rather than raising a bug 
>> immediately.
>>
>>> ... so stripping it out could mean you couldn't enter some valid
>>> directory names (but who has folders starting with a space?)
>>
>> Well ... I do now ;)
>>
>> -stephen
>>
>> -- 
>> Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
>> NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
>> http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com
> 


-- 
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Re: maybe a bug in hadoop?

Posted by Brian Bockelman <bb...@cse.unl.edu>.
Hey Stephen,

I've hit this "bug" before (rather, our admins did...).

I would be happy to see you file it  - after checking for duplicates -  
so I no longer have to warn people about it.

Brian

On Jun 10, 2009, at 6:29 AM, stephen mulcahy wrote:

> Tim Wintle wrote:
>> I thought I'd double check...
>> $touch \ file\ with\ leading\ space
>> $touch file\ without\ leading\ space
>> $ ls
>> file with leading space
>> file without leading space
>
> Oh sure, Linux will certainly allow you to create files/dirs with  
> such names (and better .. hows about
>
> touch \.\.\.
>
> or similar (which used to be a favourite of guys who had managed to  
> exploit bad permissions on ftp sites to upload material of dubious  
> copyright). But is that the intention in the Hadoop config or is it  
> a usability oversight? Hence my mail here rather than raising a bug  
> immediately.
>
>> ... so stripping it out could mean you couldn't enter some valid
>> directory names (but who has folders starting with a space?)
>
> Well ... I do now ;)
>
> -stephen
>
> -- 
> Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
> NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
> http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Re: maybe a bug in hadoop?

Posted by stephen mulcahy <st...@deri.org>.
Tim Wintle wrote:
> I thought I'd double check...
> 
> $touch \ file\ with\ leading\ space
> $touch file\ without\ leading\ space
> $ ls
>  file with leading space
> file without leading space

Oh sure, Linux will certainly allow you to create files/dirs with such 
names (and better .. hows about

touch \.\.\.

or similar (which used to be a favourite of guys who had managed to 
exploit bad permissions on ftp sites to upload material of dubious 
copyright). But is that the intention in the Hadoop config or is it a 
usability oversight? Hence my mail here rather than raising a bug 
immediately.

> ... so stripping it out could mean you couldn't enter some valid
> directory names (but who has folders starting with a space?)

Well ... I do now ;)

-stephen

-- 
Stephen Mulcahy, DI2, Digital Enterprise Research Institute,
NUI Galway, IDA Business Park, Lower Dangan, Galway, Ireland
http://di2.deri.ie    http://webstar.deri.ie    http://sindice.com

Re: maybe a bug in hadoop?

Posted by Tim Wintle <ti...@teamrubber.com>.
On Wed, 2009-06-10 at 13:56 +0100, stephen mulcahy wrote:
> Hi,
> 
> I've been running some tests with hadoop in pseudo-distributed mode. My 
> config includes the following in conf/hdfs-site.xml
> 
>     <property>
>       <name>dfs.data.dir</name>
>       <value>/hdfs/disk1, /hdfs/disk2, /hdfs/disk3, /hdfs/disk4</value>
>     </property>
> 
> When I started running hadoop I was disappointed to see that only 
> /hdfs/disk1 was created. It was only later I noticed that HADOOP_HOME 
> now contained the following new directories
> 
> HADOOP_HOME/ /hdfs/disk2
> HADOOP_HOME/ /hdfs/disk3
> HADOOP_HOME/ /hdfs/disk4
> 
> I guess any leading spaces should be stripped out of the data dir names? 
> Or maybe there is a reason for this behaviour. I thought I should 
> mention it just in case.

I thought I'd double check...

$touch \ file\ with\ leading\ space
$touch file\ without\ leading\ space
$ ls
 file with leading space
file without leading space

... so stripping it out could mean you couldn't enter some valid
directory names (but who has folders starting with a space?)

I'm sure my input wasn't very useful, but just a comment.

Tim Wintle