Non existent config file to 'fs -conf'

[Ashwin Shankar <as...@yahoo-inc.com>]

Hi,
I'm working on  HADOOP-9582<https://issues.apache.org/jira/browse/HADOOP-9582>  and I have a question about the current implementation in hadoop-common.
Here is a brief background about the bug : Basically if I give a non-exitent file to "hadoop fs  –conf NONEXISTENT_FILE",
the current implementation never complains.
But looking at the code(Configuration.loadResources()) it seems that in-fact we check if input file exists and we throw an exception if the 'quiet' flag is false.
Problem is the 'quiet' flag is always true.
Can somebody explain the rationale behind this behavior ? Would we break any use-case if we complain when non-exitent file is given as input?

Why we want this fixed :  say the user makes a typo and gives the wrong path ,the code is just going to ignore this,not complain
and use the default conf files(if the env variables are set). This would confuse the user when he finds that the configs are different from what he gave as input(typo) .
Thoughts?

Thanks,
Ashwin

Re: Non existent config file to 'fs -conf'

[Ashwin Shankar <as...@yahoo-inc.com>]

Hi Harsh,
Yes that would work. Thanks!
I'll post the updated patch soon on JIRA.

Thanks,
Ashwin

On 5/23/13 11:28 PM, "Harsh J" <ha...@cloudera.com> wrote:

>The "quiet" behavior sorta goes all the way back to the very first
>import of Nutch into Apache Incubator:
>http://svn.apache.org/viewvc?view=revision&revision=155829 and seems
>to deal with being relaxed about not finding added resources other
>than required defaults. The behavior has almost been the same for over
>8 years now :-)
>
>The quiet flag is code-settable, but the output it would produce is
>pretty verbose. I suppose we can turn it on at the FsShell level,
>while also making those "parsingŠ", etc. INFO level logs into
>checked-for DEBUG level logs. Would that suffice?
>
>On Fri, May 24, 2013 at 12:52 AM, Ashwin Shankar <as...@yahoo-inc.com>
>wrote:
>> Hi,
>> I'm working on  
>>HADOOP-9582<https://issues.apache.org/jira/browse/HADOOP-9582>  and I
>>have a question about the current implementation in hadoop-common.
>> Here is a brief background about the bug : Basically if I give a
>>non-exitent file to "hadoop fs  ­conf NONEXISTENT_FILE",
>> the current implementation never complains.
>> But looking at the code(Configuration.loadResources()) it seems that
>>in-fact we check if input file exists and we throw an exception if the
>>'quiet' flag is false.
>> Problem is the 'quiet' flag is always true.
>> Can somebody explain the rationale behind this behavior ? Would we
>>break any use-case if we complain when non-exitent file is given as
>>input?
>>
>> Why we want this fixed :  say the user makes a typo and gives the wrong
>>path ,the code is just going to ignore this,not complain
>> and use the default conf files(if the env variables are set). This
>>would confuse the user when he finds that the configs are different from
>>what he gave as input(typo) .
>> Thoughts?
>>
>> Thanks,
>> Ashwin
>>
>
>
>
>-- 
>Harsh J

Re: Non existent config file to 'fs -conf'

[Harsh J <ha...@cloudera.com>]

The "quiet" behavior sorta goes all the way back to the very first
import of Nutch into Apache Incubator:
http://svn.apache.org/viewvc?view=revision&revision=155829 and seems
to deal with being relaxed about not finding added resources other
than required defaults. The behavior has almost been the same for over
8 years now :-)

The quiet flag is code-settable, but the output it would produce is
pretty verbose. I suppose we can turn it on at the FsShell level,
while also making those "parsing…", etc. INFO level logs into
checked-for DEBUG level logs. Would that suffice?

On Fri, May 24, 2013 at 12:52 AM, Ashwin Shankar <as...@yahoo-inc.com> wrote:
> Hi,
> I'm working on  HADOOP-9582<https://issues.apache.org/jira/browse/HADOOP-9582>  and I have a question about the current implementation in hadoop-common.
> Here is a brief background about the bug : Basically if I give a non-exitent file to "hadoop fs  –conf NONEXISTENT_FILE",
> the current implementation never complains.
> But looking at the code(Configuration.loadResources()) it seems that in-fact we check if input file exists and we throw an exception if the 'quiet' flag is false.
> Problem is the 'quiet' flag is always true.
> Can somebody explain the rationale behind this behavior ? Would we break any use-case if we complain when non-exitent file is given as input?
>
> Why we want this fixed :  say the user makes a typo and gives the wrong path ,the code is just going to ignore this,not complain
> and use the default conf files(if the env variables are set). This would confuse the user when he finds that the configs are different from what he gave as input(typo) .
> Thoughts?
>
> Thanks,
> Ashwin
>



-- 
Harsh J