fs.exists report wrong information

[Mike S <mi...@gmail.com>]

I need to check on my jar file arguments validity when the run method
is called by the ToolRunner

public int run(String[] args) throws Exception
{

Path filePath = new Path(args[3]);

FileSystem fs= path .getFileSystem(new Configuration());

if(!fs.exists(path ))
{
// Do things
}

}


The stang part is that the above code works file in my Eclipse local
run but when I move the code to the cluster, it works for the path in
hdfs but if the path (arg[3]) is on local file system like
/tmpFolder/myfile, then the above code report the file as not existing
where the file is there for sure. What I am doing wrong?

Re: fs.exists report wrong information

[Harsh J <ha...@cloudera.com>]

Hi Mike,

One nit: If you use "extends Configured implements Tool" then do not
do a "new Configuration();" anywhere. Instead just call the
"getConf()" method to get a pre-created config object.

When you do a path.getFileSystem(conf) and there's no scheme in the
path's URI, the default (fs.default.name) scheme is taken from your
configs (which may source it via core-site.xml).

Hence, a valid way of testing this for different filesystems is to
make sure to provide the FS Scheme prefix:

hadoop jar test.jar hdfs://localhost:8020/file
hadoop jar test.jar file:///tmp/file

On Thu, Sep 20, 2012 at 6:12 AM, Mike S <mi...@gmail.com> wrote:
> I need to check on my jar file arguments validity when the run method
> is called by the ToolRunner
>
> public int run(String[] args) throws Exception
> {
>
> Path filePath = new Path(args[3]);
>
> FileSystem fs= path .getFileSystem(new Configuration());
>
> if(!fs.exists(path ))
> {
> // Do things
> }
>
> }
>
>
> The stang part is that the above code works file in my Eclipse local
> run but when I move the code to the cluster, it works for the path in
> hdfs but if the path (arg[3]) is on local file system like
> /tmpFolder/myfile, then the above code report the file as not existing
> where the file is there for sure. What I am doing wrong?



-- 
Harsh J