You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Mohit Anchlia <mo...@gmail.com> on 2012/02/21 23:37:19 UTC
Using hdfs:// in FileInputFormat.setInputPaths(conf, new Path(args[0]));
Can I do something like this?
FileInputFormat.*setInputPaths*(conf, *new* Path("hdfs://host:port/file"));
Examples I am seeing are all using the dir/file. I haven't seen them using
hdfs?
If I am running mapred from remote machine then how do I configure it?
Re: Using hdfs:// in FileInputFormat.setInputPaths(conf, new Path(args[0]));
Posted by Harsh J <ha...@cloudera.com>.
Hi Mohit,
Usually you'd want to just do new Path("/regular/path"); in code, to
have the flexibility to make it work across different FileSystems.
This will still work for remote clusters, if your fs.default.name
(Which is understood by hadoop as the 'default' filesystem of choice)
is a hdfs:// URI then the paths would automatically be looked up on
the HDFS host:port taken from the configuration.
On Wed, Feb 22, 2012 at 4:07 AM, Mohit Anchlia <mo...@gmail.com> wrote:
> Can I do something like this?
>
>
> FileInputFormat.setInputPaths(conf,
>
> new Path("hdfs://host:port/file"));
>
> Examples I am seeing are all using the dir/file. I haven't seen them using
> hdfs?
>
> If I am running mapred from remote machine then how do I configure it?
--
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about