You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by chaitanya krishna <ch...@gmail.com> on 2008/05/08 11:37:55 UTC
newbie how to get url paths of files in HDFS
Hi,
I want to get the "URL" paths of files that are stored in dfs. Is there
any way to get it?
Thank you
Re: newbie how to get url paths of files in HDFS
Posted by Ted Dunning <td...@veoh.com>.
That will be incredibly useful!
On 5/8/08 9:35 AM, "Doug Cutting" <cu...@apache.org> wrote:
> Ted Dunning wrote:
>> Take the fully qualified HDFS path that looks like this:
>>
>> hdfs://namenode-host-name:port/file-path
>>
>> And transform it into this:
>>
>> hdfs://namenode-host-name:web-interface-port/data/file-path
>>
>> The web-interface-port is 50070 by default. This will allow you to read HDFS
>> files via HTTP.
>
> Also, starting in release 0.18.0, Java programs can use "hdfs:" URLs.
> For example, one can create a URLClassLoader for a jar stored in HDFS.
>
> Doug
Re: newbie how to get url paths of files in HDFS
Posted by Doug Cutting <cu...@apache.org>.
Ted Dunning wrote:
> Take the fully qualified HDFS path that looks like this:
>
> hdfs://namenode-host-name:port/file-path
>
> And transform it into this:
>
> hdfs://namenode-host-name:web-interface-port/data/file-path
>
> The web-interface-port is 50070 by default. This will allow you to read HDFS files via HTTP.
Also, starting in release 0.18.0, Java programs can use "hdfs:" URLs.
For example, one can create a URLClassLoader for a jar stored in HDFS.
Doug
RE: newbie how to get url paths of files in HDFS
Posted by Ted Dunning <td...@veoh.com>.
Take the fully qualified HDFS path that looks like this:
hdfs://namenode-host-name:port/file-path
And transform it into this:
hdfs://namenode-host-name:web-interface-port/data/file-path
The web-interface-port is 50070 by default. This will allow you to read HDFS files via HTTP.
-----Original Message-----
From: Peeyush Bishnoi [mailto:peeyushb@yahoo-inc.com]
Sent: Thu 5/8/2008 5:04 AM
To: core-user@hadoop.apache.org
Subject: Re: newbie how to get url paths of files in HDFS
I apologize that I misunderstood your question , do you want to get URL
through API or through web interface.
Thanks ,
---
Peeyush
On Thu, 2008-05-08 at 16:04 +0530, Peeyush Bishnoi wrote:
> Hello Chaitanya ,
>
> using getInputpaths() you can do this . See here
>
> http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/JobConf.html#getInputPaths()
>
> ---
> Peeyush
>
> On Thu, 2008-05-08 at 15:07 +0530, chaitanya krishna wrote:
>
> > Hi,
> >
> > I want to get the "URL" paths of files that are stored in dfs. Is there
> > any way to get it?
> >
> >
> > Thank you
Re: newbie how to get url paths of files in HDFS
Posted by Peeyush Bishnoi <pe...@yahoo-inc.com>.
I apologize that I misunderstood your question , do you want to get URL
through API or through web interface.
Thanks ,
---
Peeyush
On Thu, 2008-05-08 at 16:04 +0530, Peeyush Bishnoi wrote:
> Hello Chaitanya ,
>
> using getInputpaths() you can do this . See here
>
> http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/JobConf.html#getInputPaths()
>
> ---
> Peeyush
>
> On Thu, 2008-05-08 at 15:07 +0530, chaitanya krishna wrote:
>
> > Hi,
> >
> > I want to get the "URL" paths of files that are stored in dfs. Is there
> > any way to get it?
> >
> >
> > Thank you
Re: newbie how to get url paths of files in HDFS
Posted by Peeyush Bishnoi <pe...@yahoo-inc.com>.
Hello Chaitanya ,
using getInputpaths() you can do this . See here
http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/JobConf.html#getInputPaths()
---
Peeyush
On Thu, 2008-05-08 at 15:07 +0530, chaitanya krishna wrote:
> Hi,
>
> I want to get the "URL" paths of files that are stored in dfs. Is there
> any way to get it?
>
>
> Thank you