You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Visioner Sadak <vi...@gmail.com> on 2012/09/19 15:48:33 UTC
WEBHDFS vs HDFSPROXY which one is faster
Hello experts could you judge whether webhdfs is fast or hdfsproxy is fast,
is hdfs proxy slower coz it uses https only or can we use http also in
hdfsproxy, its also mentioned in this below link,could you guys throw more
light on advantages and disadvantages of both techniques......
http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
in
disadvantages section,
Thanks....
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
thanks harsh will try it out.......
On Thu, Sep 20, 2012 at 9:09 AM, Harsh J <ha...@cloudera.com> wrote:
> Visioner,
>
> I haven't tested it myself but the URI should instead just be the
> following, since we're not gonna be using a REST API within HAR FS:
>
> har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
>
> On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > can we read HAR files frm webhdfs in that case will my url be like
> >
> > har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
> >
> > but will my browser be able to understand this
> >
> > On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <
> visioner.sadak@gmail.com>
> > wrote:
> >>
> >> Thanks harsh.....
> >>
> >>
> >> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> WebHDFS is faster/scalable as it lets the client directly access a DN
> >>> to read off of, and hence requires that the client be able to access
> >>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> >>> 'called') is instead useful as a gateway service if clients are not
> >>> allowed to access all DN nodes of a given cluster.
> >>>
> >>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> >>> <vi...@gmail.com> wrote:
> >>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> >>> > fast,
> >>> > is hdfs proxy slower coz it uses https only or can we use http also
> in
> >>> > hdfsproxy, its also mentioned in this below link,could you guys throw
> >>> > more
> >>> > light on advantages and disadvantages of both techniques......
> >>> >
> >>> >
> >>> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> >>> > in disadvantages section,
> >>> >
> >>> >
> >>> >
> >>> > Thanks....
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
thanks harsh will try it out.......
On Thu, Sep 20, 2012 at 9:09 AM, Harsh J <ha...@cloudera.com> wrote:
> Visioner,
>
> I haven't tested it myself but the URI should instead just be the
> following, since we're not gonna be using a REST API within HAR FS:
>
> har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
>
> On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > can we read HAR files frm webhdfs in that case will my url be like
> >
> > har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
> >
> > but will my browser be able to understand this
> >
> > On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <
> visioner.sadak@gmail.com>
> > wrote:
> >>
> >> Thanks harsh.....
> >>
> >>
> >> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> WebHDFS is faster/scalable as it lets the client directly access a DN
> >>> to read off of, and hence requires that the client be able to access
> >>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> >>> 'called') is instead useful as a gateway service if clients are not
> >>> allowed to access all DN nodes of a given cluster.
> >>>
> >>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> >>> <vi...@gmail.com> wrote:
> >>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> >>> > fast,
> >>> > is hdfs proxy slower coz it uses https only or can we use http also
> in
> >>> > hdfsproxy, its also mentioned in this below link,could you guys throw
> >>> > more
> >>> > light on advantages and disadvantages of both techniques......
> >>> >
> >>> >
> >>> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> >>> > in disadvantages section,
> >>> >
> >>> >
> >>> >
> >>> > Thanks....
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
thanks harsh will try it out.......
On Thu, Sep 20, 2012 at 9:09 AM, Harsh J <ha...@cloudera.com> wrote:
> Visioner,
>
> I haven't tested it myself but the URI should instead just be the
> following, since we're not gonna be using a REST API within HAR FS:
>
> har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
>
> On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > can we read HAR files frm webhdfs in that case will my url be like
> >
> > har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
> >
> > but will my browser be able to understand this
> >
> > On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <
> visioner.sadak@gmail.com>
> > wrote:
> >>
> >> Thanks harsh.....
> >>
> >>
> >> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> WebHDFS is faster/scalable as it lets the client directly access a DN
> >>> to read off of, and hence requires that the client be able to access
> >>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> >>> 'called') is instead useful as a gateway service if clients are not
> >>> allowed to access all DN nodes of a given cluster.
> >>>
> >>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> >>> <vi...@gmail.com> wrote:
> >>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> >>> > fast,
> >>> > is hdfs proxy slower coz it uses https only or can we use http also
> in
> >>> > hdfsproxy, its also mentioned in this below link,could you guys throw
> >>> > more
> >>> > light on advantages and disadvantages of both techniques......
> >>> >
> >>> >
> >>> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> >>> > in disadvantages section,
> >>> >
> >>> >
> >>> >
> >>> > Thanks....
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
thanks harsh will try it out.......
On Thu, Sep 20, 2012 at 9:09 AM, Harsh J <ha...@cloudera.com> wrote:
> Visioner,
>
> I haven't tested it myself but the URI should instead just be the
> following, since we're not gonna be using a REST API within HAR FS:
>
> har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
>
> On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > can we read HAR files frm webhdfs in that case will my url be like
> >
> > har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
> >
> > but will my browser be able to understand this
> >
> > On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <
> visioner.sadak@gmail.com>
> > wrote:
> >>
> >> Thanks harsh.....
> >>
> >>
> >> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> >>>
> >>> Hi,
> >>>
> >>> WebHDFS is faster/scalable as it lets the client directly access a DN
> >>> to read off of, and hence requires that the client be able to access
> >>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> >>> 'called') is instead useful as a gateway service if clients are not
> >>> allowed to access all DN nodes of a given cluster.
> >>>
> >>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> >>> <vi...@gmail.com> wrote:
> >>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> >>> > fast,
> >>> > is hdfs proxy slower coz it uses https only or can we use http also
> in
> >>> > hdfsproxy, its also mentioned in this below link,could you guys throw
> >>> > more
> >>> > light on advantages and disadvantages of both techniques......
> >>> >
> >>> >
> >>> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> >>> > in disadvantages section,
> >>> >
> >>> >
> >>> >
> >>> > Thanks....
> >>>
> >>>
> >>>
> >>> --
> >>> Harsh J
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Visioner,
I haven't tested it myself but the URI should instead just be the
following, since we're not gonna be using a REST API within HAR FS:
har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
<vi...@gmail.com> wrote:
> can we read HAR files frm webhdfs in that case will my url be like
>
> har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
>
> but will my browser be able to understand this
>
> On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>
> wrote:
>>
>> Thanks harsh.....
>>
>>
>> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi,
>>>
>>> WebHDFS is faster/scalable as it lets the client directly access a DN
>>> to read off of, and hence requires that the client be able to access
>>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>>> 'called') is instead useful as a gateway service if clients are not
>>> allowed to access all DN nodes of a given cluster.
>>>
>>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>>> <vi...@gmail.com> wrote:
>>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>>> > fast,
>>> > is hdfs proxy slower coz it uses https only or can we use http also in
>>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>>> > more
>>> > light on advantages and disadvantages of both techniques......
>>> >
>>> >
>>> > http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>>> > in disadvantages section,
>>> >
>>> >
>>> >
>>> > Thanks....
>>>
>>>
>>>
>>> --
>>> Harsh J
>>
>>
>
--
Harsh J
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Visioner,
I haven't tested it myself but the URI should instead just be the
following, since we're not gonna be using a REST API within HAR FS:
har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
<vi...@gmail.com> wrote:
> can we read HAR files frm webhdfs in that case will my url be like
>
> har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
>
> but will my browser be able to understand this
>
> On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>
> wrote:
>>
>> Thanks harsh.....
>>
>>
>> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi,
>>>
>>> WebHDFS is faster/scalable as it lets the client directly access a DN
>>> to read off of, and hence requires that the client be able to access
>>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>>> 'called') is instead useful as a gateway service if clients are not
>>> allowed to access all DN nodes of a given cluster.
>>>
>>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>>> <vi...@gmail.com> wrote:
>>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>>> > fast,
>>> > is hdfs proxy slower coz it uses https only or can we use http also in
>>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>>> > more
>>> > light on advantages and disadvantages of both techniques......
>>> >
>>> >
>>> > http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>>> > in disadvantages section,
>>> >
>>> >
>>> >
>>> > Thanks....
>>>
>>>
>>>
>>> --
>>> Harsh J
>>
>>
>
--
Harsh J
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Visioner,
I haven't tested it myself but the URI should instead just be the
following, since we're not gonna be using a REST API within HAR FS:
har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
<vi...@gmail.com> wrote:
> can we read HAR files frm webhdfs in that case will my url be like
>
> har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
>
> but will my browser be able to understand this
>
> On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>
> wrote:
>>
>> Thanks harsh.....
>>
>>
>> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi,
>>>
>>> WebHDFS is faster/scalable as it lets the client directly access a DN
>>> to read off of, and hence requires that the client be able to access
>>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>>> 'called') is instead useful as a gateway service if clients are not
>>> allowed to access all DN nodes of a given cluster.
>>>
>>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>>> <vi...@gmail.com> wrote:
>>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>>> > fast,
>>> > is hdfs proxy slower coz it uses https only or can we use http also in
>>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>>> > more
>>> > light on advantages and disadvantages of both techniques......
>>> >
>>> >
>>> > http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>>> > in disadvantages section,
>>> >
>>> >
>>> >
>>> > Thanks....
>>>
>>>
>>>
>>> --
>>> Harsh J
>>
>>
>
--
Harsh J
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Visioner,
I haven't tested it myself but the URI should instead just be the
following, since we're not gonna be using a REST API within HAR FS:
har://webhdfs-localhost:50070/HAR/1june2012.har/test.jpg
On Thu, Sep 20, 2012 at 2:43 AM, Visioner Sadak
<vi...@gmail.com> wrote:
> can we read HAR files frm webhdfs in that case will my url be like
>
> har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
>
> but will my browser be able to understand this
>
> On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>
> wrote:
>>
>> Thanks harsh.....
>>
>>
>> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>>>
>>> Hi,
>>>
>>> WebHDFS is faster/scalable as it lets the client directly access a DN
>>> to read off of, and hence requires that the client be able to access
>>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>>> 'called') is instead useful as a gateway service if clients are not
>>> allowed to access all DN nodes of a given cluster.
>>>
>>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>>> <vi...@gmail.com> wrote:
>>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>>> > fast,
>>> > is hdfs proxy slower coz it uses https only or can we use http also in
>>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>>> > more
>>> > light on advantages and disadvantages of both techniques......
>>> >
>>> >
>>> > http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>>> > in disadvantages section,
>>> >
>>> >
>>> >
>>> > Thanks....
>>>
>>>
>>>
>>> --
>>> Harsh J
>>
>>
>
--
Harsh J
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
can we read HAR files frm webhdfs in that case will my url be like
har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
but will my browser be able to understand this
On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks harsh.....
>
>
> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> WebHDFS is faster/scalable as it lets the client directly access a DN
>> to read off of, and hence requires that the client be able to access
>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>> 'called') is instead useful as a gateway service if clients are not
>> allowed to access all DN nodes of a given cluster.
>>
>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>> <vi...@gmail.com> wrote:
>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>> fast,
>> > is hdfs proxy slower coz it uses https only or can we use http also in
>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>> more
>> > light on advantages and disadvantages of both techniques......
>> >
>> >
>> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>> > in disadvantages section,
>> >
>> >
>> >
>> > Thanks....
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
can we read HAR files frm webhdfs in that case will my url be like
har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
but will my browser be able to understand this
On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks harsh.....
>
>
> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> WebHDFS is faster/scalable as it lets the client directly access a DN
>> to read off of, and hence requires that the client be able to access
>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>> 'called') is instead useful as a gateway service if clients are not
>> allowed to access all DN nodes of a given cluster.
>>
>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>> <vi...@gmail.com> wrote:
>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>> fast,
>> > is hdfs proxy slower coz it uses https only or can we use http also in
>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>> more
>> > light on advantages and disadvantages of both techniques......
>> >
>> >
>> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>> > in disadvantages section,
>> >
>> >
>> >
>> > Thanks....
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
can we read HAR files frm webhdfs in that case will my url be like
har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
but will my browser be able to understand this
On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks harsh.....
>
>
> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> WebHDFS is faster/scalable as it lets the client directly access a DN
>> to read off of, and hence requires that the client be able to access
>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>> 'called') is instead useful as a gateway service if clients are not
>> allowed to access all DN nodes of a given cluster.
>>
>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>> <vi...@gmail.com> wrote:
>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>> fast,
>> > is hdfs proxy slower coz it uses https only or can we use http also in
>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>> more
>> > light on advantages and disadvantages of both techniques......
>> >
>> >
>> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>> > in disadvantages section,
>> >
>> >
>> >
>> > Thanks....
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
can we read HAR files frm webhdfs in that case will my url be like
har://http-localhost:50070/webhdfs/v1/HAR/1june2012.har/test.jpg?op=OPEN
but will my browser be able to understand this
On Thu, Sep 20, 2012 at 2:38 AM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks harsh.....
>
>
> On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> WebHDFS is faster/scalable as it lets the client directly access a DN
>> to read off of, and hence requires that the client be able to access
>> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
>> 'called') is instead useful as a gateway service if clients are not
>> allowed to access all DN nodes of a given cluster.
>>
>> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
>> <vi...@gmail.com> wrote:
>> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
>> fast,
>> > is hdfs proxy slower coz it uses https only or can we use http also in
>> > hdfsproxy, its also mentioned in this below link,could you guys throw
>> more
>> > light on advantages and disadvantages of both techniques......
>> >
>> >
>> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
>> > in disadvantages section,
>> >
>> >
>> >
>> > Thanks....
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks harsh.....
On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> WebHDFS is faster/scalable as it lets the client directly access a DN
> to read off of, and hence requires that the client be able to access
> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> 'called') is instead useful as a gateway service if clients are not
> allowed to access all DN nodes of a given cluster.
>
> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> fast,
> > is hdfs proxy slower coz it uses https only or can we use http also in
> > hdfsproxy, its also mentioned in this below link,could you guys throw
> more
> > light on advantages and disadvantages of both techniques......
> >
> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> > in disadvantages section,
> >
> >
> >
> > Thanks....
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks harsh.....
On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> WebHDFS is faster/scalable as it lets the client directly access a DN
> to read off of, and hence requires that the client be able to access
> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> 'called') is instead useful as a gateway service if clients are not
> allowed to access all DN nodes of a given cluster.
>
> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> fast,
> > is hdfs proxy slower coz it uses https only or can we use http also in
> > hdfsproxy, its also mentioned in this below link,could you guys throw
> more
> > light on advantages and disadvantages of both techniques......
> >
> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> > in disadvantages section,
> >
> >
> >
> > Thanks....
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks harsh.....
On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> WebHDFS is faster/scalable as it lets the client directly access a DN
> to read off of, and hence requires that the client be able to access
> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> 'called') is instead useful as a gateway service if clients are not
> allowed to access all DN nodes of a given cluster.
>
> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> fast,
> > is hdfs proxy slower coz it uses https only or can we use http also in
> > hdfsproxy, its also mentioned in this below link,could you guys throw
> more
> > light on advantages and disadvantages of both techniques......
> >
> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> > in disadvantages section,
> >
> >
> >
> > Thanks....
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks harsh.....
On Wed, Sep 19, 2012 at 8:41 PM, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> WebHDFS is faster/scalable as it lets the client directly access a DN
> to read off of, and hence requires that the client be able to access
> all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
> 'called') is instead useful as a gateway service if clients are not
> allowed to access all DN nodes of a given cluster.
>
> On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
> <vi...@gmail.com> wrote:
> > Hello experts could you judge whether webhdfs is fast or hdfsproxy is
> fast,
> > is hdfs proxy slower coz it uses https only or can we use http also in
> > hdfsproxy, its also mentioned in this below link,could you guys throw
> more
> > light on advantages and disadvantages of both techniques......
> >
> >
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> > in disadvantages section,
> >
> >
> >
> > Thanks....
>
>
>
> --
> Harsh J
>
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Hi,
WebHDFS is faster/scalable as it lets the client directly access a DN
to read off of, and hence requires that the client be able to access
all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
'called') is instead useful as a gateway service if clients are not
allowed to access all DN nodes of a given cluster.
On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
<vi...@gmail.com> wrote:
> Hello experts could you judge whether webhdfs is fast or hdfsproxy is fast,
> is hdfs proxy slower coz it uses https only or can we use http also in
> hdfsproxy, its also mentioned in this below link,could you guys throw more
> light on advantages and disadvantages of both techniques......
>
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> in disadvantages section,
>
>
>
> Thanks....
--
Harsh J
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Hi,
WebHDFS is faster/scalable as it lets the client directly access a DN
to read off of, and hence requires that the client be able to access
all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
'called') is instead useful as a gateway service if clients are not
allowed to access all DN nodes of a given cluster.
On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
<vi...@gmail.com> wrote:
> Hello experts could you judge whether webhdfs is fast or hdfsproxy is fast,
> is hdfs proxy slower coz it uses https only or can we use http also in
> hdfsproxy, its also mentioned in this below link,could you guys throw more
> light on advantages and disadvantages of both techniques......
>
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> in disadvantages section,
>
>
>
> Thanks....
--
Harsh J
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Hi,
WebHDFS is faster/scalable as it lets the client directly access a DN
to read off of, and hence requires that the client be able to access
all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
'called') is instead useful as a gateway service if clients are not
allowed to access all DN nodes of a given cluster.
On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
<vi...@gmail.com> wrote:
> Hello experts could you judge whether webhdfs is fast or hdfsproxy is fast,
> is hdfs proxy slower coz it uses https only or can we use http also in
> hdfsproxy, its also mentioned in this below link,could you guys throw more
> light on advantages and disadvantages of both techniques......
>
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> in disadvantages section,
>
>
>
> Thanks....
--
Harsh J
Re: WEBHDFS vs HDFSPROXY which one is faster
Posted by Harsh J <ha...@cloudera.com>.
Hi,
WebHDFS is faster/scalable as it lets the client directly access a DN
to read off of, and hence requires that the client be able to access
all nodes of the HDFS cluster. The HttpFs (what hdfsproxy is now
'called') is instead useful as a gateway service if clients are not
allowed to access all DN nodes of a given cluster.
On Wed, Sep 19, 2012 at 7:18 PM, Visioner Sadak
<vi...@gmail.com> wrote:
> Hello experts could you judge whether webhdfs is fast or hdfsproxy is fast,
> is hdfs proxy slower coz it uses https only or can we use http also in
> hdfsproxy, its also mentioned in this below link,could you guys throw more
> light on advantages and disadvantages of both techniques......
>
> http://hadoop.apache.org/docs/hdfs/r0.21.0/hdfsproxy.html#Tomcat-based+Installation+and+Configuration
> in disadvantages section,
>
>
>
> Thanks....
--
Harsh J