You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Ashish Kumar9 <as...@in.ibm.com> on 2015/09/21 16:39:07 UTC
Hetergeneous Hadoop Cluster
Hi :
Has anyone tried a heterogeneous hadoop cluster with management nodes and
data nodes running on multiple linux distros. and on multiple h/w
architecture .
If so , what is the performance of such cluster .
Thanks
Ashish
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
I'm basically referring to federating multiple namenodes (connecting two
different hdfs instances under a single namespace so data can be
distributed across them). Here's the documentation for Hadoop 2.6.0 [1]
[1]
https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/Federation.html
On Fri, Sep 25, 2015 at 12:42 AM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> This is interesting . Can you share any blog/document that talks
> multi-volume HDFS instances .
>
> Thanks and Regards,
> Ashish Kumar
>
>
> From: Corey Nolet <cj...@gmail.com>
> To: user@hadoop.apache.org
> Date: 09/24/2015 10:40 PM
> Subject: Re: Hetergeneous Hadoop Cluster
> ------------------------------
>
>
>
> If the hardware is drastically different, I would think a multi-volume
> HDFS instance would be a good idea (put like-hardware in the same volumes).
>
> On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <*tgkprog@gmail.com*
> <tg...@gmail.com>> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <*ashishk4@in.ibm.com*
> <as...@in.ibm.com>> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
>
>
>
> --
> Regards
> Tushar Kapila
>
>
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
I'm basically referring to federating multiple namenodes (connecting two
different hdfs instances under a single namespace so data can be
distributed across them). Here's the documentation for Hadoop 2.6.0 [1]
[1]
https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/Federation.html
On Fri, Sep 25, 2015 at 12:42 AM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> This is interesting . Can you share any blog/document that talks
> multi-volume HDFS instances .
>
> Thanks and Regards,
> Ashish Kumar
>
>
> From: Corey Nolet <cj...@gmail.com>
> To: user@hadoop.apache.org
> Date: 09/24/2015 10:40 PM
> Subject: Re: Hetergeneous Hadoop Cluster
> ------------------------------
>
>
>
> If the hardware is drastically different, I would think a multi-volume
> HDFS instance would be a good idea (put like-hardware in the same volumes).
>
> On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <*tgkprog@gmail.com*
> <tg...@gmail.com>> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <*ashishk4@in.ibm.com*
> <as...@in.ibm.com>> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
>
>
>
> --
> Regards
> Tushar Kapila
>
>
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
I'm basically referring to federating multiple namenodes (connecting two
different hdfs instances under a single namespace so data can be
distributed across them). Here's the documentation for Hadoop 2.6.0 [1]
[1]
https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/Federation.html
On Fri, Sep 25, 2015 at 12:42 AM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> This is interesting . Can you share any blog/document that talks
> multi-volume HDFS instances .
>
> Thanks and Regards,
> Ashish Kumar
>
>
> From: Corey Nolet <cj...@gmail.com>
> To: user@hadoop.apache.org
> Date: 09/24/2015 10:40 PM
> Subject: Re: Hetergeneous Hadoop Cluster
> ------------------------------
>
>
>
> If the hardware is drastically different, I would think a multi-volume
> HDFS instance would be a good idea (put like-hardware in the same volumes).
>
> On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <*tgkprog@gmail.com*
> <tg...@gmail.com>> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <*ashishk4@in.ibm.com*
> <as...@in.ibm.com>> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
>
>
>
> --
> Regards
> Tushar Kapila
>
>
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
I'm basically referring to federating multiple namenodes (connecting two
different hdfs instances under a single namespace so data can be
distributed across them). Here's the documentation for Hadoop 2.6.0 [1]
[1]
https://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/Federation.html
On Fri, Sep 25, 2015 at 12:42 AM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> This is interesting . Can you share any blog/document that talks
> multi-volume HDFS instances .
>
> Thanks and Regards,
> Ashish Kumar
>
>
> From: Corey Nolet <cj...@gmail.com>
> To: user@hadoop.apache.org
> Date: 09/24/2015 10:40 PM
> Subject: Re: Hetergeneous Hadoop Cluster
> ------------------------------
>
>
>
> If the hardware is drastically different, I would think a multi-volume
> HDFS instance would be a good idea (put like-hardware in the same volumes).
>
> On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <*tgkprog@gmail.com*
> <tg...@gmail.com>> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <*ashishk4@in.ibm.com*
> <as...@in.ibm.com>> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
>
>
>
> --
> Regards
> Tushar Kapila
>
>
Re: Hetergeneous Hadoop Cluster
Posted by Ashish Kumar9 <as...@in.ibm.com>.
This is interesting . Can you share any blog/document that talks
multi-volume HDFS instances .
Thanks and Regards,
Ashish Kumar
From: Corey Nolet <cj...@gmail.com>
To: user@hadoop.apache.org
Date: 09/24/2015 10:40 PM
Subject: Re: Hetergeneous Hadoop Cluster
If the hardware is drastically different, I would think a multi-volume
HDFS instance would be a good idea (put like-hardware in the same
volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config
for the work it is doing. Cluster performance would not suffer just
because it is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
wrote:
Hi :
Has anyone tried a heterogeneous hadoop cluster with management nodes and
data nodes running on multiple linux distros. and on multiple h/w
architecture .
If so , what is the performance of such cluster .
Thanks
Ashish
--
Regards
Tushar Kapila
Re: Hetergeneous Hadoop Cluster
Posted by Ashish Kumar9 <as...@in.ibm.com>.
This is interesting . Can you share any blog/document that talks
multi-volume HDFS instances .
Thanks and Regards,
Ashish Kumar
From: Corey Nolet <cj...@gmail.com>
To: user@hadoop.apache.org
Date: 09/24/2015 10:40 PM
Subject: Re: Hetergeneous Hadoop Cluster
If the hardware is drastically different, I would think a multi-volume
HDFS instance would be a good idea (put like-hardware in the same
volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config
for the work it is doing. Cluster performance would not suffer just
because it is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
wrote:
Hi :
Has anyone tried a heterogeneous hadoop cluster with management nodes and
data nodes running on multiple linux distros. and on multiple h/w
architecture .
If so , what is the performance of such cluster .
Thanks
Ashish
--
Regards
Tushar Kapila
Re: Hetergeneous Hadoop Cluster
Posted by Ashish Kumar9 <as...@in.ibm.com>.
This is interesting . Can you share any blog/document that talks
multi-volume HDFS instances .
Thanks and Regards,
Ashish Kumar
From: Corey Nolet <cj...@gmail.com>
To: user@hadoop.apache.org
Date: 09/24/2015 10:40 PM
Subject: Re: Hetergeneous Hadoop Cluster
If the hardware is drastically different, I would think a multi-volume
HDFS instance would be a good idea (put like-hardware in the same
volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config
for the work it is doing. Cluster performance would not suffer just
because it is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
wrote:
Hi :
Has anyone tried a heterogeneous hadoop cluster with management nodes and
data nodes running on multiple linux distros. and on multiple h/w
architecture .
If so , what is the performance of such cluster .
Thanks
Ashish
--
Regards
Tushar Kapila
Re: Hetergeneous Hadoop Cluster
Posted by Ashish Kumar9 <as...@in.ibm.com>.
This is interesting . Can you share any blog/document that talks
multi-volume HDFS instances .
Thanks and Regards,
Ashish Kumar
From: Corey Nolet <cj...@gmail.com>
To: user@hadoop.apache.org
Date: 09/24/2015 10:40 PM
Subject: Re: Hetergeneous Hadoop Cluster
If the hardware is drastically different, I would think a multi-volume
HDFS instance would be a good idea (put like-hardware in the same
volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config
for the work it is doing. Cluster performance would not suffer just
because it is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
wrote:
Hi :
Has anyone tried a heterogeneous hadoop cluster with management nodes and
data nodes running on multiple linux distros. and on multiple h/w
architecture .
If so , what is the performance of such cluster .
Thanks
Ashish
--
Regards
Tushar Kapila
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
If the hardware is drastically different, I would think a multi-volume HDFS
instance would be a good idea (put like-hardware in the same volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
> wrote:
>
>> Hi :
>>
>> Has anyone tried a heterogeneous hadoop cluster with management nodes and
>> data nodes running on multiple linux distros. and on multiple h/w
>> architecture .
>>
>> If so , what is the performance of such cluster .
>>
>> Thanks
>> Ashish
>
>
>
>
> --
> Regards
> Tushar Kapila
>
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
If the hardware is drastically different, I would think a multi-volume HDFS
instance would be a good idea (put like-hardware in the same volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
> wrote:
>
>> Hi :
>>
>> Has anyone tried a heterogeneous hadoop cluster with management nodes and
>> data nodes running on multiple linux distros. and on multiple h/w
>> architecture .
>>
>> If so , what is the performance of such cluster .
>>
>> Thanks
>> Ashish
>
>
>
>
> --
> Regards
> Tushar Kapila
>
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
If the hardware is drastically different, I would think a multi-volume HDFS
instance would be a good idea (put like-hardware in the same volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
> wrote:
>
>> Hi :
>>
>> Has anyone tried a heterogeneous hadoop cluster with management nodes and
>> data nodes running on multiple linux distros. and on multiple h/w
>> architecture .
>>
>> If so , what is the performance of such cluster .
>>
>> Thanks
>> Ashish
>
>
>
>
> --
> Regards
> Tushar Kapila
>
Re: Hetergeneous Hadoop Cluster
Posted by Corey Nolet <cj...@gmail.com>.
If the hardware is drastically different, I would think a multi-volume HDFS
instance would be a good idea (put like-hardware in the same volumes).
On Mon, Sep 21, 2015 at 3:29 PM, Tushar Kapila <tg...@gmail.com> wrote:
> Would only matter if OS specific communication was being used between
> nodes. I assume they do not do that.
> If that is true -> It would depend on the network and each nodes config
> for the work it is doing. Cluster performance would not suffer just because
> it is heterogeneous.
>
> On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com>
> wrote:
>
>> Hi :
>>
>> Has anyone tried a heterogeneous hadoop cluster with management nodes and
>> data nodes running on multiple linux distros. and on multiple h/w
>> architecture .
>>
>> If so , what is the performance of such cluster .
>>
>> Thanks
>> Ashish
>
>
>
>
> --
> Regards
> Tushar Kapila
>
Re: Hetergeneous Hadoop Cluster
Posted by Tushar Kapila <tg...@gmail.com>.
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config for
the work it is doing. Cluster performance would not suffer just because it
is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
--
Regards
Tushar Kapila
Re: Hetergeneous Hadoop Cluster
Posted by Tushar Kapila <tg...@gmail.com>.
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config for
the work it is doing. Cluster performance would not suffer just because it
is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
--
Regards
Tushar Kapila
Re: Hetergeneous Hadoop Cluster
Posted by Tushar Kapila <tg...@gmail.com>.
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config for
the work it is doing. Cluster performance would not suffer just because it
is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
--
Regards
Tushar Kapila
Re: Hetergeneous Hadoop Cluster
Posted by Tushar Kapila <tg...@gmail.com>.
Would only matter if OS specific communication was being used between
nodes. I assume they do not do that.
If that is true -> It would depend on the network and each nodes config for
the work it is doing. Cluster performance would not suffer just because it
is heterogeneous.
On Mon, Sep 21, 2015 at 8:09 PM, Ashish Kumar9 <as...@in.ibm.com> wrote:
> Hi :
>
> Has anyone tried a heterogeneous hadoop cluster with management nodes and
> data nodes running on multiple linux distros. and on multiple h/w
> architecture .
>
> If so , what is the performance of such cluster .
>
> Thanks
> Ashish
--
Regards
Tushar Kapila