You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Oleg Ruchovets <or...@gmail.com> on 2014/11/06 08:31:51 UTC
High Availability hadoop cluster.
Hello.
We are using hortonwork distribution and want to evaluate HA capabilities.
Can community please share the best practices and potential problems?
Is it required additional hardware?
Thanks
Oleg.
Re: High Availability hadoop cluster.
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Oleg,
why don’t you talk to me?
Put 1 ZK and 1 JN on NN boxes, third one on one of the data nodes.
HA wizard will automatically remove SNN from your cluster and put second NN in it’s place instead.
If something does not work as you expected, give me a call.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
The best way to explain - do it yourself.
On Nov 6, 2014, at 6:01, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation / configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
For 17 box cluster it's probably good to run 5 ZKs and 5 JNs
So, run 2 ZKs on 2 NNs
3 ZKs on 3 DNs
same for JNs
you can start additional ZKs and JNs after you are done with initial
Enabling HA in Ambari
On Thu, Nov 6, 2014 at 3:01 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
> and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>>
>>
>> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>>
>> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>>> hortonworks distributions.
>>> Sorry for dummy question - what is JN is?
>>> Can you please please point me on some manual wiki for installation /
>>> configuration.
>>>
>>> Thanks
>>> Oleg.
>>>
>>>
>>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <
>>> apivovarov@gmail.com> wrote:
>>>
>>>> 2 boxes for 2 NNs (better dedicated boxes)
>>>> min 3 JNs
>>>> min 3 ZKs
>>>>
>>>> JNs and ZKs can share boxes with other services
>>>>
>>>>
>>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello.
>>>>> We are using hortonwork distribution and want to evaluate HA
>>>>> capabilities.
>>>>> Can community please share the best practices and potential problems?
>>>>> Is it required additional hardware?
>>>>>
>>>>> Thanks
>>>>> Oleg.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
For 17 box cluster it's probably good to run 5 ZKs and 5 JNs
So, run 2 ZKs on 2 NNs
3 ZKs on 3 DNs
same for JNs
you can start additional ZKs and JNs after you are done with initial
Enabling HA in Ambari
On Thu, Nov 6, 2014 at 3:01 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
> and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>>
>>
>> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>>
>> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>>> hortonworks distributions.
>>> Sorry for dummy question - what is JN is?
>>> Can you please please point me on some manual wiki for installation /
>>> configuration.
>>>
>>> Thanks
>>> Oleg.
>>>
>>>
>>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <
>>> apivovarov@gmail.com> wrote:
>>>
>>>> 2 boxes for 2 NNs (better dedicated boxes)
>>>> min 3 JNs
>>>> min 3 ZKs
>>>>
>>>> JNs and ZKs can share boxes with other services
>>>>
>>>>
>>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello.
>>>>> We are using hortonwork distribution and want to evaluate HA
>>>>> capabilities.
>>>>> Can community please share the best practices and potential problems?
>>>>> Is it required additional hardware?
>>>>>
>>>>> Thanks
>>>>> Oleg.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Oleg,
why don’t you talk to me?
Put 1 ZK and 1 JN on NN boxes, third one on one of the data nodes.
HA wizard will automatically remove SNN from your cluster and put second NN in it’s place instead.
If something does not work as you expected, give me a call.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
The best way to explain - do it yourself.
On Nov 6, 2014, at 6:01, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation / configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: High Availability hadoop cluster.
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Oleg,
why don’t you talk to me?
Put 1 ZK and 1 JN on NN boxes, third one on one of the data nodes.
HA wizard will automatically remove SNN from your cluster and put second NN in it’s place instead.
If something does not work as you expected, give me a call.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
The best way to explain - do it yourself.
On Nov 6, 2014, at 6:01, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation / configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
For 17 box cluster it's probably good to run 5 ZKs and 5 JNs
So, run 2 ZKs on 2 NNs
3 ZKs on 3 DNs
same for JNs
you can start additional ZKs and JNs after you are done with initial
Enabling HA in Ambari
On Thu, Nov 6, 2014 at 3:01 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
> and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>>
>>
>> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>>
>> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>>> hortonworks distributions.
>>> Sorry for dummy question - what is JN is?
>>> Can you please please point me on some manual wiki for installation /
>>> configuration.
>>>
>>> Thanks
>>> Oleg.
>>>
>>>
>>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <
>>> apivovarov@gmail.com> wrote:
>>>
>>>> 2 boxes for 2 NNs (better dedicated boxes)
>>>> min 3 JNs
>>>> min 3 ZKs
>>>>
>>>> JNs and ZKs can share boxes with other services
>>>>
>>>>
>>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello.
>>>>> We are using hortonwork distribution and want to evaluate HA
>>>>> capabilities.
>>>>> Can community please share the best practices and potential problems?
>>>>> Is it required additional hardware?
>>>>>
>>>>> Thanks
>>>>> Oleg.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
For 17 box cluster it's probably good to run 5 ZKs and 5 JNs
So, run 2 ZKs on 2 NNs
3 ZKs on 3 DNs
same for JNs
you can start additional ZKs and JNs after you are done with initial
Enabling HA in Ambari
On Thu, Nov 6, 2014 at 3:01 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
> and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>>
>>
>> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>>
>> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>>> hortonworks distributions.
>>> Sorry for dummy question - what is JN is?
>>> Can you please please point me on some manual wiki for installation /
>>> configuration.
>>>
>>> Thanks
>>> Oleg.
>>>
>>>
>>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <
>>> apivovarov@gmail.com> wrote:
>>>
>>>> 2 boxes for 2 NNs (better dedicated boxes)
>>>> min 3 JNs
>>>> min 3 ZKs
>>>>
>>>> JNs and ZKs can share boxes with other services
>>>>
>>>>
>>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello.
>>>>> We are using hortonwork distribution and want to evaluate HA
>>>>> capabilities.
>>>>> Can community please share the best practices and potential problems?
>>>>> Is it required additional hardware?
>>>>>
>>>>> Thanks
>>>>> Oleg.
>>>>>
>>>>
>>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Leonid Fedotov <lf...@hortonworks.com>.
Oleg,
why don’t you talk to me?
Put 1 ZK and 1 JN on NN boxes, third one on one of the data nodes.
HA wizard will automatically remove SNN from your cluster and put second NN in it’s place instead.
If something does not work as you expected, give me a call.
Thank you!
Leonid Fedotov
Systems Architect - Professional Services
lfedotov@hortonworks.com
office: +1 855 846 7866 ext 292
mobile: +1 650 430 1673
The best way to explain - do it yourself.
On Nov 6, 2014, at 6:01, Oleg Ruchovets <or...@gmail.com> wrote:
> Great.
> Thank you for the link.
> Just to be sure - JN can be installed on data nodes like zookeeper?
> If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK and JN on datanodes machines?
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com> wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation / configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com> wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com> wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Great.
Thank you for the link.
Just to be sure - JN can be installed on data nodes like zookeeper?
If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
and JN on datanodes machines?
Thanks
Oleg.
On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>> hortonworks distributions.
>> Sorry for dummy question - what is JN is?
>> Can you please please point me on some manual wiki for installation /
>> configuration.
>>
>> Thanks
>> Oleg.
>>
>>
>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <apivovarov@gmail.com
>> > wrote:
>>
>>> 2 boxes for 2 NNs (better dedicated boxes)
>>> min 3 JNs
>>> min 3 ZKs
>>>
>>> JNs and ZKs can share boxes with other services
>>>
>>>
>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>> wrote:
>>>
>>>> Hello.
>>>> We are using hortonwork distribution and want to evaluate HA
>>>> capabilities.
>>>> Can community please share the best practices and potential problems?
>>>> Is it required additional hardware?
>>>>
>>>> Thanks
>>>> Oleg.
>>>>
>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Great.
Thank you for the link.
Just to be sure - JN can be installed on data nodes like zookeeper?
If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
and JN on datanodes machines?
Thanks
Oleg.
On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>> hortonworks distributions.
>> Sorry for dummy question - what is JN is?
>> Can you please please point me on some manual wiki for installation /
>> configuration.
>>
>> Thanks
>> Oleg.
>>
>>
>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <apivovarov@gmail.com
>> > wrote:
>>
>>> 2 boxes for 2 NNs (better dedicated boxes)
>>> min 3 JNs
>>> min 3 ZKs
>>>
>>> JNs and ZKs can share boxes with other services
>>>
>>>
>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>> wrote:
>>>
>>>> Hello.
>>>> We are using hortonwork distribution and want to evaluate HA
>>>> capabilities.
>>>> Can community please share the best practices and potential problems?
>>>> Is it required additional hardware?
>>>>
>>>> Thanks
>>>> Oleg.
>>>>
>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Great.
Thank you for the link.
Just to be sure - JN can be installed on data nodes like zookeeper?
If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
and JN on datanodes machines?
Thanks
Oleg.
On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>> hortonworks distributions.
>> Sorry for dummy question - what is JN is?
>> Can you please please point me on some manual wiki for installation /
>> configuration.
>>
>> Thanks
>> Oleg.
>>
>>
>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <apivovarov@gmail.com
>> > wrote:
>>
>>> 2 boxes for 2 NNs (better dedicated boxes)
>>> min 3 JNs
>>> min 3 ZKs
>>>
>>> JNs and ZKs can share boxes with other services
>>>
>>>
>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>> wrote:
>>>
>>>> Hello.
>>>> We are using hortonwork distribution and want to evaluate HA
>>>> capabilities.
>>>> Can community please share the best practices and potential problems?
>>>> Is it required additional hardware?
>>>>
>>>> Thanks
>>>> Oleg.
>>>>
>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Great.
Thank you for the link.
Just to be sure - JN can be installed on data nodes like zookeeper?
If we have 2 Name Nodes and 15 Data Nodes - is it correct to install ZK
and JN on datanodes machines?
Thanks
Oleg.
On Thu, Nov 6, 2014 at 5:06 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> To Enable HA open Ambari, go to Admin, select HA, click enable HA
>
>
> http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
>
> On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
>> hortonworks distributions.
>> Sorry for dummy question - what is JN is?
>> Can you please please point me on some manual wiki for installation /
>> configuration.
>>
>> Thanks
>> Oleg.
>>
>>
>> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <apivovarov@gmail.com
>> > wrote:
>>
>>> 2 boxes for 2 NNs (better dedicated boxes)
>>> min 3 JNs
>>> min 3 ZKs
>>>
>>> JNs and ZKs can share boxes with other services
>>>
>>>
>>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>>> wrote:
>>>
>>>> Hello.
>>>> We are using hortonwork distribution and want to evaluate HA
>>>> capabilities.
>>>> Can community please share the best practices and potential problems?
>>>> Is it required additional hardware?
>>>>
>>>> Thanks
>>>> Oleg.
>>>>
>>>
>>>
>>
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
To Enable HA open Ambari, go to Admin, select HA, click enable HA
http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
> hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> 2 boxes for 2 NNs (better dedicated boxes)
>> min 3 JNs
>> min 3 ZKs
>>
>> JNs and ZKs can share boxes with other services
>>
>>
>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Hello.
>>> We are using hortonwork distribution and want to evaluate HA
>>> capabilities.
>>> Can community please share the best practices and potential problems?
>>> Is it required additional hardware?
>>>
>>> Thanks
>>> Oleg.
>>>
>>
>>
>
Re: High Availability hadoop cluster.
Posted by Andrej <an...@bsdaemon.hu>.
JN is the journalnode. You have to use it for HA. Like zookeeper, you
need odd numbers of JN.
Anyway ZKFC is neccessary too for automatic failover.
On 2014.11.06. 9:45, Oleg Ruchovets wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part
> of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov
> <apivovarov@gmail.com <ma...@gmail.com>> wrote:
>
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets
> <oruchovets@gmail.com <ma...@gmail.com>> wrote:
>
> Hello.
> We are using hortonwork distribution and want to evaluate HA
> capabilities.
> Can community please share the best practices and potential
> problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
Re: High Availability hadoop cluster.
Posted by Andrej <an...@bsdaemon.hu>.
JN is the journalnode. You have to use it for HA. Like zookeeper, you
need odd numbers of JN.
Anyway ZKFC is neccessary too for automatic failover.
On 2014.11.06. 9:45, Oleg Ruchovets wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part
> of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov
> <apivovarov@gmail.com <ma...@gmail.com>> wrote:
>
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets
> <oruchovets@gmail.com <ma...@gmail.com>> wrote:
>
> Hello.
> We are using hortonwork distribution and want to evaluate HA
> capabilities.
> Can community please share the best practices and potential
> problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
Re: High Availability hadoop cluster.
Posted by Andrej <an...@bsdaemon.hu>.
JN is the journalnode. You have to use it for HA. Like zookeeper, you
need odd numbers of JN.
Anyway ZKFC is neccessary too for automatic failover.
On 2014.11.06. 9:45, Oleg Ruchovets wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part
> of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov
> <apivovarov@gmail.com <ma...@gmail.com>> wrote:
>
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets
> <oruchovets@gmail.com <ma...@gmail.com>> wrote:
>
> Hello.
> We are using hortonwork distribution and want to evaluate HA
> capabilities.
> Can community please share the best practices and potential
> problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
To Enable HA open Ambari, go to Admin, select HA, click enable HA
http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
> hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> 2 boxes for 2 NNs (better dedicated boxes)
>> min 3 JNs
>> min 3 ZKs
>>
>> JNs and ZKs can share boxes with other services
>>
>>
>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Hello.
>>> We are using hortonwork distribution and want to evaluate HA
>>> capabilities.
>>> Can community please share the best practices and potential problems?
>>> Is it required additional hardware?
>>>
>>> Thanks
>>> Oleg.
>>>
>>
>>
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
To Enable HA open Ambari, go to Admin, select HA, click enable HA
http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
> hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> 2 boxes for 2 NNs (better dedicated boxes)
>> min 3 JNs
>> min 3 ZKs
>>
>> JNs and ZKs can share boxes with other services
>>
>>
>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Hello.
>>> We are using hortonwork distribution and want to evaluate HA
>>> capabilities.
>>> Can community please share the best practices and potential problems?
>>> Is it required additional hardware?
>>>
>>> Thanks
>>> Oleg.
>>>
>>
>>
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
To Enable HA open Ambari, go to Admin, select HA, click enable HA
http://dev.hortonworks.com.s3.amazonaws.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_using_Ambari_book/content/install-ha_2x.html
On Thu, Nov 6, 2014 at 12:45 AM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
> hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>
>> 2 boxes for 2 NNs (better dedicated boxes)
>> min 3 JNs
>> min 3 ZKs
>>
>> JNs and ZKs can share boxes with other services
>>
>>
>> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
>> wrote:
>>
>>> Hello.
>>> We are using hortonwork distribution and want to evaluate HA
>>> capabilities.
>>> Can community please share the best practices and potential problems?
>>> Is it required additional hardware?
>>>
>>> Thanks
>>> Oleg.
>>>
>>
>>
>
Re: High Availability hadoop cluster.
Posted by Andrej <an...@bsdaemon.hu>.
JN is the journalnode. You have to use it for HA. Like zookeeper, you
need odd numbers of JN.
Anyway ZKFC is neccessary too for automatic failover.
On 2014.11.06. 9:45, Oleg Ruchovets wrote:
> Our deployment is 15 data nodes and 2 NN. ZKs installed also as part
> of hortonworks distributions.
> Sorry for dummy question - what is JN is?
> Can you please please point me on some manual wiki for installation /
> configuration.
>
> Thanks
> Oleg.
>
>
> On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov
> <apivovarov@gmail.com <ma...@gmail.com>> wrote:
>
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets
> <oruchovets@gmail.com <ma...@gmail.com>> wrote:
>
> Hello.
> We are using hortonwork distribution and want to evaluate HA
> capabilities.
> Can community please share the best practices and potential
> problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
>
>
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
hortonworks distributions.
Sorry for dummy question - what is JN is?
Can you please please point me on some manual wiki for installation /
configuration.
Thanks
Oleg.
On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Hello.
>> We are using hortonwork distribution and want to evaluate HA capabilities.
>> Can community please share the best practices and potential problems?
>> Is it required additional hardware?
>>
>> Thanks
>> Oleg.
>>
>
>
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
hortonworks distributions.
Sorry for dummy question - what is JN is?
Can you please please point me on some manual wiki for installation /
configuration.
Thanks
Oleg.
On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Hello.
>> We are using hortonwork distribution and want to evaluate HA capabilities.
>> Can community please share the best practices and potential problems?
>> Is it required additional hardware?
>>
>> Thanks
>> Oleg.
>>
>
>
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
hortonworks distributions.
Sorry for dummy question - what is JN is?
Can you please please point me on some manual wiki for installation /
configuration.
Thanks
Oleg.
On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Hello.
>> We are using hortonwork distribution and want to evaluate HA capabilities.
>> Can community please share the best practices and potential problems?
>> Is it required additional hardware?
>>
>> Thanks
>> Oleg.
>>
>
>
Re: High Availability hadoop cluster.
Posted by Oleg Ruchovets <or...@gmail.com>.
Our deployment is 15 data nodes and 2 NN. ZKs installed also as part of
hortonworks distributions.
Sorry for dummy question - what is JN is?
Can you please please point me on some manual wiki for installation /
configuration.
Thanks
Oleg.
On Thu, Nov 6, 2014 at 4:04 PM, Alexander Pivovarov <ap...@gmail.com>
wrote:
> 2 boxes for 2 NNs (better dedicated boxes)
> min 3 JNs
> min 3 ZKs
>
> JNs and ZKs can share boxes with other services
>
>
> On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
> wrote:
>
>> Hello.
>> We are using hortonwork distribution and want to evaluate HA capabilities.
>> Can community please share the best practices and potential problems?
>> Is it required additional hardware?
>>
>> Thanks
>> Oleg.
>>
>
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
2 boxes for 2 NNs (better dedicated boxes)
min 3 JNs
min 3 ZKs
JNs and ZKs can share boxes with other services
On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
2 boxes for 2 NNs (better dedicated boxes)
min 3 JNs
min 3 ZKs
JNs and ZKs can share boxes with other services
On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
2 boxes for 2 NNs (better dedicated boxes)
min 3 JNs
min 3 ZKs
JNs and ZKs can share boxes with other services
On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>
Re: High Availability hadoop cluster.
Posted by Alexander Pivovarov <ap...@gmail.com>.
2 boxes for 2 NNs (better dedicated boxes)
min 3 JNs
min 3 ZKs
JNs and ZKs can share boxes with other services
On Wed, Nov 5, 2014 at 11:31 PM, Oleg Ruchovets <or...@gmail.com>
wrote:
> Hello.
> We are using hortonwork distribution and want to evaluate HA capabilities.
> Can community please share the best practices and potential problems?
> Is it required additional hardware?
>
> Thanks
> Oleg.
>