You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ambari.apache.org by Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com> on 2015/09/18 12:37:06 UTC
Component placement strategy - Ambari
Hi,
>
> Is there any recommendation on the components' placement across the cluster i.e. how the various Hadoop services have to be distributed across the cluster? Especially around which services need to be placed together and which two shouldn’t be placed together?
>
> For example, on the machine which acts as amber server, which other components can be installed? Is there any such guideline available at the moment?
>
> Thanks & Regards
> Vijay
--
The contents of this e-mail are confidential and for the exclusive use of
the intended recipient. If you receive this e-mail in error please delete
it from your system immediately and notify us either by e-mail or
telephone. You should not copy, forward or otherwise disclose the content
of the e-mail. The views expressed in this communication may not
necessarily be the view held by WHISHWORKS.
Re: Component placement strategy - Ambari
Posted by Sumit Mohanty <sm...@hortonworks.com>.
I may not have fully understood this question.
Ambari does not maintain a mapping between FQDN and IP address. It relies on the mapping to be available via external means (say /etc/hosts).
Ambari will use the FQDN as host id/name while producing config values. Say HDFS or HBASE needs ZK quorum. Ambari will set the quorum using the hostnames (e.g. FQDN by default) it obtained from the agents running on the host.
________________________________
From: Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>
Sent: Friday, September 18, 2015 3:27 PM
To: user@ambari.apache.org
Subject: Re: Component placement strategy - Ambari
Thats true that we would provide FQDNs in Ambari. However, if the underlying machines' IP addresses change due to DHCP, how would FQDNs resolve to the new IPs? Or is it always recommended to have static IPs mapped to FQDNs and mentioned in the /etc/hosts on each machine in the cluster?
Regards
Vijay
On 18 Sep 2015, at 18:17, Sumit Mohanty <sm...@hortonworks.com>> wrote:
Ambari uses FQDN and also uses FQDN for configuration values as well.
The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
Re: Component placement strategy - Ambari
Posted by Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>.
Thats true that we would provide FQDNs in Ambari. However, if the underlying machines’ IP addresses change due to DHCP, how would FQDNs resolve to the new IPs? Or is it always recommended to have static IPs mapped to FQDNs and mentioned in the /etc/hosts on each machine in the cluster?
Regards
Vijay
> On 18 Sep 2015, at 18:17, Sumit Mohanty <sm...@hortonworks.com> wrote:
>
> Ambari uses FQDN and also uses FQDN for configuration values as well.
>
--
The contents of this e-mail are confidential and for the exclusive use of
the intended recipient. If you receive this e-mail in error please delete
it from your system immediately and notify us either by e-mail or
telephone. You should not copy, forward or otherwise disclose the content
of the e-mail. The views expressed in this communication may not
necessarily be the view held by WHISHWORKS.
Re: Component placement strategy - Ambari
Posted by Sumit Mohanty <sm...@hortonworks.com>.
Ambari uses FQDN and also uses FQDN for configuration values as well.
________________________________
From: Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>
Sent: Friday, September 18, 2015 9:54 AM
To: user@ambari.apache.org
Subject: Re: Component placement strategy - Ambari
Thanks Sumit, would dig into the stack advisor bit.
One more rudimentary question - while setting up the cluster, should the machines be configured with static IPs (elastic IPs in cloud context) mandatorily or can the machines in the cluster have dynamic IPs as Ambari considers FQDNs and does the internal mapping between IPs and FQDN?
Regards
Vijay
On 18 Sep 2015, at 17:41, Sumit Mohanty <sm...@hortonworks.com>> wrote:
?It is pretty much encoded in the stack advisor logic. Its not translated to a document as such. It also evolves based on feedback received.
You can go through the stack advisor code to get some idea. A good way is to deploy a 3/5/7 node cluster and export the blueprint. That can pretty much be used as a template.
There is an effort to integrate stack advisor with blueprint as well.
________________________________
From: Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>>
Sent: Friday, September 18, 2015 7:57 AM
To: user@ambari.apache.org<ma...@ambari.apache.org>
Subject: Re: Component placement strategy - Ambari
Thanks Sumit. But can you please explain the strategy employed by Ambari, so that in case any tweaks are required, it would be helpful. For example, if the cluster is being setup by using Blueprints, what should be the placement guidelines to be considered?
Regards
Vijay
On 18 Sep 2015, at 15:04, Sumit Mohanty <sm...@hortonworks.com>> wrote:
?If you are using UI to deploy the cluster it will suggest a placement and configuration which is a good default.
________________________________
From: Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>>
Sent: Friday, September 18, 2015 3:37 AM
To: user@ambari.apache.org<ma...@ambari.apache.org>
Subject: Component placement strategy - Ambari
Hi,
Is there any recommendation on the components' placement across the cluster i.e. how the various Hadoop services have to be distributed across the cluster? Especially around which services need to be placed together and which two shouldn't be placed together?
For example, on the machine which acts as amber server, which other components can be installed? Is there any such guideline available at the moment?
Thanks & Regards
Vijay
The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
Re: Component placement strategy - Ambari
Posted by Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>.
Thanks Sumit, would dig into the stack advisor bit.
One more rudimentary question - while setting up the cluster, should the machines be configured with static IPs (elastic IPs in cloud context) mandatorily or can the machines in the cluster have dynamic IPs as Ambari considers FQDNs and does the internal mapping between IPs and FQDN?
Regards
Vijay
> On 18 Sep 2015, at 17:41, Sumit Mohanty <sm...@hortonworks.com> wrote:
>
> It is pretty much encoded in the stack advisor logic. Its not translated to a document as such. It also evolves based on feedback received.
>
> You can go through the stack advisor code to get some idea. A good way is to deploy a 3/5/7 node cluster and export the blueprint. That can pretty much be used as a template.
>
> There is an effort to integrate stack advisor with blueprint as well.
> From: Vijaya Narayana Reddy Bhoomi Reddy <vijaya.bhoomireddy@whishworks.com <ma...@whishworks.com>>
> Sent: Friday, September 18, 2015 7:57 AM
> To: user@ambari.apache.org <ma...@ambari.apache.org>
> Subject: Re: Component placement strategy - Ambari
>
>
> Thanks Sumit. But can you please explain the strategy employed by Ambari, so that in case any tweaks are required, it would be helpful. For example, if the cluster is being setup by using Blueprints, what should be the placement guidelines to be considered?
>
> Regards
> Vijay
>
>
>
>> On 18 Sep 2015, at 15:04, Sumit Mohanty <smohanty@hortonworks.com <ma...@hortonworks.com>> wrote:
>>
>> If you are using UI to deploy the cluster it will suggest a placement and configuration which is a good default.
>> From: Vijaya Narayana Reddy Bhoomi Reddy <vijaya.bhoomireddy@whishworks.com <ma...@whishworks.com>>
>> Sent: Friday, September 18, 2015 3:37 AM
>> To: user@ambari.apache.org <ma...@ambari.apache.org>
>> Subject: Component placement strategy - Ambari
>>
>> Hi,
>>>
>>> Is there any recommendation on the components' placement across the cluster i.e. how the various Hadoop services have to be distributed across the cluster? Especially around which services need to be placed together and which two shouldn’t be placed together?
>>>
>>> For example, on the machine which acts as amber server, which other components can be installed? Is there any such guideline available at the moment?
>>>
>>> Thanks & Regards
>>> Vijay
>>
>>
>> The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
>
>
> The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
--
The contents of this e-mail are confidential and for the exclusive use of
the intended recipient. If you receive this e-mail in error please delete
it from your system immediately and notify us either by e-mail or
telephone. You should not copy, forward or otherwise disclose the content
of the e-mail. The views expressed in this communication may not
necessarily be the view held by WHISHWORKS.
Re: Component placement strategy - Ambari
Posted by Sumit Mohanty <sm...@hortonworks.com>.
?It is pretty much encoded in the stack advisor logic. Its not translated to a document as such. It also evolves based on feedback received.
You can go through the stack advisor code to get some idea. A good way is to deploy a 3/5/7 node cluster and export the blueprint. That can pretty much be used as a template.
There is an effort to integrate stack advisor with blueprint as well.
________________________________
From: Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>
Sent: Friday, September 18, 2015 7:57 AM
To: user@ambari.apache.org
Subject: Re: Component placement strategy - Ambari
Thanks Sumit. But can you please explain the strategy employed by Ambari, so that in case any tweaks are required, it would be helpful. For example, if the cluster is being setup by using Blueprints, what should be the placement guidelines to be considered?
Regards
Vijay
On 18 Sep 2015, at 15:04, Sumit Mohanty <sm...@hortonworks.com>> wrote:
?If you are using UI to deploy the cluster it will suggest a placement and configuration which is a good default.
________________________________
From: Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>>
Sent: Friday, September 18, 2015 3:37 AM
To: user@ambari.apache.org<ma...@ambari.apache.org>
Subject: Component placement strategy - Ambari
Hi,
Is there any recommendation on the components' placement across the cluster i.e. how the various Hadoop services have to be distributed across the cluster? Especially around which services need to be placed together and which two shouldn't be placed together?
For example, on the machine which acts as amber server, which other components can be installed? Is there any such guideline available at the moment?
Thanks & Regards
Vijay
The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
Re: Component placement strategy - Ambari
Posted by Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>.
Thanks Sumit. But can you please explain the strategy employed by Ambari, so that in case any tweaks are required, it would be helpful. For example, if the cluster is being setup by using Blueprints, what should be the placement guidelines to be considered?
Regards
Vijay
> On 18 Sep 2015, at 15:04, Sumit Mohanty <sm...@hortonworks.com> wrote:
>
> If you are using UI to deploy the cluster it will suggest a placement and configuration which is a good default.
> From: Vijaya Narayana Reddy Bhoomi Reddy <vijaya.bhoomireddy@whishworks.com <ma...@whishworks.com>>
> Sent: Friday, September 18, 2015 3:37 AM
> To: user@ambari.apache.org <ma...@ambari.apache.org>
> Subject: Component placement strategy - Ambari
>
> Hi,
>>
>> Is there any recommendation on the components' placement across the cluster i.e. how the various Hadoop services have to be distributed across the cluster? Especially around which services need to be placed together and which two shouldn’t be placed together?
>>
>> For example, on the machine which acts as amber server, which other components can be installed? Is there any such guideline available at the moment?
>>
>> Thanks & Regards
>> Vijay
>
>
> The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.
--
The contents of this e-mail are confidential and for the exclusive use of
the intended recipient. If you receive this e-mail in error please delete
it from your system immediately and notify us either by e-mail or
telephone. You should not copy, forward or otherwise disclose the content
of the e-mail. The views expressed in this communication may not
necessarily be the view held by WHISHWORKS.
Re: Component placement strategy - Ambari
Posted by Sumit Mohanty <sm...@hortonworks.com>.
?If you are using UI to deploy the cluster it will suggest a placement and configuration which is a good default.
________________________________
From: Vijaya Narayana Reddy Bhoomi Reddy <vi...@whishworks.com>
Sent: Friday, September 18, 2015 3:37 AM
To: user@ambari.apache.org
Subject: Component placement strategy - Ambari
Hi,
Is there any recommendation on the components' placement across the cluster i.e. how the various Hadoop services have to be distributed across the cluster? Especially around which services need to be placed together and which two shouldn't be placed together?
For example, on the machine which acts as amber server, which other components can be installed? Is there any such guideline available at the moment?
Thanks & Regards
Vijay
The contents of this e-mail are confidential and for the exclusive use of the intended recipient. If you receive this e-mail in error please delete it from your system immediately and notify us either by e-mail or telephone. You should not copy, forward or otherwise disclose the content of the e-mail. The views expressed in this communication may not necessarily be the view held by WHISHWORKS.