[jira] [Commented] (AMBARI-1783) Specification for a cluster blueprint consumable by Ambari

["Brian Swan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMBARI-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633330#comment-13633330 ] 

Brian Swan commented on AMBARI-1783:
------------------------------------

This is a great start. Some thoughts...

• Allowing the various sections (Stack Def., Stack Conf., Host Manifest, Host-Component Mapping) to be consumable separately would be helpful for a number of reasons:
		○ Stack Def will be written by a stack author. A cluster admin (someone deploying a cluster) does not need to be concerned with these. He *does* need to be concerned with the Host Manifest and Host-Component Mapping (which describe the topology of the cluster).
		○ In some cases, host details (as specified in the Host Manifest) will be controlled by other tools (such as System Center Virtual Machine Manger or an agent in a Cloud environment).
		○ In some cases, Host-Component Mapping will be controlled by a tool (such as System Center Virtual Machine Manager, or an agent in a Cloud environment) and will not depend on this content in the blueprint (i.e. it needs to be ignorable).
		○ Introducing a level of abstraction to the Host Manifest and Host-Component Mapping content that allows a cluster admin to define the topology of a cluster without specifying host details of mapping details would help with the two previous points. For example:

			"Node": [
			        {
			            "Name": "Master",
			            "Count": 1,
			            "Component": [ "openjdk7", "core", "namenode", "resourcemanager"],
			            "Type": "HeadNode"
			        },
			        {
			            "Name": "Slave",
			            "Count": 4,
			            "Component": [ "openjdk7", "core", "datanode", "nodemanager", "drawbridge"],
			            "Type": "WorkerNode"
			        }
			    ],
			

	• Providing for an option to instal custom software (such as Python or a specific version of the JDK) across a cluster could be very useful. This could be as simple as a reference to a script that knows how to install software. For example, this could be part of the Stack Definition (though this doesn't adhere exactly to the proposed syntax):

		"Service": [
		        {
		            "Name": "OracleJDK",
		            "Component": [
		                {
		                    "Cardinality": "*",
		                    "IsRequired": false,
		                    "IsServerComponent": false,
		                    "Name": "OracleJDK",
		                    "Package": {
		                        "Type": "custom",
		                        "Name": "OracleJDK",
		                        "ScriptFileName": "OracleJDK.ps1"
		                    }
		                }
		            ]
		        },
		
	• Resource URIs in the Stack Configuration would allow for identification of where distributions (whether they be Hadoop distributions or custom software) could be downloaded. In the example below, the Resources identify where the hdinsight and python packages should be downloaded from:

		"Distros": [
		        { "Name": "hdinsight", "Version": "1.0.1", "FileName": "hdp-azure-1.0.1.package.zip" },
			{ "Name": "python", "Version": "2.6", "FileName": "python.zip"}
		    ],
		    "Resources": [
			{"HDP":"http://mycontainer.blob.core.windows.net/hdp-azure-1.01.package.zip"},
			{"Python":"http://mycontainer.blob.core.windows.net/python.zip"}]
		    ],
		
		
	• A Users section in the Stack Configuration would allow for users to be created on machines in the cluster as they are provisioned.  Hadoop services would run as these users. This section could be ignored when it isn't needed.
		"Users": [ 
		        { 
		            "Groups": [ 
		                "hadoopadmin" 
		            ], 
		            "Name": "hadoopuser1", 
		            "Password": "pwd", 
		            "Type": "local" 
		        } 
		    ],
		

                
> Specification for a cluster blueprint consumable by Ambari
> ----------------------------------------------------------
>
>                 Key: AMBARI-1783
>                 URL: https://issues.apache.org/jira/browse/AMBARI-1783
>             Project: Ambari
>          Issue Type: New Feature
>    Affects Versions: 1.3.1
>            Reporter: Sumit Mohanty
>            Assignee: Sumit Mohanty
>             Fix For: 1.3.1
>
>         Attachments: Ambari-BluePrint-0.1.docx, CompleteBlueprint.txt, HostComponentMapping.txt, HostManifest.txt, StackConfiguration.txt, StackDefinition.txt
>
>
> Deployment of a hadoop cluster can be modeled as the mapping of a stack definition to a set of host while satisfying placement constraints. A stack definition refers to the description of various services that comprise a hadoop stack (e.g. HDP-1.2.1), components associated with the services, and configuration properties. A set of available hosts is specified through a host-manifest that lists the available hosts and relevant properties/characteristics of each host. A cluster blueprint is specification of a hadoop stack mapped to a set of hosts. Mapping to a host can be a direct mapping - e.g. deploy this component X on host Y or a host can be specified as a set of requirements which is used to select the most appropriate host(s) from a host-manifest.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira