You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Cabir Zounaidou <cz...@hortonworks.com> on 2014/09/12 09:44:33 UTC
Review Request 25572: Associating repository latest url for deploying
cluster using blueprint.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
Summary (updated)
-----------------
Associating repository latest url for deploying cluster using blueprint.
Bugs: 19947
https://issues.apache.org/jira/browse/19947
Repository: ambari
Description (updated)
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java f40979c
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java d5f575a
Diff: https://reviews.apache.org/r/25572/diff/
Testing (updated)
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by John Speidel <js...@hortonworks.com>.
> On Sept. 12, 2014, 7:15 p.m., John Speidel wrote:
> > Please don't merge this.
> > The blueprint processing code shouldn't be dealing with repo url's.
> >
> > Instead, what should be done is to change the code that writes out the xml file during installation.
> > There are 3 relevant fields in the xml:
> > "base_url" : value to be used unless overridden by an explicit call
> > "default_base_url" : value specified in the stack xml
> > "latest_base_url" : latest obtained from an external URL query. May be null
> >
> > When writing this file, base_url should be set to the value of latest_base_url if it is not null. This is only done once when Ambari is installed so it will remain consistent after a cluster is created. This way, the latest will be used by default when deploying via a blueprint and the repo url will not change even if the latest value does. A user can still explicitly set a value for the repo url which will override base_url if desired.
> >
> > Please see continuing conversation in Jira.
>
> Cabir Zounaidou wrote:
> John, i wont' merge until you say Ship It. I liked the idea of writing the latest url (from repoinfo.xml) in an xml during Ambari installation. But what i don't understand is that the deploying cluster via a blueprint, still needs to use this xml file (created using installation time) and update the repo resources using repo api's, is that okay or am i missing something here? Also, we need to have some processing code on the repo url still, if the scenario is that before deploying the cluster via BP, user has updated the stack repository url using the rest api.
Cabir, what I am suggesting is that there will be no repo related logic in the blueprint code. The BP processing code will not pass a any repo information during install operations.
All logic for determining the proper repo url is in the agent code. This code already exists and handles the case where the user explicitly sets the repo url via the api.
The only change will be that for cases where we can obtain a "latest" repo, this will be set in the configuration during installation (once) as the value that is used by default instead of the value from the stack xml.
So, the existing agent code will work as is with this change since all we are doing is changing how we initialize the value that the agent was already using.
The code for handling the explicit setting of the repo url already exists and will remain unchanged.
When a user sets the repo url explicitly prior to an install, the code that provides that repo url to the agent will work with our change as all we are doing is changing how we set the default value.
Please ping me if you still have questions and we can talk.
- John
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53203
-----------------------------------------------------------
On Sept. 12, 2014, 7:08 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 12, 2014, 7:08 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java f40979c
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java d5f575a
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
> On Sept. 12, 2014, 7:15 p.m., John Speidel wrote:
> > Please don't merge this.
> > The blueprint processing code shouldn't be dealing with repo url's.
> >
> > Instead, what should be done is to change the code that writes out the xml file during installation.
> > There are 3 relevant fields in the xml:
> > "base_url" : value to be used unless overridden by an explicit call
> > "default_base_url" : value specified in the stack xml
> > "latest_base_url" : latest obtained from an external URL query. May be null
> >
> > When writing this file, base_url should be set to the value of latest_base_url if it is not null. This is only done once when Ambari is installed so it will remain consistent after a cluster is created. This way, the latest will be used by default when deploying via a blueprint and the repo url will not change even if the latest value does. A user can still explicitly set a value for the repo url which will override base_url if desired.
> >
> > Please see continuing conversation in Jira.
John, i wont' merge until you say Ship It. I liked the idea of writing the latest url (from repoinfo.xml) in an xml during Ambari installation. But what i don't understand is that the deploying cluster via a blueprint, still needs to use this xml file (created using installation time) and update the repo resources using repo api's, is that okay or am i missing something here? Also, we need to have some processing code on the repo url still, if the scenario is that before deploying the cluster via BP, user has updated the stack repository url using the rest api.
- Cabir
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53203
-----------------------------------------------------------
On Sept. 12, 2014, 7:08 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 12, 2014, 7:08 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java f40979c
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java d5f575a
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by John Speidel <js...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53203
-----------------------------------------------------------
Please don't merge this.
The blueprint processing code shouldn't be dealing with repo url's.
Instead, what should be done is to change the code that writes out the xml file during installation.
There are 3 relevant fields in the xml:
"base_url" : value to be used unless overridden by an explicit call
"default_base_url" : value specified in the stack xml
"latest_base_url" : latest obtained from an external URL query. May be null
When writing this file, base_url should be set to the value of latest_base_url if it is not null. This is only done once when Ambari is installed so it will remain consistent after a cluster is created. This way, the latest will be used by default when deploying via a blueprint and the repo url will not change even if the latest value does. A user can still explicitly set a value for the repo url which will override base_url if desired.
Please see continuing conversation in Jira.
- John Speidel
On Sept. 12, 2014, 7:08 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 12, 2014, 7:08 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java f40979c
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java d5f575a
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by John Speidel <js...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53677
-----------------------------------------------------------
You should have unit tests for the code changes.
As best I can tell the code changes look reasonable but because I don't know this code at all, I can't know if it is doing the right thing.
You should make sure that you get it +1'd by somebody that has knowledge in this area (possibly Nate).
- John Speidel
On Sept. 16, 2014, 3:59 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 16, 2014, 3:59 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
> On Sept. 19, 2014, 11:29 a.m., Nate Cole wrote:
> > This latest patch looks good. Only for tests - please make sure all the various paths-to-default specified in the JIRA are covered in test cases. It looks like you have only one of them. So we need:
> >
> > Scenario: user has internet and does nothing to repos via api
> > use the latest
> >
> > Scenario: user has internet and but calls to set repos via api
> > use whatever they set
> >
> > Scenario: user has no internet and does nothing to repos via api
> > use the default
> >
> > Scenario: user has no internet and but calls to set repos via api
> > use whatever they set
Thanks very much for the review. Now, added more testcases to test each of the scenarios.
- Cabir
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53952
-----------------------------------------------------------
On Sept. 18, 2014, 10:09 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 10:09 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53952
-----------------------------------------------------------
This latest patch looks good. Only for tests - please make sure all the various paths-to-default specified in the JIRA are covered in test cases. It looks like you have only one of them. So we need:
Scenario: user has internet and does nothing to repos via api
use the latest
Scenario: user has internet and but calls to set repos via api
use whatever they set
Scenario: user has no internet and does nothing to repos via api
use the default
Scenario: user has no internet and but calls to set repos via api
use whatever they set
- Nate Cole
On Sept. 18, 2014, 6:09 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 6:09 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
> On Sept. 19, 2014, 8:09 p.m., Nate Cole wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java, line 203
> > <https://reviews.apache.org/r/25572/diff/6/?file=696999#file696999line203>
> >
> > May want an assertion here. Since this has been done quietly, you would never know
I will add an assertion.
> On Sept. 19, 2014, 8:09 p.m., Nate Cole wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java, line 259
> > <https://reviews.apache.org/r/25572/diff/6/?file=696999#file696999line259>
> >
> > Assert this has happened
I will add.
> On Sept. 19, 2014, 8:09 p.m., Nate Cole wrote:
> > ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java, lines 1555-1565
> > <https://reviews.apache.org/r/25572/diff/6/?file=696999#file696999line1555>
> >
> > Does this directory get cleaned up anytime?
Junit will automatically cleans the directory when the test passes or fails.
- Cabir
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review54007
-----------------------------------------------------------
On Sept. 19, 2014, 7:53 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 19, 2014, 7:53 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review54007
-----------------------------------------------------------
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
<https://reviews.apache.org/r/25572/#comment93871>
May want an assertion here. Since this has been done quietly, you would never know
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
<https://reviews.apache.org/r/25572/#comment93872>
Assert this has happened
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
<https://reviews.apache.org/r/25572/#comment93874>
Does this directory get cleaned up anytime?
- Nate Cole
On Sept. 19, 2014, 3:53 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 19, 2014, 3:53 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review54037
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Sept. 19, 2014, 5:31 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 19, 2014, 5:31 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
> ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 20, 2014, 2:07 a.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
Changes
-------
Fixed the testcase failure in the apache build environment.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
:
:
"hostLevelParams": {
"jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
"ambari_db_rca_password": "mapred",
"java_home": "/usr/jdk64/jdk1.7.0_45",
"ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
"jce_name": "UnlimitedJCEPolicyJDK7.zip",
"oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
"repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
"group_list": "[\"hadoop\",\"users\"]",
:
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 19, 2014, 9:31 p.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
Changes
-------
Addressed Nate's comments.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
:
:
"hostLevelParams": {
"jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
"ambari_db_rca_password": "mapred",
"java_home": "/usr/jdk64/jdk1.7.0_45",
"ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
"jce_name": "UnlimitedJCEPolicyJDK7.zip",
"oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
"repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
"group_list": "[\"hadoop\",\"users\"]",
:
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 19, 2014, 7:53 p.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
Changes
-------
Minor fix in the testcase. No need to set the injector on AmbariMetaInfo as it is already injected.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
:
:
"hostLevelParams": {
"jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
"ambari_db_rca_password": "mapred",
"java_home": "/usr/jdk64/jdk1.7.0_45",
"ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
"jce_name": "UnlimitedJCEPolicyJDK7.zip",
"oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
"repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
"group_list": "[\"hadoop\",\"users\"]",
:
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 19, 2014, 7:13 p.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
Changes
-------
Updated with more testcases.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
:
:
"hostLevelParams": {
"jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
"ambari_db_rca_password": "mapred",
"java_home": "/usr/jdk64/jdk1.7.0_45",
"ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
"jce_name": "UnlimitedJCEPolicyJDK7.zip",
"oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
"repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
"group_list": "[\"hadoop\",\"users\"]",
:
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 18, 2014, 10:09 p.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
Changes
-------
Updating the baseUrl with the latestUrl url is now done at repository loading time.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/state/stack/LatestRepoCallable.java 47d93f1
ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java d13e5c8
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
:
:
"hostLevelParams": {
"jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
"ambari_db_rca_password": "mapred",
"java_home": "/usr/jdk64/jdk1.7.0_45",
"ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
"jce_name": "UnlimitedJCEPolicyJDK7.zip",
"oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
"repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
"group_list": "[\"hadoop\",\"users\"]",
:
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
> On Sept. 18, 2014, 6:43 p.m., Nate Cole wrote:
> >
I think it would be better to move the logic to StackInfo.getRepositories() because it is been called by AmbariMetaInfo.getRepository(...) and AmbariMetaInfo.getRepositories(...) overloaded methods.
Any suggestions?
> On Sept. 18, 2014, 6:43 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java, lines 886-905
> > <https://reviews.apache.org/r/25572/diff/3/?file=693144#file693144line886>
> >
> > Could this be handled on server startup when we load all the repos? Doesn't seem like it should take a Custom Command execution to make this decision.
>
> Cabir Zounaidou wrote:
> Sure, that can be done too. I will make the changes and will update the patch.
>
> Mahadev Konar wrote:
> Good point - Nate can you suggest where in the code we can add it?
>
> Nate Cole wrote:
> In AmbariMetaInfo is where all the repo data is pulled out of XML and put into "proper" objects. Look for List<RepositoryInfo> getRepository(...).
Basically, move the above logic inside AmbariMetaInfo.getRepository() right? Correct me if i am wrong.
- Cabir
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53845
-----------------------------------------------------------
On Sept. 18, 2014, 2:36 a.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 2:36 a.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java 898efbf
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
> On Sept. 18, 2014, 6:43 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java, lines 886-905
> > <https://reviews.apache.org/r/25572/diff/3/?file=693144#file693144line886>
> >
> > Could this be handled on server startup when we load all the repos? Doesn't seem like it should take a Custom Command execution to make this decision.
Sure, that can be done too. I will make the changes and will update the patch.
- Cabir
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53845
-----------------------------------------------------------
On Sept. 18, 2014, 2:36 a.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 2:36 a.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java 898efbf
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Nate Cole <nc...@hortonworks.com>.
> On Sept. 18, 2014, 2:43 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java, lines 886-905
> > <https://reviews.apache.org/r/25572/diff/3/?file=693144#file693144line886>
> >
> > Could this be handled on server startup when we load all the repos? Doesn't seem like it should take a Custom Command execution to make this decision.
>
> Cabir Zounaidou wrote:
> Sure, that can be done too. I will make the changes and will update the patch.
>
> Mahadev Konar wrote:
> Good point - Nate can you suggest where in the code we can add it?
In AmbariMetaInfo is where all the repo data is pulled out of XML and put into "proper" objects. Look for List<RepositoryInfo> getRepository(...).
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53845
-----------------------------------------------------------
On Sept. 17, 2014, 10:36 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 17, 2014, 10:36 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java 898efbf
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Mahadev Konar <ma...@apache.org>.
> On Sept. 18, 2014, 6:43 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java, lines 886-905
> > <https://reviews.apache.org/r/25572/diff/3/?file=693144#file693144line886>
> >
> > Could this be handled on server startup when we load all the repos? Doesn't seem like it should take a Custom Command execution to make this decision.
>
> Cabir Zounaidou wrote:
> Sure, that can be done too. I will make the changes and will update the patch.
Good point - Nate can you suggest where in the code we can add it?
- Mahadev
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53845
-----------------------------------------------------------
On Sept. 18, 2014, 2:36 a.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 2:36 a.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java 898efbf
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
> On Sept. 18, 2014, 6:43 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java, lines 886-905
> > <https://reviews.apache.org/r/25572/diff/3/?file=693144#file693144line886>
> >
> > Could this be handled on server startup when we load all the repos? Doesn't seem like it should take a Custom Command execution to make this decision.
>
> Cabir Zounaidou wrote:
> Sure, that can be done too. I will make the changes and will update the patch.
>
> Mahadev Konar wrote:
> Good point - Nate can you suggest where in the code we can add it?
>
> Nate Cole wrote:
> In AmbariMetaInfo is where all the repo data is pulled out of XML and put into "proper" objects. Look for List<RepositoryInfo> getRepository(...).
>
> Cabir Zounaidou wrote:
> Basically, move the above logic inside AmbariMetaInfo.getRepository() right? Correct me if i am wrong.
AmbariMetaInfo method "private List<RepositoryInfo> getRepository(File repositoryFile, StackInfo stack,
List<LatestRepoCallable> lookupList)" is where the repository is getting initialized. I will do the logic here unless there is any objection.
- Cabir
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53845
-----------------------------------------------------------
On Sept. 18, 2014, 2:36 a.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 18, 2014, 2:36 a.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java 898efbf
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/#review53845
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
<https://reviews.apache.org/r/25572/#comment93706>
Could this be handled on server startup when we load all the repos? Doesn't seem like it should take a Custom Command execution to make this decision.
- Nate Cole
On Sept. 17, 2014, 10:36 p.m., Cabir Zounaidou wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25572/
> -----------------------------------------------------------
>
> (Updated Sept. 17, 2014, 10:36 p.m.)
>
>
> Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
>
>
> Bugs: AMBARI-7286
> https://issues.apache.org/jira/browse/AMBARI-7286
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Steps:
> 1. Create a blueprint
> {"configurations": [], "host_groups": [
> {"name": "host1",
> "cardinality": "1",
> "components": [
> {"name": "DATANODE" },
> {"name": "SECONDARY_NAMENODE" },
> {"name": "NAMENODE" },
> {"name": "ZOOKEEPER_SERVER" },
> {"name": "ZOOKEEPER_CLIENT" },
> {"name": "HDFS_CLIENT" }
> ]
> }
> ], "Blueprints": {
> "blueprint_name": "hdfszk",
> "stack_name": "HDP",
> "stack_version": "2.1"
> }}
>
> 2. Create a cluster template to deploy using blueprint
> {
> "configurations": [
> {
>
> "cluster-env":{
> "tag":"version1",
> "properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
> }
> }
>
> ],
> "blueprint": "hdfszk",
> "host_groups": [
> {
> "name": "host1",
> "hosts": [
> { "fqdn": "c6401.ambari.apache.org" }
> ]
> }
> ]
> }
>
> 3. Deploy an instance of cluster using the above cluster template
>
> 4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java 898efbf
>
> Diff: https://reviews.apache.org/r/25572/diff/
>
>
> Testing
> -------
>
> 1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
> 2. Manually tested deploying a cluster from the UI
> 3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
> 3.1 Create a blueprint using the API
> curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
> 3.2 Create a cluster from the template which was using the blueprint
> curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
> 3.3 Checked the status of the task until it is completed.
> 3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
> 3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
> :
> :
> "hostLevelParams": {
> "jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
> "ambari_db_rca_password": "mapred",
> "java_home": "/usr/jdk64/jdk1.7.0_45",
> "ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
> "jce_name": "UnlimitedJCEPolicyJDK7.zip",
> "oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
> "repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
> "group_list": "[\"hadoop\",\"users\"]",
>
> :
>
>
> Thanks,
>
> Cabir Zounaidou
>
>
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 18, 2014, 2:36 a.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, Nate Cole, Robert Nettleton, and Srimanth Gunturi.
Changes
-------
Updated with testcases for the changes.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java 898efbf
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
:
:
"hostLevelParams": {
"jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
"ambari_db_rca_password": "mapred",
"java_home": "/usr/jdk64/jdk1.7.0_45",
"ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
"jce_name": "UnlimitedJCEPolicyJDK7.zip",
"oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
"repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
"group_list": "[\"hadoop\",\"users\"]",
:
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 16, 2014, 3:59 p.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
Changes
-------
Updated the patch. Now the latest url is set during the install of cluster via blueprint execution command generation. The agent will receive the latest url as base url if deployed cluster via blueprint. Verified the command-x.json file generated under the /var/lib/ambari-agent.data/ directory and the latest url is sent as expected.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java 156427d
Diff: https://reviews.apache.org/r/25572/diff/
Testing (updated)
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
3.5 Also verified the generated command file (/var/lib/ambari-agent/data/command-x.json) for the latest url. It sets the baseUrl="http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\", which is same as the latestBaseUrl.
:
:
"hostLevelParams": {
"jdk_location": "http://c6401.ambari.apache.org:8080/resources/",
"ambari_db_rca_password": "mapred",
"java_home": "/usr/jdk64/jdk1.7.0_45",
"ambari_db_rca_url": "jdbc:postgresql://c6401.ambari.apache.org/ambarirca",
"jce_name": "UnlimitedJCEPolicyJDK7.zip",
"oracle_jdbc_url": "http://c6401.ambari.apache.org:8080/resources//ojdbc6.jar",
"repo_info": "[{\"baseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\",\"osType\":\"redhat6\",\"repoId\":\"HDP-2.1\",\"repoName\":\"HDP\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0/\",\"latestBaseUrl\":\"http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.1.5.0\"},{\"baseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"osType\":\"redhat6\",\"repoId\":\"HDP-UTILS-1.1.0.19\",\"repoName\":\"HDP-UTILS\",\"defaultBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\",\"latestBaseUrl\":\"http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.19/repos/centos6\"}]",
"group_list": "[\"hadoop\",\"users\"]",
:
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 12, 2014, 7:08 p.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
Changes
-------
Updated with external bug id.
Bugs: AMBARI-7286
https://issues.apache.org/jira/browse/AMBARI-7286
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java f40979c
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java d5f575a
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
Thanks,
Cabir Zounaidou
Re: Review Request 25572: Associating repository latest url for
deploying cluster using blueprint.
Posted by Cabir Zounaidou <cz...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25572/
-----------------------------------------------------------
(Updated Sept. 12, 2014, 7:46 a.m.)
Review request for Ambari, Erik Bergenholtz, John Speidel, Mahadev Konar, and Robert Nettleton.
Bugs: 19947
https://issues.apache.org/jira/browse/19947
Repository: ambari
Description
-------
Steps:
1. Create a blueprint
{"configurations": [], "host_groups": [
{"name": "host1",
"cardinality": "1",
"components": [
{"name": "DATANODE" },
{"name": "SECONDARY_NAMENODE" },
{"name": "NAMENODE" },
{"name": "ZOOKEEPER_SERVER" },
{"name": "ZOOKEEPER_CLIENT" },
{"name": "HDFS_CLIENT" }
]
}
], "Blueprints": {
"blueprint_name": "hdfszk",
"stack_name": "HDP",
"stack_version": "2.1"
}}
2. Create a cluster template to deploy using blueprint
{
"configurations": [
{
"cluster-env":{
"tag":"version1",
"properties":{"security_enabled":"false","kerberos_domain": "EXAMPLE.COM","ignore_groupsusers_create": "false", "smokeuser": "ambari-qa", "user_group": "hadoop"}
}
}
],
"blueprint": "hdfszk",
"host_groups": [
{
"name": "host1",
"hosts": [
{ "fqdn": "c6401.ambari.apache.org" }
]
}
]
}
3. Deploy an instance of cluster using the above cluster template
4. Verify the URL's in the Admin/repositories page of UI. It was not be using the latest url specified in the repoinfo.xml of the given stack and version before this fix.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java f40979c
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java d5f575a
Diff: https://reviews.apache.org/r/25572/diff/
Testing
-------
1. Ran the unit tests. The ClusterResourceProviderTest.java has unit for deploying cluster using blueprint.
2. Manually tested deploying a cluster from the UI
3. Ran the REST-API's to test deploying cluster using blueprint and verified the repository urls.
3.1 Create a blueprint using the API
curl -i --user admin:adminhttp://c6401.ambari.apache.org:8080/api/v1/blueprints/blueprint1 -X POST -d @/tmp/myblueprint_template_json -H "X-Requested-By: ambari"
3.2 Create a cluster from the template which was using the blueprint
curl -i --user admin:admin -H "X-Requested-By: ambari" -X POST -d @/tmp/cluster_blueprint_instance.json http://c6401.ambari.apache.org:8080/api/v1/clusters/cl1
3.3 Checked the status of the task until it is completed.
3.4 Verified in the UI that Admin/repositories page shows up the latest repo url (specified in repoinfo.xml) for the stack selected.
Thanks,
Cabir Zounaidou