You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ambari.apache.org by sw...@apache.org on 2014/02/13 21:49:33 UTC
[2/2] git commit: AMBARI-4665. Update Ambari API docs to add Config
Groups and Request Scehdule resources. (swagle)
AMBARI-4665. Update Ambari API docs to add Config Groups and Request Scehdule resources. (swagle)
Project: http://git-wip-us.apache.org/repos/asf/ambari/repo
Commit: http://git-wip-us.apache.org/repos/asf/ambari/commit/613be7a6
Tree: http://git-wip-us.apache.org/repos/asf/ambari/tree/613be7a6
Diff: http://git-wip-us.apache.org/repos/asf/ambari/diff/613be7a6
Branch: refs/heads/trunk
Commit: 613be7a656e043c7781b380be0a13b0aae24ae3e
Parents: d3ea89d
Author: Siddharth Wagle <sw...@hortonworks.com>
Authored: Thu Feb 13 12:32:12 2014 -0800
Committer: Siddharth Wagle <sw...@hortonworks.com>
Committed: Thu Feb 13 12:48:09 2014 -0800
----------------------------------------------------------------------
ambari-server/docs/api/v1/config-groups.md | 142 ++++++++++++
ambari-server/docs/api/v1/configuration.md | 58 ++---
ambari-server/docs/api/v1/host-resources.md | 2 +-
ambari-server/docs/api/v1/index.md | 13 +-
ambari-server/docs/api/v1/request-schedules.md | 235 ++++++++++++++++++++
5 files changed, 410 insertions(+), 40 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/ambari/blob/613be7a6/ambari-server/docs/api/v1/config-groups.md
----------------------------------------------------------------------
diff --git a/ambari-server/docs/api/v1/config-groups.md b/ambari-server/docs/api/v1/config-groups.md
new file mode 100644
index 0000000..d5e2e76
--- /dev/null
+++ b/ambari-server/docs/api/v1/config-groups.md
@@ -0,0 +1,142 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Configuration Group
+A Configuration group or Config group is type of resource that supports grouping of configuration resources and host resources for a service.
+Host is identified using the registered hostname of the host and the configuration resource is identified by the type and tag.
+
+Example list of all config groups defined for a cluster:
+
+ GET /api/v1/clusters/c1/config_groups
+ {
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/config_groups",
+ "items" : [
+ {
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/config_groups/2",
+ "ConfigGroup" : {
+ "cluster_name" : "c1",
+ "id" : 2
+ }
+ }
+ ]
+ }
+
+### Intuition
+Hosts grouped together based on certain similar characteristics will essentially share the same configuration for a Service and its components deployed on those hosts; consequently, different configuration for a different service.
+
+## Management
+Get call to view details of a particular config group identified by an :id
+
+ GET /api/v1/clusters/c1/config_groups/:id
+ {
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/config_groups/:id",
+ "ConfigGroup" : {
+ "cluster_name" : "c1",
+ "description" : "Next Gen Slaves Group",
+ "desired_configs" : [
+ {
+ "tag" : "nextgen1",
+ "type" : "core-site",
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/configurations?type=core-site&tag=nextgen1"
+ }
+ ],
+ "group_name" : "Test Datanode group",
+ "hosts" : [
+ {
+ "host_name" : "host1",
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/hosts/host1"
+ }
+ ],
+ "id" : :id,
+ "tag" : "HDFS"
+ }
+ }
+
+ 200 OK
+
+To create a new config group, it is mandatory to provide a config group name, a tag and the cluster name to which it belongs.
+The tag as seen in this example is a name of the service. Two config groups with the same tag cannot be associated with the same host.
+
+ POST /api/v1/clusters/c1/config_groups
+ [
+ {
+ "ConfigGroup": {
+ "cluster_name": "c1",
+ "group_name": "hdfs-nextgenslaves",
+ "tag": "HDFS",
+ "description": "HDFS configs for rack added on May 19, 2010",
+ "hosts": [
+ {
+ "host_name": "host1"
+ }
+ ],
+ "desired_configs": [
+ {
+ "type": "core-site",
+ "tag": "nextgen1",
+ "properties": {
+ "key": "value"
+ }
+ }
+ ]
+ }
+ }
+ ]
+
+ 201 Created
+
+Notice that the desired_configs specifies the property set for the configuration resource. In this particular case, a new configuration will be created with the type and tag specified and associated with the resulting Config group if the configuration does not exist.
+If a configuration with given type and tag exists, the property set would be ignored since a configuration is immutable.
+
+An update on a Config group expects the entire resource definition to be sent with the PUT request.
+
+ PUT /api/v1/clusters/c1/config_groups/2
+ [
+ {
+ "ConfigGroup": {
+ "cluster_name": "c1",
+ "group_name": "hdfs-nextgenslaves",
+ "tag": "HDFS",
+ "description": "HDFS configs for rack added on May 19, 2010",
+ "hosts": [
+ {
+ "host_name": "host1",
+ "host_name": "host2",
+ "host_name": "host3"
+ }
+ ],
+ "desired_configs": [
+ {
+ "type": "core-site",
+ "tag": "nextgen1"
+ },
+ {
+ "type": "hdfs-site",
+ "tag": "nextgen1"
+ }
+ ]
+ }
+ }
+ ]
+
+ 202 Accepted
+
+A Config group identified by the :id can be deleted.
+
+ DELETE /api/v1/clusters/c1/config_groups/:id
+
+ 200 OK
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/ambari/blob/613be7a6/ambari-server/docs/api/v1/configuration.md
----------------------------------------------------------------------
diff --git a/ambari-server/docs/api/v1/configuration.md b/ambari-server/docs/api/v1/configuration.md
index 886a2bc..e3c3be6 100644
--- a/ambari-server/docs/api/v1/configuration.md
+++ b/ambari-server/docs/api/v1/configuration.md
@@ -75,22 +75,25 @@ To create a new configuration, the below call can be made. Creating a configurat
}
-## Overrides
-Configurations are used by individual components on a host. The design is to define and apply configuration at the cluster level, with optional overrides per host. The order of application is as follows:
-
-1. Set a configuration to the cluster.
-3. Set optional configuration overrides to a host.
-
-The optional host overrides contain only the properties which need to be changed on a host
-from the cluster level configuration. Be aware you cannot create ONLY an override and expect
-it to be applied. There must always be a cluster-level configuration of the same type.
-
## Desired Configuration
Desired configurations is the set of `tag`s that defines what you desire the cluster to have. It
is possible to save configurations, even if they are not applied. You can define `tag`s having
the same type, but tag must be unique and only one of them is active for the cluster at any
time.
+## Overrides
+Overrides are enabled by the Config Group resource. (see [Config Groups](config-groups.md))
+Configurations are used by individual components on a host. The design is to define and apply configuration at the cluster level, with optional overrides per host. The order of application is as follows:
+
+For a given host:
+1. Get all desired configurations defined for the cluster.
+2. Find all config groups for the host.
+3. Apply the config group overrides on top of the cluster configurations.
+
+The optional config group overrides contain only the properties which need to be changed on a host from the cluster level configuration.
+There must always be a cluster-level configuration of the same type.
+
+
### Cluster
There are two ways to apply a configuration to a cluster.
@@ -138,38 +141,12 @@ The list of all desired configs is reported in a GET call:
"tag" : "version1"
},
"core-site": {
- "tag": "version1",
- "host_overrides": [
- {
- "host_name": "h0",
- "tag": "version2"
- },
- {
- "host_name": "h1",
- "tag": "version3"
- }
- ]
+ "tag": "version1"
},
/* etc */
}
}
}
-Notice that a configuration type that is overridden shows the host name and override tag.
-
-### Host
-Host overrides are set the same way as clusters:
-
- PUT /api/v1/clusters/c1/hosts/h0
- {
- "Hosts": {
- "desired_config": {
- "type": "core-site",
- "tag": "version2",
- "properties": { "a": "b" }
- }
- }
- }
-Again, if you supply properties, the configuration is created, then assigned. Otherwise, the configuration pair is simply assigned.
To list the desired configs for a host:
@@ -180,13 +157,18 @@ To list the desired configs for a host:
"host_name" : "h0",
"desired_configs" : {
"global" : {
- "tag" : "version3"
+ "overrides" : {
+ "2" : "version10"
+ },
+ "default" : "version1"
},
/* etc */
}
}
}
+Notice overrides for a configuration type are listed with the key as the config group id and value is the tag identifying the configuration resource.
+
##Actual Configuration
Actual configuration represents the set of `tag`s that identify the cluster's current configuration. When configurations are changed, they are saved into the backing database, even if the host has not yet received the change. When a
host receives the desired configuration changes AND applies them, it will respond with the applied tags. This is called the actual configuration.
http://git-wip-us.apache.org/repos/asf/ambari/blob/613be7a6/ambari-server/docs/api/v1/host-resources.md
----------------------------------------------------------------------
diff --git a/ambari-server/docs/api/v1/host-resources.md b/ambari-server/docs/api/v1/host-resources.md
index e776bca..898aeb9 100644
--- a/ambari-server/docs/api/v1/host-resources.md
+++ b/ambari-server/docs/api/v1/host-resources.md
@@ -77,7 +77,7 @@ limitations under the License.
</tr>
<tr>
<td>Hosts/disk_info</td>
- <td>The host disk infromation</td>
+ <td>The host disk information</td>
</tr>
<tr>
<td>Hosts/host_status</td>
http://git-wip-us.apache.org/repos/asf/ambari/blob/613be7a6/ambari-server/docs/api/v1/index.md
----------------------------------------------------------------------
diff --git a/ambari-server/docs/api/v1/index.md b/ambari-server/docs/api/v1/index.md
index 8157ae7..38b046e 100644
--- a/ambari-server/docs/api/v1/index.md
+++ b/ambari-server/docs/api/v1/index.md
@@ -295,6 +295,11 @@ Configuration resources are sets of key/value pairs that configure the services
[Configuration Resource Overview](configuration.md)
+#### config_groups
+Config group is type of resource that supports grouping of configuration resources and host resources for a service.
+
+[Config Group Overview](config-groups.md)
+
#### requests
Request resources are groups of tasks that were created to carry out an instruction.
@@ -305,6 +310,11 @@ Task resources are the individual tasks that make up a request resource.
[Task Resources](task-resources.md)
+#### request_schedules
+A request schedule defines a batch of requests to be executed based on a schedule.
+
+[Request Schedule Resources](request-schedules.md)
+
#### workflows
Workflow resources are DAGs of MapReduce jobs in a Hadoop cluster.
@@ -1078,7 +1088,8 @@ The "end" keyword indicates the end of the set of resources and is equivalent to
/api/v1/clusters/cl1/requests?to=-1&page_size=10
-The default ordering of the resources (by the natural ordering of the resource key properties) is implied.
+The default ordering of the resources (by the natural ordering of the resource key properties) is implied.
+
HTTP Return Codes
----
http://git-wip-us.apache.org/repos/asf/ambari/blob/613be7a6/ambari-server/docs/api/v1/request-schedules.md
----------------------------------------------------------------------
diff --git a/ambari-server/docs/api/v1/request-schedules.md b/ambari-server/docs/api/v1/request-schedules.md
new file mode 100644
index 0000000..c42fce0
--- /dev/null
+++ b/ambari-server/docs/api/v1/request-schedules.md
@@ -0,0 +1,235 @@
+<!---
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Request Schedule
+A request schedule resource identifies a batch of request sub resources to be executed according to a schedule or a point in time execution if no schedule is specified. The schedule determines the start time for the batch of requests.
+A batch request is a sub-resource of the request schedule and represents the Http request that will be executed on the server. It captures the request properties along with the status, return code and an optional request id for long running/asynchronous operations.
+
+Example list of all request schedules defined for a cluster:
+
+ GET /api/v1/clusters/c1/request_schedules
+ {
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/request_schedules",
+ "items" : [
+ {
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/request_schedules/2",
+ "RequestSchedule" : {
+ "cluster_name" : "c1",
+ "id" : 2
+ }
+ }
+ ]
+ }
+
+All batch requests are executed linearly ordered by the order_id field and if a batch request fails, subsequent batch requests will no longer be executed.
+
+Example to create a request schedule with batch requests to perform Rolling restarts for a set of 3 hosts at a time
+
+ POST /api/v1/clusters/c1/request_schedules
+
+ [
+ {
+ "RequestSchedule": {
+ "batch": [
+ {
+ "requests": [
+ {
+ "order_id": 1,
+ "request_type": "POST",
+ "request_uri": "/api/v1/clusters/c1/requests",
+ "RequestBodyInfo": {
+ "RequestInfo": {
+ "context": "ROLLING-RESTART NODEMANAGERs batch 1 of 2",
+ "command": "RESTART",
+ "service_name": "YARN",
+ "component_name": "NODEMANAGER",
+ "hosts": "host1,host2,host3"
+ }
+ }
+ },
+ {
+ "order_id": 2,
+ "request_type": "POST",
+ "request_uri": "/api/v1/clusters/c1/requests",
+ "RequestBodyInfo": {
+ "RequestInfo": {
+ "context": "ROLLING-RESTART NODEMANAGERs batch 2 of 2",
+ "command": "RESTART",
+ "service_name": "YARN",
+ "component_name": "NODEMANAGER",
+ "hosts": "host4,host5,host6"
+ }
+ }
+ }
+ ]
+ },
+ {
+ "batch_settings": {
+ "batch_separation_in_seconds": 120,
+ "task_failure_tolerance": 1
+ }
+ }
+ ]
+ }
+ }
+ ]
+
+ 201 Created
+
+The batch_settings allow for specifying how much time in seconds should be the separation between two requests and a failure tolerance count which identifies the total number of failed tasks, if at all, resulting from the batch requests are acceptable before the batch execution is aborted.
+
+Example to create a recurring request schedule.
+
+ POST /api/v1/clusters/c1/request_schedules
+
+ [
+ {
+ "RequestSchedule": {
+ "batch": [
+ {
+ "requests": [
+ {
+ "order_id": 1,
+ "request_type": "POST",
+ "request_uri": "/api/v1/clusters/c1/requests",
+ "RequestBodyInfo": {
+ "RequestInfo": {
+ "context": "ROLLING-RESTART NODEMANAGERs batch 1 of 1",
+ "command": "RESTART",
+ "service_name": "YARN",
+ "component_name": "NODEMANAGER",
+ "hosts": "host1,host2,host3"
+ }
+ }
+ }
+ ]
+ },
+ {
+ "batch_settings": {
+ "batch_separation_in_minutes": "15",
+ "task_failure_tolerance": "3"
+ }
+ }
+ ],
+ "schedule": {
+ "minutes": "30",
+ "hours": "0",
+ "days_of_month": "*",
+ "month": "*",
+ "day_of_week": "?",
+ "startTime": "",
+ "endTime": "2013-11-18T14:29:29-0800"
+ }
+ }
+ }
+ ]
+
+ 201 Created
+
+**Schedule specification**
+
+The schedule is a Quartz scheduler cron expression minus the seconds specifier.
+
+<table>
+ <tr>
+ <td>Field</td>
+ <td>Allowed values</td>
+ <td>Allowed special characters</td>
+ </tr>
+ <tr>
+ <td>Minutes</td>
+ <td>0-59</td>
+ <td>, - * /</td>
+ </tr>
+ <tr>
+ <td>Hours</td>
+ <td>0-23</td>
+ <td>, - * /</td>
+ </tr>
+ <tr>
+ <td>Day of month</td>
+ <td>1-31</td>
+ <td>, - * / L W</td>
+ </tr>
+ <tr>
+ <td>Month</td>
+ <td>1-12 or JAN-DEC</td>
+ <td>, - * /</td>
+ </tr>
+ <tr>
+ <td>Day of week</td>
+ <td>1-7 or SUN-SAT</td>
+ <td>, - * ? / L #</td>
+ </tr>
+ <tr>
+ <td>Year</td>
+ <td>empty, 1970-2099</td>
+ <td>, - * /</td>
+ </tr>
+</table>
+
+**Note**: A Request schedule is immutable, update on the resource is not allowed.
+
+An example of GET call on a request schedule identified by the :id that has completed execution.
+
+ GET /api/v1/clusters/c1/request_schedules/:id
+ {
+ "href" : "http://your.ambari.server/api/v1/clusters/c1/request_schedules/:id",
+ "RequestSchedule" : {
+ "batch" : {
+ "batch_requests" : [
+ {
+ "order_id" : 1,
+ "request_type" : "POST",
+ "request_uri" : "/api/v1/clusters/c1/requests",
+ "request_body" : {
+ "RequestInfo": {
+ "context": "ROLLING-RESTART NODEMANAGERs batch 1 of 1",
+ "command": "RESTART",
+ "service_name": "HDFS",
+ "component_name": "DATANODE",
+ "hosts": "host1,host2,host3"
+ }
+ }
+ }
+ ],
+ "batch_settings" : {
+ "batch_separation_in_seconds" : 10,
+ "task_failure_tolerance_limit" : 1
+ }
+ },
+ "cluster_name" : "c1",
+ "description" : null,
+ "id" : :id,
+ "last_execution_status" : "COMPLETED",
+ "schedule" : { },
+ "status" : "COMPLETED"
+ }
+ }
+
+ 200 OK
+
+A delete on request schedule results in marking the request schedule as DISABLED and suspends all future executions of the batch requests minus the currently executing batch request.
+
+ DELETE /api/v1/clusters/c1/request_schedules/:id
+
+ 200 OK
+
+
+
+
+