[jira] [Created] (AMBARI-11209) Blueprints processor needs stronger validation of Blueprint JSON structure

["Erik Bergenholtz (JIRA)" <ji...@apache.org>]

Erik Bergenholtz created AMBARI-11209:
-----------------------------------------

             Summary: Blueprints processor needs stronger validation of Blueprint JSON structure
                 Key: AMBARI-11209
                 URL: https://issues.apache.org/jira/browse/AMBARI-11209
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.1.0
            Reporter: Erik Bergenholtz
            Assignee: Emil Anca
             Fix For: 2.1.0


The Blueprints processor needs to have stronger validation during the process of handling a POST for a newly-submitted Blueprint from a REST client.
The Blueprints structure requires that the "configurations" element be an array of Maps, with each Map representing a given configuration type ("core-site", "storm-site", etc).
Typically, the configurations structure should look like:
{code}
""configurations": [
{
"mapred-env": {
"properties":
{ "mapreduce_log_dir_prefix": "/grid/0/log/hadoop-mapreduce" }
}
},
{ "mapred-site": {
"properties":
{ "install-test-mapred-site": "install-test-mapred-site-VALUE" }
}
},
{ "yarn-env": {
"properties":
{ "yarn_log_dir_prefix": "/grid/0/log/hadoop" }
}
}
]"
The snippet above is just an example to illustrate the expected format.
Recent changes to the Blueprint's structure to handle "properties_attributes" (such as "final") have modified the structure of a Blueprint, so that configuration properties are now contained in a "properties" map, internal to each configuration type. Creating new Blueprints using this newer format (the older format is still acceptable to the processor) require that the configuration types are still enclosed in a map, even though an internal map is also present, which may be a source of confusion.
An example of an incorrect format would be:
""configurations": [
{
"mapred-env": {
"properties":
{ "mapreduce_log_dir_prefix": "/grid/0/log/hadoop-mapreduce" }
},
"mapred-site": {
"properties":
{ "install-test-mapred-site": "install-test-mapred-site-VALUE" }
},
"yarn-env": {
"properties":
{ "yarn_log_dir_prefix": "/grid/0/log/hadoop" }
}
]
{code}
Please note that in the example above a single map is created, with all configuration elements being added to that entry's configuration type. This causes a corrupted Blueprint to be stored and used for deployments, since all configuration elements added in the original Blueprint will be registered under one of the config types used in the original Blueprint. This will cause many types of cluster startup failures, as the configuration elements will be added to incorrect locations.

The underlying problem was that brackets were missing from the initial Blueprint, although this was not immediately obvious.
The JSON parser used for the Blueprint submission will accept the incorrect format, since it is valid JSON, but the Blueprint processor needs a higher level of validation, such that this error is caught immediately, before any cluster deployment is attempted.
When the Blueprint processor receives a POST submission with a new Blueprint that includes this incorrect format for "configurations", the processor must reject this submission, and send back a relevant error code/message, so that the user can resolve this problem prior to attempting a cluster deployment.
This issue will not affect the older format for Blueprints, and so any backwards-compatible Blueprints used should work fine. This issue will only occur when new Blueprints are submitted that use the newer syntax/format for defining configuration properties (with support for properties_attributes) as well.
This JIRA is being created to track this issue, as it could potentially be a user experience issue going forward, and should probably be resolved in Ambari 2.1, if possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)