[jira] [Commented] (AMBARI-19130) Downgrade Can Create Multiple Mappings For Latest Configs

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMBARI-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729855#comment-15729855 ] 

Hadoop QA commented on AMBARI-19130:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12842189/AMBARI-19130.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 2 new or modified test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of javac compiler warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in ambari-server:

                  org.apache.ambari.server.controller.metrics.JMXPropertyProviderTest

Test results: https://builds.apache.org/job/Ambari-trunk-test-patch/9580//testReport/
Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/9580//console

This message is automatically generated.

> Downgrade Can Create Multiple Mappings For Latest Configs
> ---------------------------------------------------------
>
>                 Key: AMBARI-19130
>                 URL: https://issues.apache.org/jira/browse/AMBARI-19130
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.2.0
>            Reporter: Jonathan Hurley
>            Assignee: Jonathan Hurley
>            Priority: Critical
>             Fix For: 2.5.0
>
>         Attachments: AMBARI-19130.patch
>
>
> During a downgrade which crosses major stack versions (such as from HDP 2.y to HDP 2.x), Ambari attempts to set the latest configurations from HDP 2.x as "current". However, after the downgrade succeeds, the database fails with the following consistency check:
> {code}
> ERROR - You have config(s), in cluster c1, that is(are) selected more than once in clusterconfigmapping table: ranger-storm-plugin-properties,mapred-env,webhcat-env,hive-exec-log4j,hive-log4j,webhcat-log4j,storm-env,yarn-log4j,hcat-env
> {code}
> The problem is actually not with {{clusterconfig}} but with the state of the {{clusterconfigmapping}} table _before_ upgrade. Apparently, it is possible to have multiple mappings for the same config. We match on the config type and tag (such as hdfs-site / version1234566789). There should never, EVER be duplicate mappings for the same config and version tag.
> The current downgrade code doesn't "break" after finding its first match:
> {code}
>       for(ClusterConfigMappingEntity configMappingEntity: configMappingEntities){
>         String type = configMappingEntity.getType();
>         String tag =  configMappingEntity.getTag();
>         for (ClusterConfigMappingEntity latest : latestConfigMappingByStack) {
>           String latestType = latest.getType();
>           String latestTag = latest.getTag();
>           // find the latest config of a given mapping entity
>           if (StringUtils.equals(type, latestType) && StringUtils.equals(tag, latestTag)) {
>             LOG.info("{} with version tag {} is selected for stack {}", type, tag, stackId.toString());
>             configMappingEntity.setSelected(1);
>           }
>         }
>       }
> {code}
> I'm not sure that breaking will work here since we don't know the ordering. We could order the results and then break on the first match. In any event, it's a problem. But it's only a problem for the latest version of a config since that's what we're going to try to make selected.
> Here's a query from a test cluster which shows that if this cluster were to be downgraded, 5 configs would have multiple mappings created. This is because for the latest config, there are multiple mappings in {{clusterconfigmapping}}.
> {noformat}
> SELECT
>   mapping.type_name,
>   mapping.version_tag,
>   COUNT(*)
> FROM clusterconfigmapping mapping
> JOIN (SELECT
>   config.type_name,
>   config.version_tag,
>   MAX(config.version) AS latest_version
> FROM clusterconfig config
> GROUP BY config.type_name) AS latestConfig
>   ON latestConfig.type_name = mapping.type_name
>   AND latestConfig.version_tag = mapping.version_tag
> GROUP BY
>     type_name, version_tag  
> HAVING COUNT(*) > 1
> +-------------------------------+----------------------+----------+
> | type_name                     | version_tag          | COUNT(*) |
> +-------------------------------+----------------------+----------+
> | hcat-env                      | version1             |        5 |
> | hive-exec-log4j               | version1             |        5 |
> | hive-log4j                    | version1             |        5 |
> | ranger-hive-plugin-properties | version1436918769763 |        3 |
> | webhcat-env                   | version1             |        5 |
> +-------------------------------+----------------------+----------+
> 5 rows in set (0.00 sec)
> {noformat}
> STR:
> # Install a cluster, and instrument the {{clusterconfigmapping}} table to have multiple mappings for a given type (with only 1 currently selected).
> # Upgrade and then downgrade 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)