You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Matt (JIRA)" <ji...@apache.org> on 2017/08/11 00:00:07 UTC
[jira] [Commented] (AMBARI-21364) DB consistency checker throws
errors for missing 'parquet-logging' and 'product-info' configs after
Ambari upgrade
[ https://issues.apache.org/jira/browse/AMBARI-21364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16122563#comment-16122563 ]
Matt commented on AMBARI-21364:
-------------------------------
I had a similar issue with a service I'm working on. When I upgrade to a new version(of the service) that has a new dictionaryName, the database startup checks fail because it doesn't find the new property in the current config version. I found this ticket, built Ambari branch-2.5, and added optional set to true for my new configFile. This got me passed the startup check but the service failed to start with the an error stating that it was unable to find my new parameter in the configurations dictionary!
I took a look at the PR and it appears like it appends optional properties to the configuration but I'm not sure what this is appending to. Does this append to the configuration dictionary? If not, how can one add to the configuration dictionary?
https://github.com/apache/ambari/blob/19d4200e719854b0d304e0d7be5e20f33cda46ff/ambari-server/src/main/java/org/apache/ambari/server/state/ClientConfigFileDefinition.java#L86
https://github.com/apache/ambari/commit/19d4200e719854b0d304e0d7be5e20f33cda46ff
> DB consistency checker throws errors for missing 'parquet-logging' and 'product-info' configs after Ambari upgrade
> ------------------------------------------------------------------------------------------------------------------
>
> Key: AMBARI-21364
> URL: https://issues.apache.org/jira/browse/AMBARI-21364
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.5.2
> Reporter: Vivek Sharma
> Assignee: Dmitry Lysnichenko
> Priority: Blocker
> Labels: upgrade
> Fix For: 2.5.2
>
> Attachments: AMBARI-21364.patch, AMBARI-21364.trunk.patch, AMBARI-21364.trunk.patch
>
>
> *STR*
> # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129
> # Upgrade Ambari to 2.5.2.0-74 (hash: fd30644590991deb41241454d6e9091ed7a38e92)
> # Run "ambari-server start"
> {code}
> root@ctr-e133-1493418528701-156570-01-000005:/hwqe/hadoopqe# ambari-server restart
> Using python /usr/bin/python
> Restarting ambari-server
> Waiting for server stop...
> Ambari Server stopped
> Ambari Server running with administrator privileges.
> Organizing resource files at /var/lib/ambari-server/resources...
> Ambari database consistency check started...
> Server PID at: /var/run/ambari-server/ambari-server.pid
> Server out at: /var/log/ambari-server/ambari-server.out
> Server log at: /var/log/ambari-server/ambari-server.log
> Waiting for server start..................
> DB configs consistency check failed. Run "ambari-server start --skip-database-check" to skip. You may try --auto-fix-database flag to attempt to fix issues automatically. If you use this "--skip-database-check" option, do not make any changes to your cluster topology or perform a cluster upgrade until you correct the database consistency issues. See /var/log/ambari-server/ambari-server-check-database.log for more details on the consistency issues.
> ERROR: Exiting with exit code -1.
> REASON: Ambari Server java process has stopped. Please check the logs for more information.
> {code}
> DB log: ambari-server-check-database.log
> {code}
> 2017-06-27 13:51:38,743 INFO - Executing query 'GET_SERVICES_WITH_CONFIGS'
> 2017-06-27 13:51:38,748 INFO - Comparing service configs from stack with configs that we got from db
> 2017-06-27 13:51:38,748 INFO - Getting services from metainfo
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / KAFKA
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / PIG
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / ZEPPELIN
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / LOGSEARCH
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / MAPREDUCE2
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / SLIDER
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / HIVE
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / TEZ
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / HBASE
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / OOZIE
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / FLUME
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / MAHOUT
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / HDFS
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / DRUID
> 2017-06-27 13:51:38,748 INFO - Processing HDP-2.6 / AMBARI_METRICS
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SPARK
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SMARTSENSE
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / AMBARI_INFRA
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / YARN
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / FALCON
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SPARK2
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / ZOOKEEPER
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / ATLAS
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / SQOOP
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / STORM
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / KNOX
> 2017-06-27 13:51:38,749 INFO - Processing HDP-2.6 / KERBEROS
> 2017-06-27 13:51:38,749 INFO - Comparing required service configs from stack with mapped service configs from db
> 2017-06-27 13:51:38,751 ERROR - Required config(s): product-info is(are) not available for service SMARTSENSE with service config version 2 in cluster cl1
> 2017-06-27 13:51:38,751 ERROR - Required config(s): parquet-logging is(are) not available for service HIVE with service config version 7 in cluster cl1
> 2017-06-27 13:51:38,751 INFO - Getting services which has mapped configs which are not selected in clusterconfigmapping
> 2017-06-27 13:51:38,755 INFO - Checking Topology tables
> 2017-06-27 13:51:38,757 INFO - Checking for tables with large physical size
> 2017-06-27 13:51:38,759 INFO - The database table host_role_command is currently 16.703 MB and is within normal limits (3000.000)
> 2017-06-27 13:51:38,760 INFO - The database table execution_command is currently 12.516 MB and is within normal limits (3000.000)
> 2017-06-27 13:51:38,761 INFO - The database table stage is currently 0.109 MB and is within normal limits (3000.000)
> 2017-06-27 13:51:38,761 INFO - The database table request is currently 0.078 MB and is within normal limits (3000.000)
> 2017-06-27 13:51:38,762 INFO - The database table alert_history is currently 0.563 MB and is within normal limits (3000.000)
> 2017-06-27 13:51:38,762 INFO - ******************************* Check database completed *******************************
> {code}
> Looks like the issue is because the DB query does not return 'parquet-logging' and 'product-info' in type_name, while they exist in the respective metainfo files. See attached SQL query output and below:
> {code}
> root@ctr-e133-1493418528701-156570-01-000005:/var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0# cat /var/lib/ambari-server/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml | grep -C2 parquet
> <configFile>
> <type>env</type>
> <fileName>parquet-logging.properties</fileName>
> <dictionaryName>parquet-logging</dictionaryName>
> </configFile>
> </configFiles>
> --
> <config-type>webhcat-site</config-type>
> <config-type>webhcat-env</config-type>
> <config-type>parquet-logging</config-type>
> <config-type>ranger-hive-plugin-properties</config-type>
> <config-type>ranger-hive-audit</config-type>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)