[jira] [Updated] (AMBARI-25569) Reassess Ambari Metrics data migration

["Tamas Payer (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AMBARI-25569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tamas Payer updated AMBARI-25569:
---------------------------------
    Labels: metric-collector migration pull-request-available  (was: metric-collector migration)

> Reassess Ambari Metrics data migration
> --------------------------------------
>
>                 Key: AMBARI-25569
>                 URL: https://issues.apache.org/jira/browse/AMBARI-25569
>             Project: Ambari
>          Issue Type: Task
>          Components: ambari-metrics
>    Affects Versions: 2.7.3, 2.7.4, 2.7.5
>            Reporter: Tamas Payer
>            Assignee: Tamas Payer
>            Priority: Major
>              Labels: metric-collector, migration, pull-request-available
>             Fix For: 2.7.6
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The data migration process of Ambari Metrics as described at [https://docs.cloudera.com/HDPDocuments/Ambari-2.7.5.0/bk_ambari-upgrade-major/content/upgrading_HDP_post_upgrade_tasks.html]
> is causing issues, like not migrating data that would be expected by the user. (e.g. Yarn Queue metrics other than the root queue's.)
> The data migration is usually called by the
>  
> {code:java}
> /usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ upgrade_start /etc/ambari-metrics-collector/conf/metrics_whitelist
> {code}
> command where the whitelist is specified.
> The migration code only looks for the metrics that are present in the whitelist file. This is true even in the case when the AMS Whitelisting is not enabled. The user will only have those metrics migrated that are present in the whitelist file, which is usually not all that are required.
>  
> I suggest the following change:
> - If whitelist file parameter i*s provided* then
>     - migrate only the metrics that are in the whitelist file
> - if whitelist file parameter is *not provided* then
>     - if whitelisting is *enabled* then
>         - discover the whitelist file configured in AMS and
>         migrate only the metrics that are in the whitelist file
>     - if whitelisting is *disabled* then
>         - migrate *all the metrics* present in the database
>  
> Furthermore, the migration process frequently dies silently while saving the metadata. The log message "Saving metadata to store..." is present in the logs but the "Metadata was saved." is mostly never there, but there are no other error messages.
> I suggest revising the current solution where the saving of the metadata is triggered in a Shutdown hook.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)