[jira] [Commented] (YARN-1734) RM should get the updated Configurations when it transits from Standby to Active

["Xuan Gong (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-1734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13904477#comment-13904477 ] 

Xuan Gong commented on YARN-1734:
---------------------------------

If FileSystemBasedConfigurationProvider is enabled, when RMs are initiated, they will read the latest Configuration files from remote file system. But It is very possible that the active RM refresh its configuration several times (update the configuration files several times), when  the failover happens, the next active RM which will transit from standby to active can not get the latest configurations.

My proposal:
1. When we call transitionToActive, we need to manually call all the refresh*s which can make sure that this active RM can update its configuration.
2. During this process, if there is any exceptions for these refresh*s, we will catch them and print out instead of bringing down the RM.
3. And we will return an inputStream from ConfigurationProvider. This may not be enough in future. I would like to create a new data structure named ConfigurationContext which can contain:
* String configurationName
* InputStream contentInputStream
* long lastModifiedTime

And we can add more information when we need, such as permission, user, groups, etc. And The configurationProvider can return ConfigurationContext instead of InputStream.


> RM should get the updated Configurations when it transits from Standby to Active
> --------------------------------------------------------------------------------
>
>                 Key: YARN-1734
>                 URL: https://issues.apache.org/jira/browse/YARN-1734
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>            Priority: Critical
>         Attachments: YARN-1734.1.patch
>
>
> Currently, we have ConfigurationProvider which can support LocalConfiguration, and FileSystemBasedConfiguration. When HA is enabled, and FileSystemBasedConfiguration is enabled, RM can not get the updated Configurations when it transits from Standby to Active



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)