[jira] [Commented] (YARN-5947) Create DerbySchedulerConfigurationStore class using Derby as backing store

["Jonathan Hung (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16040148#comment-16040148 ] 

Jonathan Hung commented on YARN-5947:
-------------------------------------

Attached YARN-5947-YARN-5734.001.patch for an initial pass at the LevelDB implementation.

It stores three things:
# confKey -> confVal
# "committedTxn" -> last good txn id stored in the db
# log.x -> LogMutation for txn x, for historical purposes / recovery

Currently it relies on the fact that MutableCSConfigurationProvider has a synchronized method which does the logMutation and the confirmMutation, so that the logs cannot be logged/confirmed out of order.

This patch also handles recovery of configuration, e.g. if a config is logged and the RM dies, it will replay this log (and store it if the subsequent scheduler reinitialize succeeds).

Also there are no unit tests. Will add that.

> Create DerbySchedulerConfigurationStore class using Derby as backing store
> --------------------------------------------------------------------------
>
>                 Key: YARN-5947
>                 URL: https://issues.apache.org/jira/browse/YARN-5947
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Jonathan Hung
>            Assignee: Jonathan Hung
>         Attachments: YARN-5947.001.patch, YARN-5947-YARN-5734.001.patch
>
>
> DerbySchedulerConfigurationStore will extend YarnConfigurationStore for storing scheduler configuration in a Derby embedded DB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org