[jira] [Updated] (HBASE-21641) RegionServer Group based WAL creation & replication

["Nikhil Bafna (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-21641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nikhil Bafna updated HBASE-21641:
---------------------------------
    Description: 
I'm looking at extending HBASE-6721 to apply it to WALs such that WALs are created & replicated within an RSGroup. This extends multi-tenancy to WALs also, and not just cover Hbase data. I was working out of 1.2.x code. The feedback on mailing list was to continue the discussion on Jira and target 2.x
  
 The approach I'm using is
 - Strategy interface for WAL placement on the filesystem. Default to delegate it to respective filesystem (which is the old behavior). FavoredNode strategy computes the favoured nodes from the RSGroup memberships.
 - FavoredNode strategy requires instance of hbase.Server, to get the current server name and an instance of Zookeeper watch to listen for changes to RSGroup memberships
 - The strategy is initialised in HRegionServer init and set in a static field in DefaultWALProvider
 - DefaultWALProvider.Writer takes the strategy in its init, and invokes it before output stream creation and passes the favored nodes information to DistributedFileSystem.create()
  
 Few questions
 - Any glaring miss in the approach?
 - I have hesitation in setting the strategy in static field in DefaultWALProvider. I would have preferred it to be passed in "init" itself, but that change seems to be too expansive.
 - Also, this introduces the dependency of server/zookeeper instance inside the WAL code path, which seems to be not there till now. Is that an explicit choice to keep them separate?

  was:
I'm looking at extending HBASE-6721 to apply it to WALs such that WALs are created & replicated within an RSGroup. This extends multi-tenancy to WALs also, and not just cover Hbase data. I was working out of 1.2.x code. The feedback on mailing list was to continue the discussion on Jira and target 2.x
 
The approach I'm using is
- Strategy interface for WAL placement on the filesystem. Default to delegate it to respective filesystem (which is the old behavior). FavoredNode strategy computes the favoured nodes from the RSGroup memberships.
- FavoredNode strategy requires instance of hbase.Server, to get the current server name and an instance of Zookeeper watch to listen for changes to RSGroup memberships
- The strategy is initialised in HRegionServer init and set in a static field in DefaultWALProvider
- DefaultWALProvider.Writer takes the strategy in its init, and invokes it before output stream creation and passes the favored nodes information to DistributedFileSystem.create()
 
Few questions
- Any glaring miss in the approach?
- I have hesitation in setting the strategy in static field in DefaultWALProvider. I would have preferred it to be passed in "init" itself, but that change seems to be too expansive.
- Also, this introduces the dependency of server/zookeeper instance inside the WAL code path, which seems to be not there till now. Is that an explicit choice to keep them separate?
- If it seems like a useful change, should I open a JIRA and add patches and seek feedback there?


> RegionServer Group based WAL creation & replication
> ---------------------------------------------------
>
>                 Key: HBASE-21641
>                 URL: https://issues.apache.org/jira/browse/HBASE-21641
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>            Reporter: Nikhil Bafna
>            Priority: Minor
>
> I'm looking at extending HBASE-6721 to apply it to WALs such that WALs are created & replicated within an RSGroup. This extends multi-tenancy to WALs also, and not just cover Hbase data. I was working out of 1.2.x code. The feedback on mailing list was to continue the discussion on Jira and target 2.x
>   
>  The approach I'm using is
>  - Strategy interface for WAL placement on the filesystem. Default to delegate it to respective filesystem (which is the old behavior). FavoredNode strategy computes the favoured nodes from the RSGroup memberships.
>  - FavoredNode strategy requires instance of hbase.Server, to get the current server name and an instance of Zookeeper watch to listen for changes to RSGroup memberships
>  - The strategy is initialised in HRegionServer init and set in a static field in DefaultWALProvider
>  - DefaultWALProvider.Writer takes the strategy in its init, and invokes it before output stream creation and passes the favored nodes information to DistributedFileSystem.create()
>   
>  Few questions
>  - Any glaring miss in the approach?
>  - I have hesitation in setting the strategy in static field in DefaultWALProvider. I would have preferred it to be passed in "init" itself, but that change seems to be too expansive.
>  - Also, this introduces the dependency of server/zookeeper instance inside the WAL code path, which seems to be not there till now. Is that an explicit choice to keep them separate?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)