[jira] [Commented] (HBASE-14511) StoreFile.Writer Meta Plugin

["Enis Soztutar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15081173#comment-15081173 ] 

Enis Soztutar commented on HBASE-14511:
---------------------------------------

bq. Enis Soztutar, I would prefer not to put this into the coprocessor. Coprocessor API is already overcrowded. If I give you table, region and table-level plugins will it suffice?
I was suggesting coprocessors since they already have the required features that we can piggy-back (per-table definition, class loading, Context objects, etc). If we can achieve these with a plugin framework, it is fine with me. I think we need at least per-table loading of plugins and context objects and maybe a reference to the Region as well. 

bq. Actually, the plugin has already access to column family configuration. The config object being passed to plugin comes from Store (which has all column families overrides).
It is not the configuration, but the plugin has to have a reference to the TableDescriptor at least if not more. I would imagine, having a reference to the region and store might also be useful. 

> StoreFile.Writer Meta Plugin
> ----------------------------
>
>                 Key: HBASE-14511
>                 URL: https://issues.apache.org/jira/browse/HBASE-14511
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>         Attachments: HBASE-14511-v3.patch, HBASE-14511-v4.patch, HBASE-14511.v1.patch, HBASE-14511.v2.patch
>
>
> During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had to modify the existing code of a StoreFile.Writer to add additional meta-info required by these new  policies. I think that it should be done by means of a new Plugin framework, because this seems to be a general capability/feature. 
> As a future enhancement this can become a part of a more general StoreFileWriter/Reader plugin architecture. But I need only Meta section of a store file.
> This could be used, for example, to collect rowkeys distribution information during hfile creation. This info can be used later to find the optimal region split key or to create optimal set of sub-regions for M/R jobs or other jobs which can operate on a sub-region level.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)