You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Todd Lipcon (Created) (JIRA)" <ji...@apache.org> on 2012/02/22 07:48:54 UTC
[jira] [Created] (HBASE-5447) Support for custom filters with
PB-based RPC
Support for custom filters with PB-based RPC
--------------------------------------------
Key: HBASE-5447
URL: https://issues.apache.org/jira/browse/HBASE-5447
Project: HBase
Issue Type: Sub-task
Reporter: Todd Lipcon
Assignee: Todd Lipcon
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "Gregory Chanan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418055#comment-13418055 ]
Gregory Chanan commented on HBASE-5447:
---------------------------------------
I should note -- there are some filters that allow combination of other filters, e.g. boolean logic ands/ors between filters. I'd need to think about if we could get that to work with a hybrid scheme.
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-5447.
--------------------------
Resolution: Fixed
Fix Version/s: 0.96.0
Assignee: Gregory Chanan (was: Todd Lipcon)
Hadoop Flags: Reviewed
Closing. Assigned Gregory.
Regards custom filters, let them come out of the woodwork. We'll help them make the convertion to pb.
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Gregory Chanan
> Fix For: 0.96.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418059#comment-13418059 ]
Zhihong Ted Yu commented on HBASE-5447:
---------------------------------------
bq. just derive from FilterBasePB (where they are required to implement the readFields and write methods, as today)
I guess there was a typo above: FilterBaseWritable should have been used.
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "Gregory Chanan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454666#comment-13454666 ]
Gregory Chanan commented on HBASE-5447:
---------------------------------------
Should we close this now that HBASE-6454 and HBASE-6477 are in? Or do we think there is more to do?
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422305#comment-13422305 ]
stack commented on HBASE-5447:
------------------------------
@Gregory I don't think there many user-made filters that are not part of core code base. Would go along w/ your option A above. We can help those w/ user-made filters do convertion to 0.96 when/if they show up trying to figure how to convert. Thanks.
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "Gregory Chanan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418052#comment-13418052 ]
Gregory Chanan commented on HBASE-5447:
---------------------------------------
Some thoughts on this issue...
First off: I'm not sure if this JIRA is referring to the built-in existing filters, or filters that end-users would write (or some combination of both). So, what does "custom" mean here? I'm going to refer to both built-in and new filters below -- I think we need to make a decision for each.
Today, we are using the Writable methods for each filter type in order to create the Filter on the other end of the rpc (together with the filter class name, this is enough to construct the filter). This work but means the filters are not extensible in a forward-compatible way. Options:
1) Require a protobuf definition for each filter class
This would allow us to make each filter forward-compatible in the manner that all other rpcs are done. I found 16 built-in filter classes where this would need to be done, so I don't think it's a huge undertaking. I'm even offering to do it :). I think this is the best option for the built-in filters: most users probably don't write their own filters and we might as well make them extensible.
The downside is that for new filters that end-users would write, we would require them to install pb, write pb definitions, and write java conversions (convert pb filter to Filter and visa-versa) which makes writing filters for users more difficult -- they may not even care about forward compatibility.
2) Keep the existing Writable interface (or use something similar)
This would require no work, but doesn't give us forward compatibility.
We could do also a hybrid scheme, where users have the option to either create PB-compatible filters or Writable filters (e.g. we could have something like FilterBaseWritable and FilterBasePB), where all our builit-in filters derive from FilterBasePB and end-users who don't want to mess with PB can just derive from FilterBasePB (where they are required to implement the readFields and write methods, as today). The downside here, of course, is more code to maintain and test for both of the filter base classes.
So, to recap, the questions I think we need to answer are:
A) What should be done about the built-in filters (I suggest converting fully to PB)
B) What should be done about new filters that end-users write
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "Gregory Chanan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418522#comment-13418522 ]
Gregory Chanan commented on HBASE-5447:
---------------------------------------
bq. I guess there was a typo above: FilterBaseWritable should have been used.
Correct, thanks for pointing that out.
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418418#comment-13418418 ]
Jimmy Xiang commented on HBASE-5447:
------------------------------------
+1 on fully converting to PB.
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5447) Support for custom filters with
PB-based RPC
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418599#comment-13418599 ]
Zhihong Ted Yu commented on HBASE-5447:
---------------------------------------
Maybe using sub-tasks, we should provide some tools which help users write / validate custom PB-based filters.
> Support for custom filters with PB-based RPC
> --------------------------------------------
>
> Key: HBASE-5447
> URL: https://issues.apache.org/jira/browse/HBASE-5447
> Project: HBase
> Issue Type: Sub-task
> Components: ipc, master, migration, regionserver
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira