You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Matteo Bertozzi (JIRA)" <ji...@apache.org> on 2012/10/13 22:20:02 UTC
[jira] [Created] (HBASE-6992) Coprocessors semantic issues: post
async operations, helper methods, ...
Matteo Bertozzi created HBASE-6992:
--------------------------------------
Summary: Coprocessors semantic issues: post async operations, helper methods, ...
Key: HBASE-6992
URL: https://issues.apache.org/jira/browse/HBASE-6992
Project: HBase
Issue Type: Brainstorming
Components: Coprocessors
Affects Versions: 0.94.2, 0.92.2, 0.96.0
Reporter: Matteo Bertozzi
Assignee: Matteo Bertozzi
Discussion ticket around coprocessor pre/post semantic.
For each rpc in HMaster we have a pre/post operation that allows a coprocessor
to execute some code before and after the operation
* preOperation()
* my operation
* postOperation()
This is used for example by the AccessController to verify if the user can execute or not the operation.
Everything is fine, unless the master operation is asynchronous (like create/delete table)
* preOperation()
* executor.submit(new OperationHandler())
* postOperation()
The pre operation is still fine, since is executed before the operation and need to throw exceptions to the client in case of failures...
The post operation, instead, is no longer post... is just post submit. And if someone subscribe to postCreateTable() the notification can arrive before the table creation.
To "solve" this problem, HBASE-5584 added pre/post handlers and now the situation looks like this:
{code}
client request client response
| |
+------+-- submit op --+----+--- (HMaster)
pre op post op
(executor) +---- handler ----+
pre handler post handler
{code}
Now, we've two types of pre/post operation and the semantical correct are preOperation() and postOperationHandler()
since the preOperation() needs to reply to the client (e.g AccessController NotAllowException) and the postOperatioHandler() is really post operation.
postOperation() is not post... and preOperationHandler() can't communicate with the client.
The AccessController coprocessor uses the postOperation() that is fine for the sync operation like addColumn(), deleteColumn()... but in case of failure of async operations like deleteTable() we've removed rights that we still need.
I think that we should get back just to the single pre/post operation but with the right semantic...
Other then the "when is executed" problem, we've also functions that can be described with other simpler functions
for example: modifyTable() is just a helper to avoid multiple addColumn()/deleteColumn() calls
but the problem here is that modifyTable() has its own pre/post operation() and if I've implemented the pre/post addColumn I don't get notified when I call modifyTable(). This is another problem in the access controller coprocessor
In this case I'm not sure what the best solution can be... but in this way, adding new helper methods means breaking the coprocessors, because they don't get notified even if something is changed...
Any idea, thoughts, ...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6992) Coprocessors semantic issues: post
async operations, helper methods, ...
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481666#comment-13481666 ]
Andrew Purtell commented on HBASE-6992:
---------------------------------------
The basic problem is we need pre/post on the RPC path for managing interactions with the client, and pre/post on the background async processing. There is no single semantic here. Perhaps the naming can be improved, though, to make this clearer or less confusing. Suggestions?
> Coprocessors semantic issues: post async operations, helper methods, ...
> ------------------------------------------------------------------------
>
> Key: HBASE-6992
> URL: https://issues.apache.org/jira/browse/HBASE-6992
> Project: HBase
> Issue Type: Brainstorming
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2, 0.96.0
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
>
> Discussion ticket around coprocessor pre/post semantic.
> For each rpc in HMaster we have a pre/post operation that allows a coprocessor
> to execute some code before and after the operation
> * preOperation()
> * my operation
> * postOperation()
> This is used for example by the AccessController to verify if the user can execute or not the operation.
> Everything is fine, unless the master operation is asynchronous (like create/delete table)
> * preOperation()
> * executor.submit(new OperationHandler())
> * postOperation()
> The pre operation is still fine, since is executed before the operation and need to throw exceptions to the client in case of failures...
> The post operation, instead, is no longer post... is just post submit. And if someone subscribe to postCreateTable() the notification can arrive before the table creation.
> To "solve" this problem, HBASE-5584 added pre/post handlers and now the situation looks like this:
> {code}
> client request client response
> | |
> +------+-- submit op --+----+--- (HMaster)
> pre op post op
> (executor) +---- handler ----+
> pre handler post handler
> {code}
> Now, we've two types of pre/post operation and the semantical correct are preOperation() and postOperationHandler()
> since the preOperation() needs to reply to the client (e.g AccessController NotAllowException) and the postOperatioHandler() is really post operation.
> postOperation() is not post... and preOperationHandler() can't communicate with the client.
> The AccessController coprocessor uses the postOperation() that is fine for the sync operation like addColumn(), deleteColumn()... but in case of failure of async operations like deleteTable() we've removed rights that we still need.
> I think that we should get back just to the single pre/post operation but with the right semantic...
> Other then the "when is executed" problem, we've also functions that can be described with other simpler functions
> for example: modifyTable() is just a helper to avoid multiple addColumn()/deleteColumn() calls
> but the problem here is that modifyTable() has its own pre/post operation() and if I've implemented the pre/post addColumn I don't get notified when I call modifyTable(). This is another problem in the access controller coprocessor
> In this case I'm not sure what the best solution can be... but in this way, adding new helper methods means breaking the coprocessors, because they don't get notified even if something is changed...
> Any idea, thoughts, ...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6992) Coprocessors semantic issues: post
async operations, helper methods, ...
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475788#comment-13475788 ]
ramkrishna.s.vasudevan commented on HBASE-6992:
-----------------------------------------------
[~mbertozzi]
As you noticed the reason for introducing the pre/postOperationHandlers as part of HBASE-5584 is first of all to have control on the async operations.
In our use case the create/delete coprocessors where used to create/delete another table with some modified HTD.
Generally any async operations we need to wait for the successful operation and that too when we have coprocessors we need to wait for the operation to be done on the coprocessor hook tii.
When we tried to see the exisiting impl the reason for having pre/Post operation hook for async operations and not waiting for the operation to get completed is to not to block the RPC threads that does the operation.
So we thought of having other hooks that is sync with the handlers so that the RPC threads need not wait.
The similar case is applicable for enable/disable table also. The new hooks now gives us the advantage of being sure that the postOpreationHandler hook will be exeucted only on success of the main operation.
bq.but in case of failure of async operations like deleteTable() we've removed rights that we still need.
But the above problem should be a problem even without the new hooks right?
bq.for example: modifyTable() is just a helper to avoid multiple addColumn()/deleteColumn() calls
but the problem here is that modifyTable() has its own pre/post operation()
This is again a general problem right?
> Coprocessors semantic issues: post async operations, helper methods, ...
> ------------------------------------------------------------------------
>
> Key: HBASE-6992
> URL: https://issues.apache.org/jira/browse/HBASE-6992
> Project: HBase
> Issue Type: Brainstorming
> Components: Coprocessors
> Affects Versions: 0.92.2, 0.94.2, 0.96.0
> Reporter: Matteo Bertozzi
> Assignee: Matteo Bertozzi
>
> Discussion ticket around coprocessor pre/post semantic.
> For each rpc in HMaster we have a pre/post operation that allows a coprocessor
> to execute some code before and after the operation
> * preOperation()
> * my operation
> * postOperation()
> This is used for example by the AccessController to verify if the user can execute or not the operation.
> Everything is fine, unless the master operation is asynchronous (like create/delete table)
> * preOperation()
> * executor.submit(new OperationHandler())
> * postOperation()
> The pre operation is still fine, since is executed before the operation and need to throw exceptions to the client in case of failures...
> The post operation, instead, is no longer post... is just post submit. And if someone subscribe to postCreateTable() the notification can arrive before the table creation.
> To "solve" this problem, HBASE-5584 added pre/post handlers and now the situation looks like this:
> {code}
> client request client response
> | |
> +------+-- submit op --+----+--- (HMaster)
> pre op post op
> (executor) +---- handler ----+
> pre handler post handler
> {code}
> Now, we've two types of pre/post operation and the semantical correct are preOperation() and postOperationHandler()
> since the preOperation() needs to reply to the client (e.g AccessController NotAllowException) and the postOperatioHandler() is really post operation.
> postOperation() is not post... and preOperationHandler() can't communicate with the client.
> The AccessController coprocessor uses the postOperation() that is fine for the sync operation like addColumn(), deleteColumn()... but in case of failure of async operations like deleteTable() we've removed rights that we still need.
> I think that we should get back just to the single pre/post operation but with the right semantic...
> Other then the "when is executed" problem, we've also functions that can be described with other simpler functions
> for example: modifyTable() is just a helper to avoid multiple addColumn()/deleteColumn() calls
> but the problem here is that modifyTable() has its own pre/post operation() and if I've implemented the pre/post addColumn I don't get notified when I call modifyTable(). This is another problem in the access controller coprocessor
> In this case I'm not sure what the best solution can be... but in this way, adding new helper methods means breaking the coprocessors, because they don't get notified even if something is changed...
> Any idea, thoughts, ...?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira