You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "ramkrishna.s.vasudevan (Created) (JIRA)" <ji...@apache.org> on 2012/03/22 16:26:22 UTC
[jira] [Created] (HBASE-5617) Provide coprocessor hooks in put flow
while rollbackMemstore.
Provide coprocessor hooks in put flow while rollbackMemstore.
-------------------------------------------------------------
Key: HBASE-5617
URL: https://issues.apache.org/jira/browse/HBASE-5617
Project: HBase
Issue Type: Improvement
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.96.0
With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
puts that happened in the prePut.
We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239510#comment-13239510 ]
Hadoop QA commented on HBASE-5617:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520114/HBASE-5617_1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
org.apache.hadoop.hbase.master.TestSplitLogManager
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1321//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1321//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1321//console
This message is automatically generated.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241395#comment-13241395 ]
Zhihong Yu commented on HBASE-5617:
-----------------------------------
rollbackMemstore() doesn't throw exception.
I want to understand why we need two hooks for the rollback scenario.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241435#comment-13241435 ]
ramkrishna.s.vasudevan commented on HBASE-5617:
-----------------------------------------------
@Lars
I will update in other places also.
@Ted
What do you think? Only one hook is enough? For our use case one hook is enough.
>From your previous comment i felt you suggested like its better we have two.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Andrew Purtell (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239675#comment-13239675 ]
Andrew Purtell commented on HBASE-5617:
---------------------------------------
-1 on patch, API changes exceed the scope of this issue as defined.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5617) Provide coprocessor hooks in put flow
while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5617:
------------------------------------------
Attachment: HBASE-5617_2.patch
@Andy
This patch deals only with what is intended in the title of the JIRA.
Can i raise another JIRA for adding other hooks?
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Anoop Sam John (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240981#comment-13240981 ]
Anoop Sam John commented on HBASE-5617:
---------------------------------------
{quote}
Do we need two hooks for memstore rollback ?
{quote}
+1 for pre and post... As per the use case one can use pre or post hook...
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239460#comment-13239460 ]
ramkrishna.s.vasudevan commented on HBASE-5617:
-----------------------------------------------
Pls review the patch. This patch also adds additional hooks to the doMiniBatchPut.
We found some gaps in the coprocessor hooks available and hence added the new hooks.
For eg, the preBatchPut hook will help us to do any puts after the timestamp is updated in the kv and it will be called once with all the current puts that has acquired the locks.
The postBatchPut hook is needed before the completeMemStoreInsert so that any new puts we do using the hooks can also adhere to the principles of mvcc.
Similarly a final hook is also introduced so that any clean up operation can be done.
Rollback memstore is also needed with a hook.
Pls feel free to comment on the new hooks added.
If only rollback memstore hook is needed then i will prepare a patch with that alone.
This is just a version so that folks can get an idea on what has been added and where it has been added.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5617) Provide coprocessor hooks in put flow
while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5617:
------------------------------------------
Status: Patch Available (was: Open)
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Lars Hofhansl (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241379#comment-13241379 ]
Lars Hofhansl commented on HBASE-5617:
--------------------------------------
In 0.94+ there are more places where we rollback the memstore.
Check out HRegion.mutateRowsWithLocks in 0.94 and HRegion.processRowsWithLocks in 0.96.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241446#comment-13241446 ]
Zhihong Yu commented on HBASE-5617:
-----------------------------------
I think one hook should be enough.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5617) Provide coprocessor hooks in put flow
while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5617:
------------------------------------------
Status: Open (was: Patch Available)
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5617) Provide coprocessor hooks in put flow
while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5617:
------------------------------------------
Attachment: HBASE-5617_1.patch
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241043#comment-13241043 ]
ramkrishna.s.vasudevan commented on HBASE-5617:
-----------------------------------------------
@Stack
I will be adding test cases for the rollback scenarios. I wanted to get the things clarified before giving a patch.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242231#comment-13242231 ]
ramkrishna.s.vasudevan commented on HBASE-5617:
-----------------------------------------------
bq. Check out HRegion.mutateRowsWithLocks in 0.94 and HRegion.processRowsWithLocks in 0.96.
These apis roll back the kvs directly. In my patch i thought of having putList.
So can we now pass kvs?
Or can we pass List<Mutation> and leave the user to do a instanceof check and proceed?
Also is the name ok to have rollbackMemstore in it or only rollback? Kindly let me know your opinion on this.
Also in trunk the processor.postProcess is done even if there is a failure. But in 0.94 its not.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476065#comment-13476065 ]
ramkrishna.s.vasudevan commented on HBASE-5617:
-----------------------------------------------
I would like to revive this patch? Thougths...?
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242421#comment-13242421 ]
Zhihong Yu commented on HBASE-5617:
-----------------------------------
I think passing List<KeyValue> is general.
rollbackMemstore would be a good name. See javadoc for Store.rollback().
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239567#comment-13239567 ]
stack commented on HBASE-5617:
------------------------------
No test Ram that new functionality works?
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241399#comment-13241399 ]
Hadoop QA commented on HBASE-5617:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12520451/HBASE-5617_2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegion
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1343//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1343//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1343//console
This message is automatically generated.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch, HBASE-5617_2.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240982#comment-13240982 ]
ramkrishna.s.vasudevan commented on HBASE-5617:
-----------------------------------------------
@Andy
Yes, this patch adds some more hooks other than the one specified in the JIRA title.
My intention was to know if it is ok to add new hooks like the one in the patch.
If it is fine i can raise another JIRA for the same and leave this JIRA with only hooks for rollbackOfMemStore.
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5617) Provide coprocessor hooks in put flow
while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5617:
------------------------------------------
Component/s: coprocessors
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5617) Provide coprocessor hooks in put flow
while rollbackMemstore.
Posted by "ramkrishna.s.vasudevan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-5617:
------------------------------------------
Status: Patch Available (was: Open)
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5617) Provide coprocessor hooks in put
flow while rollbackMemstore.
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239537#comment-13239537 ]
Zhihong Yu commented on HBASE-5617:
-----------------------------------
Haven't gone through the patch yet.
The lines w.r.t. prePut() seem to be unrelated. It would be nice to remove them.
Do we need two hooks for memstore rollback ?
> Provide coprocessor hooks in put flow while rollbackMemstore.
> -------------------------------------------------------------
>
> Key: HBASE-5617
> URL: https://issues.apache.org/jira/browse/HBASE-5617
> Project: HBase
> Issue Type: Improvement
> Components: coprocessors
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 0.96.0
>
> Attachments: HBASE-5617_1.patch
>
>
> With coprocessors hooks while put happens we have the provision to create new puts to other tables or regions. These puts can be done with writeToWal as false.
> In 0.94 and above the puts are first written to memstore and then to WAL. If any failure in the WAL append or sync the memstore is rollbacked.
> Now the problem is that if the put that happens in the main flow fails there is no way to rollback the
> puts that happened in the prePut.
> We can add coprocessor hooks to like pre/postRoolBackMemStore. Is any one hook enough here?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira