You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2011/07/24 03:22:09 UTC
[jira] [Created] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Extend the WALObserver API to accomodate log archival
-----------------------------------------------------
Key: HBASE-4132
URL: https://issues.apache.org/jira/browse/HBASE-4132
Project: HBase
Issue Type: Improvement
Components: regionserver
Reporter: dhruba borthakur
Assignee: dhruba borthakur
The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HBASE-4132:
------------------------------------
Status: Patch Available (was: Open)
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070752#comment-13070752 ]
stack commented on HBASE-4132:
------------------------------
@Dhruba
bq. Given the above, do you think that this patch needs to touch LogCleaner at all? If so, what is ur proposal?
No proposal. Just wanted to point you at some utility we have already that you might not have known about and that might have helped you composing your addition.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113956#comment-13113956 ]
Hudson commented on HBASE-4132:
-------------------------------
Integrated in HBase-TRUNK #2249 (See [https://builds.apache.org/job/HBase-TRUNK/2249/])
HBASE-4132 Addendum 2 adjusts the expected value for preLogRolledCalled.size()
tedyu :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.add2, 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070832#comment-13070832 ]
stack commented on HBASE-4132:
------------------------------
+1 on one interface only. J-D!
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113866#comment-13113866 ]
Lars Hofhansl commented on HBASE-4132:
--------------------------------------
Any plan to also add this to org.apache.hadoop.hbase.coprocessor.WALObserver?
I'm happy to prepare a patch for that, maybe in a different jira.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4132:
--------------------------
Attachment: 4132.addendum
Here is the addendum.
It seems that number of times preLogRolled is called is not fixed.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HBASE-4132:
------------------------------------
Attachment: walArchive4.addendum.txt
This was an assert that my earlier patch added to the unit test. This assert was incorrect. Here is the correct fix for it.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.add2, 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt, walArchive4.addendum.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070835#comment-13070835 ]
stack commented on HBASE-4132:
------------------------------
Or, one should inherit from the other rather than repeat.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Gary Helmling (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070825#comment-13070825 ]
Gary Helmling commented on HBASE-4132:
--------------------------------------
Hmm, this seems to be a confusion of {{org.apache.hadoop.hbase.regionserver.wal.WALObserver}} and {{org.apache.hadoop.hbase.coprocessor.WALObserver}}. Not surprising since both classes have the same name. I think the former is the "WAL listener" used in replication and the latter is the coprocessor interface for WALs. I know the former has been around longer, but maybe we should consider renaming it to WALListener. Or maybe we should bite the bullet and combine these two interfaces to one. (I say that knowing very little about replication and whether it would make sense/be feasible to convert it to a coprocessor implementation).
Anyway, I see no problem adding {{pre/postArchiveStart}} and {{pre/postArchiveCompleted}} to {{org.apache.hadoop.hbase.coprocessor.WALObserver}}, as Andy mentions. Would that be sufficient, or should we look at adding the logRoll and logClose events from {{o.a.h.h.regionserver.wal.WALObserver}} as well?
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4132:
-------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to TRUNK. Thanks for the patch Dhruba.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HBASE-4132:
------------------------------------
Attachment: walArchive.txt
Added two new methods to the WALObserver interface:
{code}
/**
* The WAL needs to be archived.
* It is going to be moved from oldPath to newPath.
* @param oldPath the path to the old hlog
* @param newPath the path to the new hlog
*/
public void logArchiveStart(Path oldPath, Path newPath) throws IOException;
/**
* The WAL has been archived.
* It is moved from oldPath to newPath.
* @param oldPath the path to the old hlog
* @param newPath the path to the new hlog
* @param archivalWasSuccessful true, if the archival was successful
*/
public void logArchiveComplete(Path oldPath, Path newPath,
boolean archivalWasSuccessful) throws IOException;
{code}
Any backward compatibility issues I need to think about? Especially since this has been a public API.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113929#comment-13113929 ]
Hudson commented on HBASE-4132:
-------------------------------
Integrated in HBase-TRUNK #2248 (See [https://builds.apache.org/job/HBase-TRUNK/2248/])
HBASE-4132 Addendum adjusts the expected value for preLogRolledCalled.size()
tedyu :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070877#comment-13070877 ]
Andrew Purtell commented on HBASE-4132:
---------------------------------------
bq. Hmm, this seems to be a confusion of org.apache.hadoop.hbase.regionserver.wal.WALObserver and org.apache.hadoop.hbase.coprocessor.WALObserver.
Aha.
I agree with J-D, should do a rename.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4132:
--------------------------
Attachment: 4132.add2
Second attempt at making Jenkins happy.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.add2, 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Jonathan Gray (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13112249#comment-13112249 ]
Jonathan Gray commented on HBASE-4132:
--------------------------------------
Looks good. One thing:
{code}oldPath = new Path("/DUMMY-No-preexisting-logfile");{code}
Should we support passing a null path or at least use a static?
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113893#comment-13113893 ]
Ted Yu commented on HBASE-4132:
-------------------------------
Running TestLogRolling#testLogRollOnPipelineRestart in debugger, I got:
{code}
java.lang.AssertionError: preLogRolledCalled has size of 3
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:311)
{code}
The first two calls were:
{code}
TestLogRolling$1.preLogRoll(Path, Path) line: 245
HLog.rollWriter(boolean) line: 564
LogRoller.run() line: 93
{code}
The third one was:
{code}
TestLogRolling$1.preLogRoll(Path, Path) line: 245
HLog.rollWriter(boolean) line: 564
TestLogRolling.testLogRollOnPipelineRestart() line: 310
{code}
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070588#comment-13070588 ]
Andrew Purtell commented on HBASE-4132:
---------------------------------------
@dhruba Thanks. For example should be preArchiveStart and postArchiveStart, preArchiveCompleted and postArchiveCompleted. In part it is a naming convention, in part it is a contract: Pre hooks allow introduction of preprocessing and also important override of default behavior and associated short-circuiting of base processing or any additional coprocessors. Post hooks allow introduction of postprocessing and modification of return values.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109793#comment-13109793 ]
dhruba borthakur commented on HBASE-4132:
-----------------------------------------
I am getting back to this one. Here is the modified proposal as suggested by Andrew. I will remove WALActionsListener.logRolled and introduce the following methods to WALActionsListener:
{code}
/**
* The WAL is going to be rolled.
* @param oldPath the path to the old hlog
* @param newPath the path to the new hlog
*/
public void preLogRoll(Path oldPath, Path newPath) throws IOException;
/**
* The WAL has been rolled.
* @param oldPath the path to the old hlog
* @param newPath the path to the new hlog
*/
public void postLogRoll(Path oldPath, Path newPath) throws IOException;
/**
* The WAL is going to be archived.
* @param oldPath the path to the old hlog
* @param newPath the path to the new hlog
*/
public void preLogArchive(Path oldPath, Path newPath) throws IOException;
/**
* The WAL has been archived.
* @param oldPath the path to the old hlog
* @param newPath the path to the new hlog
*/
public void postLogArchive(Path oldPath, Path newPath) throws IOException;
{code}
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070246#comment-13070246 ]
Andrew Purtell commented on HBASE-4132:
---------------------------------------
The WALObserver API additions should follow the same practice as providing before (pre) and after (post) hooks as everywhere else.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070863#comment-13070863 ]
Jean-Daniel Cryans commented on HBASE-4132:
-------------------------------------------
It could be weird, if we just merge them then the Replication class (and others implementing wal.WALObserver in the future) would have imports for CP classes since ObserverContext is passed in the cp.WALObserver methods. I'd prefer a rename of either or both.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-4132:
----------------------------------
Summary: Extend the WALActionsListener API to accomodate log archival (was: Extend the WALObserver API to accomodate log archival)
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-4132:
----------------------------------
Status: Open (was: Patch Available)
Cancelling patch.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070097#comment-13070097 ]
stack commented on HBASE-4132:
------------------------------
There is also master/LogCleaner.java. You know about that? It takes plugins that say whether or not a log should be deleted. You interested in this at all?
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HBASE-4132:
------------------------------------
Attachment: walArchive3.txt
Addressed Jgrays' review comments. The API documents that oldPath may be null.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-4132) Extend the
WALActionsListener API to accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113895#comment-13113895 ]
Ted Yu edited comment on HBASE-4132 at 9/24/11 5:25 AM:
--------------------------------------------------------
Here is the addendum.
TestLogRolling#testLogRollOnPipelineRestart passed on my MacBook.
Let me see if the addendum helps on Jenkins.
was (Author: yuzhihong@gmail.com):
Here is the addendum.
It seems that number of times preLogRolled is called is not fixed.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093378#comment-13093378 ]
Ted Yu commented on HBASE-4132:
-------------------------------
@Andrew:
Do you think the following signature is close to what's in your mind ?
{code}
+ public Path preLogArchiveStart(Path oldPath, Path newPath) throws IOException
{code}
Basically null return value means skipping log archival. Otherwise the return value should be used as the effective newPath.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu resolved HBASE-4132.
---------------------------
Resolution: Fixed
TRUNK build 2257 passed.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.add2, 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt, walArchive4.addendum.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070095#comment-13070095 ]
stack commented on HBASE-4132:
------------------------------
+1 on patch. We could put it into a 0.90.5 if wanted since this is all internal APIs. Otherwise it'll go into 0.92.
I'll let it hang out a little while before committing. J-D or Gary might have opinions.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070349#comment-13070349 ]
dhruba borthakur commented on HBASE-4132:
-----------------------------------------
stack: the LogCleaner api allows archived logs to be deleted according to a configurable policy. One can set hbase.master.logcleaner.plugins to setup his own policy. In that sense, it is already pluggable. Moreover, this is done by the master whereas the WALObserver interface is in the RegionServer. Given the above, do you think that this patch needs to touch LogCleaner at all? If so, what is ur proposal?
Andrew: "The WALObserver API additions should follow the same practice as providing before (pre) and after (post) hooks as everywhere else". In that sense, it already has logRollRequested and logRolled. Similarly, I added logArchiveStart and logArchiveCompleted. The remaining one is "visitLogEntryBeforeWrite". are you suggesting that we add a "visitLogEntryAfterWrite" as well?
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070946#comment-13070946 ]
Jean-Daniel Cryans commented on HBASE-4132:
-------------------------------------------
+1
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4132:
-------------------------
Fix Version/s: 0.92.0
Release Note: Add pre and post archiving methods to WALObserver
Status: Patch Available (was: Open)
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070107#comment-13070107 ]
dhruba borthakur commented on HBASE-4132:
-----------------------------------------
stack: I did not even know the existence of master/LogCleaner. Thanks a bunch for pointing it out. Definitely will look into that.
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HBASE-4132:
------------------------------------
Assignee: dhruba borthakur
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4132:
--------------------------
Fix Version/s: (was: 0.92.0)
0.94.0
Lack of response.
Can move back into 0.92 when there is a clear use case.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113777#comment-13113777 ]
Hudson commented on HBASE-4132:
-------------------------------
Integrated in HBase-TRUNK #2246 (See [https://builds.apache.org/job/HBase-TRUNK/2246/])
HBASE-4132 Extend the WALActionsListener API to accomodate log archival
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALActionsListener.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114100#comment-13114100 ]
Ted Yu commented on HBASE-4132:
-------------------------------
I was thinking along the same line.
Addendum applied to TRUNK.
Thanks for the addendum Dhruba.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.add2, 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt, walArchive4.addendum.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "gaojinchao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093403#comment-13093403 ]
gaojinchao commented on HBASE-4132:
-----------------------------------
Whether we can add the following api?
/**
* The WAL needs to be archived. It is going to be moved from oldPath to
* newPath.
*
* @param oldPath
* the path to the old hlog
* @param newPath
* the path to the new hlog
* @return true if default behavior should be bypassed, false otherwise
*/
boolean preArchiveLog(Path oldPath, Path newPath) throws IOException;
/**
* The WAL has been archived. It is moved from oldPath to newPath.
*
* @param oldPath
* the path to the old hlog
* @param newPath
* the path to the new hlog
* @param archivalWasSuccessful
* true, if the archival was successful
*/
void postArchiveLog(Path oldPath, Path newPath,
boolean archivalWasSuccessful) throws IOException;
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reopened HBASE-4132:
---------------------------
This is from TRUNK build 2249:
{code}
java.lang.AssertionError: preLogRolledCalled has size of 4
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.hadoop.hbase.regionserver.wal.TestLogRolling.testLogRollOnPipelineRestart(TestLogRolling.java:491)
{code}
We need to decide the proper assertion w.r.t. preLogRolledCalled which may have size of 3 or 4.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.add2, 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALActionsListener API
to accomodate log archival
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13114110#comment-13114110 ]
Hudson commented on HBASE-4132:
-------------------------------
Integrated in HBase-TRUNK #2253 (See [https://builds.apache.org/job/HBase-TRUNK/2253/])
HBASE-4132 Addendum from Dhruba
tedyu :
Files :
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestLogRolling.java
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: 4132.add2, 4132.addendum, walArchive.txt, walArchive2.txt, walArchive3.txt, walArchive4.addendum.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4132) Extend the WALObserver API to
accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070887#comment-13070887 ]
Ted Yu commented on HBASE-4132:
-------------------------------
org.apache.hadoop.hbase.regionserver.wal.WALObserver is mostly internal.
It is used for LogRoller and replication.
Shall we rename it to org.apache.hadoop.hbase.regionserver.wal.WALActionsListener ?
See HRegionServer.getWALActionListeners():
{code}
// Replication handler is an implementation of WALActionsListener.
listeners.add(this.replicationHandler);
{code}
> Extend the WALObserver API to accomodate log archival
> -----------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
dhruba borthakur updated HBASE-4132:
------------------------------------
Attachment: walArchive2.txt
New patch that introduces pre/post methods in WALActionsListener.
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Fix For: 0.94.0
>
> Attachments: walArchive.txt, walArchive2.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4132) Extend the WALActionsListener API to
accomodate log archival
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reassigned HBASE-4132:
-----------------------------
Assignee: (was: dhruba borthakur)
> Extend the WALActionsListener API to accomodate log archival
> ------------------------------------------------------------
>
> Key: HBASE-4132
> URL: https://issues.apache.org/jira/browse/HBASE-4132
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: dhruba borthakur
> Fix For: 0.92.0
>
> Attachments: walArchive.txt
>
>
> The WALObserver interface exposes the log roll events. It would be nice to extend it to accomodate log archival events as well.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira