You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2010/06/04 08:36:52 UTC
[jira] Created: (HBASE-2667) TestHLog.testSplit failing in trunk
TestHLog.testSplit failing in trunk
-----------------------------------
Key: HBASE-2667
URL: https://issues.apache.org/jira/browse/HBASE-2667
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: stack
Build is broke because this test won't pass.
Seems to be two issues at least:
1. we need to change the test so it keeps logs in one subdir, archives in another and split products in yet another as per regionserver. New split code does cleanup of old logs dir if successful. In this test it was removing the created splits so when verify went to run, the log no longer existed
2. The verify of splits tests that sequence numbers are in order. They don't seem to be (the assertion failure was masked by a close reader exception).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2667) TestHLog.testSplit failing in trunk
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875871#action_12875871 ]
stack commented on HBASE-2667:
------------------------------
You stuff is cleaner than my crap Cosmin. I committed it. Thanks.
> TestHLog.testSplit failing in trunk
> -----------------------------------
>
> Key: HBASE-2667
> URL: https://issues.apache.org/jira/browse/HBASE-2667
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Fix For: 0.21.0
>
> Attachments: 2667-prelim.txt, 2667.txt, HBASE-2667-v2.patch
>
>
> Build is broke because this test won't pass.
> Seems to be two issues at least:
> 1. we need to change the test so it keeps logs in one subdir, archives in another and split products in yet another as per regionserver. New split code does cleanup of old logs dir if successful. In this test it was removing the created splits so when verify went to run, the log no longer existed
> 2. The verify of splits tests that sequence numbers are in order. They don't seem to be (the assertion failure was masked by a close reader exception).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-2667) TestHLog.testSplit failing in trunk
Posted by "Cosmin Lehene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875881#action_12875881 ]
Cosmin Lehene commented on HBASE-2667:
--------------------------------------
My previous, cleaner stuff broke it :P.
> TestHLog.testSplit failing in trunk
> -----------------------------------
>
> Key: HBASE-2667
> URL: https://issues.apache.org/jira/browse/HBASE-2667
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Fix For: 0.21.0
>
> Attachments: 2667-prelim.txt, 2667.txt, HBASE-2667-v2.patch
>
>
> Build is broke because this test won't pass.
> Seems to be two issues at least:
> 1. we need to change the test so it keeps logs in one subdir, archives in another and split products in yet another as per regionserver. New split code does cleanup of old logs dir if successful. In this test it was removing the created splits so when verify went to run, the log no longer existed
> 2. The verify of splits tests that sequence numbers are in order. They don't seem to be (the assertion failure was masked by a close reader exception).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-2667) TestHLog.testSplit failing in trunk
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-2667.
--------------------------
Fix Version/s: 0.21.0
Resolution: Fixed
Committed. Change is small and build is broke.
> TestHLog.testSplit failing in trunk
> -----------------------------------
>
> Key: HBASE-2667
> URL: https://issues.apache.org/jira/browse/HBASE-2667
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Fix For: 0.21.0
>
> Attachments: 2667-prelim.txt, 2667.txt
>
>
> Build is broke because this test won't pass.
> Seems to be two issues at least:
> 1. we need to change the test so it keeps logs in one subdir, archives in another and split products in yet another as per regionserver. New split code does cleanup of old logs dir if successful. In this test it was removing the created splits so when verify went to run, the log no longer existed
> 2. The verify of splits tests that sequence numbers are in order. They don't seem to be (the assertion failure was masked by a close reader exception).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2667) TestHLog.testSplit failing in trunk
Posted by "Cosmin Lehene (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cosmin Lehene updated HBASE-2667:
---------------------------------
Attachment: HBASE-2667-v2.patch
Added a new unit test TestHLogSplit.testSplitPreservesEdits that checks edits integrity after split.
Changed the looping through the edits to a simple for and fixed parseHLog to append (addLast) to the list of edits.
> TestHLog.testSplit failing in trunk
> -----------------------------------
>
> Key: HBASE-2667
> URL: https://issues.apache.org/jira/browse/HBASE-2667
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Fix For: 0.21.0
>
> Attachments: 2667-prelim.txt, 2667.txt, HBASE-2667-v2.patch
>
>
> Build is broke because this test won't pass.
> Seems to be two issues at least:
> 1. we need to change the test so it keeps logs in one subdir, archives in another and split products in yet another as per regionserver. New split code does cleanup of old logs dir if successful. In this test it was removing the created splits so when verify went to run, the log no longer existed
> 2. The verify of splits tests that sequence numbers are in order. They don't seem to be (the assertion failure was masked by a close reader exception).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2667) TestHLog.testSplit failing in trunk
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2667:
-------------------------
Attachment: 2667-prelim.txt
Some preliminary work. Does clean up dir locations in test. Adds logging. No fix yet.
> TestHLog.testSplit failing in trunk
> -----------------------------------
>
> Key: HBASE-2667
> URL: https://issues.apache.org/jira/browse/HBASE-2667
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Attachments: 2667-prelim.txt
>
>
> Build is broke because this test won't pass.
> Seems to be two issues at least:
> 1. we need to change the test so it keeps logs in one subdir, archives in another and split products in yet another as per regionserver. New split code does cleanup of old logs dir if successful. In this test it was removing the created splits so when verify went to run, the log no longer existed
> 2. The verify of splits tests that sequence numbers are in order. They don't seem to be (the assertion failure was masked by a close reader exception).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-2667) TestHLog.testSplit failing in trunk
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-2667:
-------------------------
Attachment: 2667.txt
The test checked sequenceid ordering in the produced split. They were being written in the reverse order -- newest first. The reconstruction log over in HStore expects to read the edits oldest through to newest so changed how we make the recovered edits files.
Here is the change:
{code}
@@ -1552,10 +1552,11 @@ public class HLog implements HConstants, Syncable {
try {
int editsCount = 0;
WriterAndPath wap = logWriters.get(region);
- for (ListIterator<Entry> iterator = entries.listIterator();
- iterator.hasNext();) {
- Entry logEntry = iterator.next();
-
+ // We put edits onto the Stack ordered oldest sequence id to newest.
+ // Pop them off starting with the oldest.
+ for (ListIterator<Entry> iterator = entries.listIterator(entries.size());
+ iterator.hasPrevious();) {
+ Entry logEntry = iterator.previous();
{code}
> TestHLog.testSplit failing in trunk
> -----------------------------------
>
> Key: HBASE-2667
> URL: https://issues.apache.org/jira/browse/HBASE-2667
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Attachments: 2667-prelim.txt, 2667.txt
>
>
> Build is broke because this test won't pass.
> Seems to be two issues at least:
> 1. we need to change the test so it keeps logs in one subdir, archives in another and split products in yet another as per regionserver. New split code does cleanup of old logs dir if successful. In this test it was removing the created splits so when verify went to run, the log no longer existed
> 2. The verify of splits tests that sequence numbers are in order. They don't seem to be (the assertion failure was masked by a close reader exception).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.