You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Alex Newman (Created) (JIRA)" <ji...@apache.org> on 2011/11/21 23:06:53 UTC
[jira] [Created] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
If I call split fast enough, while inserting, rows disappear.
--------------------------------------------------------------
Key: HBASE-4841
URL: https://issues.apache.org/jira/browse/HBASE-4841
Project: HBase
Issue Type: Bug
Reporter: Alex Newman
I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Newman updated HBASE-4841:
-------------------------------
Description:
I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
- How many rows
- How wide the rows are
- The frequency of the split.
The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
was:I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Attachments: 1
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "ramkrishna.s.vasudevan (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan reassigned HBASE-4841:
---------------------------------------------
Assignee: ramkrishna.s.vasudevan
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154581#comment-13154581 ]
Alex Newman commented on HBASE-4841:
------------------------------------
Since this can cause dataloss it may make sense to increase the priority.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Attachments: 1
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Newman updated HBASE-4841:
-------------------------------
Attachment: log
Here is a log of this script taking the HBase server out.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160288#comment-13160288 ]
Jean-Daniel Cryans commented on HBASE-4841:
-------------------------------------------
Well the first log is the region offline issue I described, the second one has data loss indeed but as it was Nov. 21, and providing that it was a recent checkout, then it would be 4853 yeah.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159771#comment-13159771 ]
Jean-Daniel Cryans commented on HBASE-4841:
-------------------------------------------
The test passes for me 100% of the time on both trunk and 0.92 when I wrap the admin.split in order to catch the region offline exception that it gets sometimes.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "ramkrishna.s.vasudevan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155672#comment-13155672 ]
ramkrishna.s.vasudevan commented on HBASE-4841:
-----------------------------------------------
{code}
LOG.warn("Why is there a raw encodedRegionName in lastSeqWritten? name=" +
Bytes.toString(encodedRegionName) + ", seqid=" + l);
{code}
is getting repeated in the logs.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Newman updated HBASE-4841:
-------------------------------
Attachment: log2
here is a log of the wrong number of rows being returned
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160405#comment-13160405 ]
stack commented on HBASE-4841:
------------------------------
So we can close this as fixed by 4853?
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154682#comment-13154682 ]
Alex Newman commented on HBASE-4841:
------------------------------------
I realized it may be easier If I post the log for the unit test, rather than requiring you to run it. It's on the way.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Attachments: 1
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160408#comment-13160408 ]
Alex Newman commented on HBASE-4841:
------------------------------------
Agreed this test passes, although we might want have a test like this somewhere. On the other hand it is pretty high level.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Newman updated HBASE-4841:
-------------------------------
Attachment: 1
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Attachments: 1
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4841:
-------------------------
Priority: Critical (was: Major)
Upped priority
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159824#comment-13159824 ]
stack commented on HBASE-4841:
------------------------------
How does that make it so we don't miss rows J-D? (I can see how it would make splits work but not sure how it keeps scanner finding all rows)
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155684#comment-13155684 ]
stack commented on HBASE-4841:
------------------------------
Yeah, thats being fixed over in hbase-4853
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Alex Newman (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160249#comment-13160249 ]
Alex Newman commented on HBASE-4841:
------------------------------------
@JD if you look at the above posted logs you will see that it was failing with
junit.framework.AssertionFailedError: We are missing some rows
I'm guessing it was HBASE-4853.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4841) If I call split fast enough, while
inserting, rows disappear.
Posted by "Jean-Daniel Cryans (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160242#comment-13160242 ]
Jean-Daniel Cryans commented on HBASE-4841:
-------------------------------------------
Sorry if I'm not being clear.
So the claim is that this test shows we can lose rows while splitting too fast. I tried the test myself on both 0.92 and trunk multiple times.
The only times I got errors (not failures, which would mean it's failing an assertion), is when we call admin.split every so often in the test:
{code}
for (int i = 0; i != NUMBER_OF_ROWS; i++) {
byte[] rowName = Bytes.toBytes(i);
Put put = new Put(rowName);
for (int j = 0; j != NUMBER_OF_COLS; j++) {
put.add(CF, String.valueOf(j).getBytes(), Bytes.toBytes(j * i));
}
if (i % NUM_ROWS_BEFORE_SPLIT == 0 && i != 0) {
admin.split(TABLE_NAME);
LOG.info("Splitting");
}
htable.put(put);
if (i % NUM_ROWS_BEFORE_OUTPUT == 0 && i!= 0 ) {
LOG.info("Inserted Row:" + i);
}
{code}
The problem is that if you call split on a table that has a region offline, the exception bubbles all the way up and, in this case, kills the test. That's why I needed to catch and move forward.
After this change, the test passes 100% of the time for both 0.92 and trunk.
Now what I'm wondering is if the test in Alex's case was failing or "erroring". If the former, it's either an unknown bug or HBASE-4853. If the latter, then it's the issue I saw and there's no data loss.
> If I call split fast enough, while inserting, rows disappear.
> --------------------------------------------------------------
>
> Key: HBASE-4841
> URL: https://issues.apache.org/jira/browse/HBASE-4841
> Project: HBase
> Issue Type: Bug
> Reporter: Alex Newman
> Assignee: ramkrishna.s.vasudevan
> Priority: Critical
> Attachments: 1, log, log2
>
>
> I'll attach a unit test for this. Basically if you call split, while inserting data you can get to the point to where the cluster becomes unstable, or rows will disappear. The unit test gives you some flexibility of:
> - How many rows
> - How wide the rows are
> - The frequency of the split.
> The default settings crash unit tests or cause the unit tests to fail on my laptop. On my macbook air, i could actually turn down the number of total rows, and the frequency of the splits which is surprising. I think this is because the macbook air has much better IO than my backup acer.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira