You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Aleksandr Shulman (JIRA)" <ji...@apache.org> on 2012/11/09 02:34:12 UTC
[jira] [Created] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Aleksandr Shulman created HBASE-7131:
----------------------------------------
Summary: Race condition after table is re-enabled: regions are incorrectly reported as being available.
Key: HBASE-7131
URL: https://issues.apache.org/jira/browse/HBASE-7131
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.96.0
Reporter: Aleksandr Shulman
Assignee: Jimmy Xiang
For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-7131:
-------------------------------
Resolution: Not A Problem
Status: Resolved (was: Patch Available)
Close this issue as Not A Problem. Filed a new issue HBASE-7144 on the retry issue. Will re-use the same test on HBASE-7144, without clean the region cache.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494130#comment-13494130 ]
ramkrishna.s.vasudevan commented on HBASE-7131:
-----------------------------------------------
Present in 0.94 also i think.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Attachments: HBase-7131-v1.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-7131:
-------------------------------
Component/s: (was: Region Assignment)
test
Priority: Minor (was: Major)
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494506#comment-13494506 ]
Hadoop QA commented on HBASE-7131:
----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552920/trunk-7131.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 87 warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 16 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3295//console
This message is automatically generated.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494433#comment-13494433 ]
Jimmy Xiang commented on HBASE-7131:
------------------------------------
After the table is re-enabled, the region could be assigned to a different region server. However, the HBase client caches the old connection, that's why the NotServingRegion exception. Once you add a line to clear the region cache, then it should work as expected:
original.getConnection().clearRegionCache();
I was thinking how should the HBase client to deal with such scenario better.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-7131:
-------------------------------
Attachment: trunk-7131.patch
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang updated HBASE-7131:
-------------------------------
Status: Patch Available (was: Open)
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494406#comment-13494406 ]
Jimmy Xiang commented on HBASE-7131:
------------------------------------
Thanks for reporting this issue. It is a test problem. I will fix the test case you uploaded.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Aleksandr Shulman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494425#comment-13494425 ]
Aleksandr Shulman commented on HBASE-7131:
------------------------------------------
Can you elaborate?
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Aleksandr Shulman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498283#comment-13498283 ]
Aleksandr Shulman commented on HBASE-7131:
------------------------------------------
No problem. Thanks everyone for looking into this issue thoroughly.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Aleksandr Shulman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494497#comment-13494497 ]
Aleksandr Shulman commented on HBASE-7131:
------------------------------------------
Good call. I tried your fix and it stopped the flakiness, with 40 consecutive successful runs.
Perhaps we should clear the region cache automatically after a table is enabled or disabled?
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Jimmy Xiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494822#comment-13494822 ]
Jimmy Xiang commented on HBASE-7131:
------------------------------------
@Ram, yes, it should be done automatically. I will look into it in HBASE-7144.
@Aleks, thanks for verifying the test. We will use the same test in HBASE-7144, without region cache clean.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "Aleksandr Shulman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aleksandr Shulman updated HBASE-7131:
-------------------------------------
Attachment: HBase-7131-v1.patch
Test to verify the fix. Right now it is flaky (which demonstrates the bug).
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Attachments: HBase-7131-v1.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7131) Race condition after table is
re-enabled: regions are incorrectly reported as being available.
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-7131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494584#comment-13494584 ]
ramkrishna.s.vasudevan commented on HBASE-7131:
-----------------------------------------------
Yes Jimmy. This is what i observed yesterday. It should be done automatically i suppose.
> Race condition after table is re-enabled: regions are incorrectly reported as being available.
> ----------------------------------------------------------------------------------------------
>
> Key: HBASE-7131
> URL: https://issues.apache.org/jira/browse/HBASE-7131
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.96.0
> Reporter: Aleksandr Shulman
> Assignee: Jimmy Xiang
> Priority: Minor
> Attachments: HBase-7131-v1.patch, trunk-7131.patch
>
>
> For a table that is re-enabled shortly after it is disabled, regions that are reported to be online are not. This is manifested by a flush attempt throwing a NotServingRegion exception despite all regions from the original table reporting that they are online.
> I have a test in place that verifies this flaky behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira