You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Enis Soztutar (JIRA)" <ji...@apache.org> on 2012/08/18 03:45:37 UTC
[jira] [Created] (HBASE-6608) HBASE-6160 fix does not work
Enis Soztutar created HBASE-6608:
------------------------------------
Summary: HBASE-6160 fix does not work
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.2, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
Region A is split to -> B, C
Region B is split to -> D, E
After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
{code}
HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438053#comment-13438053 ]
Zhihong Ted Yu commented on HBASE-6608:
---------------------------------------
Integrated to 0.92 and 0.94 as well.
Thanks for the patch, Enis.
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437422#comment-13437422 ]
stack commented on HBASE-6608:
------------------------------
+1
Makes me think we should not be comparing 'offline' when doing HRI#compareTo.
Good find Enis.
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437241#comment-13437241 ]
Zhihong Ted Yu commented on HBASE-6608:
---------------------------------------
Test failures are not related to the patch.
Will integrate Monday if there is no objection.
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) HBASE-6160 fix does not work
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6608:
----------------------------------
Fix Version/s: 0.94.2
0.96.0
Hadoop Flags: Reviewed
Status: Patch Available (was: Open)
Nice finding, Enis.
> HBASE-6160 fix does not work
> -----------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.2, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.96.0, 0.94.2
>
> Attachments: hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437435#comment-13437435 ]
Zhihong Ted Yu commented on HBASE-6608:
---------------------------------------
Integrated to trunk.
@Enis, please attach patch(es) for 0.92 / 0.94
Thanks for the review, Stack.
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437498#comment-13437498 ]
Hudson commented on HBASE-6608:
-------------------------------
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #136 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/136/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1374676)
Result = FAILURE
tedyu :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) HBASE-6160 fix does not work
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437218#comment-13437218 ]
Hadoop QA commented on HBASE-6608:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541460/hbase-6608_v1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2609//console
This message is automatically generated.
> HBASE-6160 fix does not work
> -----------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.2, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.96.0, 0.94.2
>
> Attachments: hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) Fix for HBASE-6160, META entries from
daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6608:
----------------------------------
Status: Open (was: Patch Available)
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437442#comment-13437442 ]
Hudson commented on HBASE-6608:
-------------------------------
Integrated in HBase-TRUNK #3239 (See [https://builds.apache.org/job/HBase-TRUNK/3239/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1374676)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448302#comment-13448302 ]
Hudson commented on HBASE-6608:
-------------------------------
Integrated in HBase-0.94-security-on-Hadoop-23 #7 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/7/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1375158)
Result = FAILURE
Tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438068#comment-13438068 ]
stack commented on HBASE-6608:
------------------------------
@Enis I'd say new issue. Thanks boss.
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) Fix for HBASE-6160, META entries from
daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HBASE-6608:
---------------------------------
Attachment: hbase-6608_v1-0.92+0.94.patch
Thanks for the reviews. Attaching a patch for 0.92 and 0.94.
bq. Makes me think we should not be comparing 'offline' when doing HRI#compareTo.
I agree. I see no circumstance that comparing HRI.offline would be needed in compareTo() comparison. I can provide an addendum patch if you think we should remove that.
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) Fix for HBASE-6160, META entries from
daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6608:
----------------------------------
Affects Version/s: (was: 0.92.2)
0.92.1
Summary: Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (was: HBASE-6160 fix does not work )
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438162#comment-13438162 ]
Hudson commented on HBASE-6608:
-------------------------------
Integrated in HBase-0.94 #407 (See [https://builds.apache.org/job/HBase-0.94/407/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1375158)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) HBASE-6160 fix does not work
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437239#comment-13437239 ]
Hadoop QA commented on HBASE-6608:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12541465/6608-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2611//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2611//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2611//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2611//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2611//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2611//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2611//console
This message is automatically generated.
> HBASE-6160 fix does not work
> -----------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.2, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) HBASE-6160 fix does not work
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HBASE-6608:
-------------------------------
Fix Version/s: 0.92.2
Adding 0.92.2 in the fix versions. This bug should be fixed there too.
> HBASE-6160 fix does not work
> -----------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.2, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) HBASE-6160 fix does not work
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HBASE-6608:
---------------------------------
Attachment: hbase-6608_v1.patch
Attaching a patch, which changes the HashSet to use region names instead of HRI's.
> HBASE-6160 fix does not work
> -----------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.2, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Attachments: hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439517#comment-13439517 ]
Hudson commented on HBASE-6608:
-------------------------------
Integrated in HBase-0.94-security #48 (See [https://builds.apache.org/job/HBase-0.94-security/48/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1375158)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6608) HBASE-6160 fix does not work
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Ted Yu updated HBASE-6608:
----------------------------------
Attachment: 6608-v2.patch
Looks like Enis based his patch off a stale workspace.
Here is updated patch where TestCatalogJanitor passes.
> HBASE-6160 fix does not work
> -----------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.2, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438240#comment-13438240 ]
Hudson commented on HBASE-6608:
-------------------------------
Integrated in HBase-0.92 #508 (See [https://builds.apache.org/job/HBase-0.92/508/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1375159)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries
from daughters can be deleted before parent entries, shouldn't compare
HRegionInfo's
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440156#comment-13440156 ]
Hudson commented on HBASE-6608:
-------------------------------
Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1375159)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
> Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-6608
> URL: https://issues.apache.org/jira/browse/HBASE-6608
> Project: HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.92.1, 0.96.0, 0.94.2
> Reporter: Enis Soztutar
> Assignee: Enis Soztutar
> Fix For: 0.92.2, 0.96.0, 0.94.2
>
> Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch, hbase-6608_v1.patch
>
>
> Our nightlies discovered that the patch for HBASE-6160 did not actually fix the issue of "META entries from daughters can be deleted before parent entries". Instead of reopening the HBASE-6160, it is cleaner to track it here.
> The original issue is:
> {quote}
> HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests:
> Region A is split to -> B, C
> Region B is split to -> D, E
> After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B.
> {quote}
> The problem with the fix seems to be that we keep and compare HRegionInfo's in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are not equal.
> {code}
> HashSet<HRegionInfo> parentNotCleaned = new HashSet<HRegionInfo>(); //regions whose parents are still around
> for (Map.Entry<HRegionInfo, Result> e : splitParents.entrySet()) {
> if (!parentNotCleaned.contains(e.getKey()) && cleanParent(e.getKey(), e.getValue())) {
> cleaned++;
> } else {
> ...
> {code}
> In the above case, Meta row for region A will contain a serialized version of B that is not offline. However Meta row for region B will contain a serialized version of B that is offline (MetaEditor.offlineParentInMeta() does that). So the deserialized version we put to HashSet and the deserialized version we query contains() from HashSet are different in the offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira