You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2010/04/01 08:21:27 UTC
[jira] Resolved: (HBASE-2365) Double-assignment around split
[ https://issues.apache.org/jira/browse/HBASE-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-2365.
--------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Committed branch and trunk. Thanks for reviewing and testing J-D (Though, I'm sure this change will come back to bite us).
> Double-assignment around split
> ------------------------------
>
> Key: HBASE-2365
> URL: https://issues.apache.org/jira/browse/HBASE-2365
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 0.20.4, 0.21.0
>
> Attachments: 2365.patch, zheng.tgz
>
>
> Its looking like we have a split updating .META. with daughter regions and then before we process the split in master, one of the daughters has already been assigned. On processing of the split, we assign daughter again.
> I thought this had been fixed previously? Doesn't seem so. Need to look again.
> Here is evidence for region named:
> {code}summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017{code}
> First master-side:
> {code}
> 2010-03-15 16:06:52,153 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning region summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017 to cactus208,60020,12686305486412010-03-15 16:06:52,156 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner scan of 245 row(s) of meta region {server: 172.16.1.209:60020, regionname: .META.,,1, startKey: <>} complete2010-03-15 16:06:52,156 INFO org.apache.hadoop.hbase.master.BaseScanner: All 1 .META. region(s) scanned2010-03-15 16:06:52,841 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning region summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E5\x25A4\x25A7\x25E8\x25BF\x259E\x25E5\x2588\x2598\x25E5\x25B8\x2588\x25E5\x2582\x2585\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E5\x2587\x25B9\x25E9\x2599\x25B7\x25E4\x25BF\x25AE\x25E5\x25A4\x258D\x25E6\x2596\x25BD\x25E5\x25B7\x25A5\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E7\x2594\x25A8\x25E5\x2593\x2581\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017 to cactus210,60020,12686305508862010-03-15 16:06:54,377 INFO org.apache.hadoop.hbase.master.ServerManager: Processing MSG_REPORT_SPLIT: summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E5\x25A4\x25A7\x25E8\x25BF\x259E\x25E5\x2588\x2598\x25E5\x25B8\x2588\x25E5\x2582\x2585\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E5\x2587\x25B9\x25E9\x2599\x25B7\x25E4\x25BF\x25AE\x25E5\x25A4\x258D\x25E6\x2596\x25BD\x25E5\x25B7\x25A5\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E7\x2594\x25A8\x25E5\x2593\x2581\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268611349836: Daughters; summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E5\x25A4\x25A7\x25E8\x25BF\x259E\x25E5\x2588\x2598\x25E5\x25B8\x2588\x25E5\x2582\x2585\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E5\x2587\x25B9\x25E9\x2599\x25B7\x25E4\x25BF\x25AE\x25E5\x25A4\x258D\x25E6\x2596\x25BD\x25E5\x25B7\x25A5\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E7\x2594\x25A8\x25E5\x2593\x2581\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017, summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017 from cactus209,60020,1268630548451; 1 of 32010-03-15 16:06:54,388 INFO org.apache.hadoop.hbase.master.RegionManager: Assigning region summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017 to cactus209,60020,1268630548451
> {code}
> Its hard to read but above is an assignment, the split message, then what seems to be same region being assigned again.
> Here is RS side on 209 server:
> {code}
> 2010-03-15 16:06:29,727 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017
> 2010-03-15 16:06:29,792 INFO org.apache.hadoop.hbase.regionserver.HRegion: region summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017/1011052036 available; sequence id is 199443346
> 2010-03-15 16:06:29,792 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E5\x25A4\x25A7\x25E8\x25BF\x259E\x25E5\x2588\x2598\x25E5\x25B8\x2588\x25E5\x2582\x2585\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E5\x2587\x25B9\x25E9\x2599\x25B7\x25E4\x25BF\x25AE\x25E5\x25A4\x258D\x25E6\x2596\x25BD\x25E5\x25B7\x25A5\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E7\x2594\x25A8\x25E5\x2593\x2581\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017
> 2010-03-15 16:06:29,793 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017
> 2010-03-15 16:06:29,944 INFO org.apache.hadoop.hbase.regionserver.HRegion: region summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E5\x25A4\x25A7\x25E8\x25BF\x259E\x25E5\x2588\x2598\x25E5\x25B8\x2588\x25E5\x2582\x2585\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E5\x2587\x25B9\x25E9\x2599\x25B7\x25E4\x25BF\x25AE\x25E5\x25A4\x258D\x25E6\x2596\x25BD\x25E5\x25B7\x25A5\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E7\x2594\x25A8\x25E5\x2593\x2581\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017/1971466363 available; sequence id is 199443345
> 2010-03-15 16:06:32,750 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_CLOSE_WITHOUT_REPORT: summary,SITE_0000000032\x01pt\x0120100314000000\x01\x25E7\x258C\x25AE\x25E5\x258E\x25BF\x25E5\x2586\x2580\x25E9\x25B9\x25B0\x25E6\x2591\x25A9\x25E6\x2593\x25A6\x25E6\x259D\x2590\x25E6\x2596\x2599\x25E5\x258E\x2582\x2B\x25E6\x25B1\x25BD\x25E8\x25BD\x25A6\x25E9\x2585\x258D\x25E4\x25BB\x25B6\x25EF\x25BC\x258C\x25E5\x2598\x2580\x25E9\x2593\x2583\x25E9\x2593\x2583--\x25E7\x259C\x259F\x25E5\x25AE\x259E\x25E5\x25AE\x2589\x25E5\x2585\x25A8\x25E7\x259A\x2584\x25E7\x2594\x25B5\x25E8\x25AF\x259D\x25E3\x2580\x2581\x25E7\x25BD\x2591\x25E7\x25BB\x259C\x25E4\x25BA\x2592\x25E5\x258A\x25A8\x25E4\x25BA\x25A4\x25E5\x258F\x258B\x25E7\x25A4\x25BE\x25E5\x258C\x25BA\x25EF\x25BC\x2581,1268640385017: Duplicate assignment
> {code}
> See how we end with 'Duplicate assignment' message?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.