You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Jim Kellerman (JIRA)" <ji...@apache.org> on 2009/01/06 00:07:44 UTC
[jira] Updated: (HBASE-1098) IllegalStateException: Cannot set a
region to be closed it it was not already marked as closing
[ https://issues.apache.org/jira/browse/HBASE-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Kellerman updated HBASE-1098:
---------------------------------
Issue Type: Sub-task (was: Bug)
Parent: HBASE-1104
> IllegalStateException: Cannot set a region to be closed it it was not already marked as closing
> -----------------------------------------------------------------------------------------------
>
> Key: HBASE-1098
> URL: https://issues.apache.org/jira/browse/HBASE-1098
> Project: Hadoop HBase
> Issue Type: Sub-task
> Environment: apurtel cluster
> Reporter: stack
> Assignee: Jim Kellerman
> Fix For: 0.19.0
>
> Attachments: 1098-p2.patch, 1098.patch
>
>
> content,ec0ec4d005ee7da14cbd1182339f8cf4,1230416299322 is problem region. below is snippet from master log. Looks to me like things are proceeding in order.
> {code}
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Server XX.XX.XX.51:60020 is overloaded. Server load: 74 avg: 48.0, slop: 0.1
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Choosing to reassign 26 regions. mostLoadedRegions has 10 regions in it.
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,3ea6e20cf48a32fc063912ccae0e960e,1230416307426
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,07730cecbb5b13364cbe17af5778f983,1230448573272
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,e8c750a5983137306e6b3a84eb1fd6e6,1230424193882
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,29be0a07dece4e9bb318c4d253807287,1230356680860
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,23893c82d792ff5c9d3ff99123bc1773,1230390013767
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,e76475012b59500d25000aa7658bc279,1230498762282
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region result_domain,cn.com.sg.www,1230498785950
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,c7b7e7a475765e16d0bbf1e51c95e85d,1230519854914
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,ec0ec4d005ee7da14cbd1182339f8cf4,1230416299322
> 2008-12-29 12:19:32,074 DEBUG org.apache.hadoop.hbase.master.RegionManager: Going to close region content,e16b2ea9f6da8b32fa96ab3490e3d849,1230424086678
> 2008-12-29 12:19:32,074 INFO org.apache.hadoop.hbase.master.RegionManager: Skipped 0 region(s) that are in transition states
> 2008-12-29 12:19:32,076 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: PendingOpenOperation from XX.XX.XX.51:60020
> 2008-12-29 12:19:32,076 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen$1: content,42b19221f3a60945595b18017d6bacfa,1230357228381 open on XX.XX.XX.51:60020
> 2008-12-29 12:19:32,076 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen$1: updating row content,42b19221f3a60945595b18017d6bacfa,1230357228381 in region .META.,,1 with startcode 1230571104777 and server XX.XX.XX.51:60020
> 2008-12-29 12:19:32,077 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: PendingOpenOperation from XX.XX.XX.51:60020
> 2008-12-29 12:19:32,077 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen$1: content,3c619bc7a2c824fb3c4f5d747b0f2f12,1230354673241 open on XX.XX.XX.51:60020
> 2008-12-29 12:19:32,077 INFO org.apache.hadoop.hbase.master.ProcessRegionOpen$1: updating row content,3c619bc7a2c824fb3c4f5d747b0f2f12,1230354673241 in region .META.,,1 with startcode 1230571104777 and server XX.XX.XX.51:60020
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,36946541ed9a62f419cf7238d32a6a38,1230448587552, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,58221f5f3a79b7dd5e23c339c507a7c6,1230487059978, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,94b170cae87992c33c3913d74bb257e4,1230498360162, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,bb34d860d76c1eabbe2eb7d2ce8148d9,1230416266955, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,8ac5fbdd63b313ce252838a189c73438,1230530000480, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,f9f5b2d035349cf4292ed7dd7e9823f0,1230505507313, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,4b30e2a8b7949ed62e097824969f8bca,1230448501152, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,0c4b7c2825f674510a511233e9e0c539,1230530122456, false
> 2008-12-29 12:19:32,078 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,43f83c42e08bb0f4b1f6d1125f39f018,1230498357831, false
> 2008-12-29 12:19:32,299 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,43f83c42e08bb0f4b1f6d1125f39f018,1230498357831 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,300 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,bb34d860d76c1eabbe2eb7d2ce8148d9,1230416266955 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,301 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,94b170cae87992c33c3913d74bb257e4,1230498360162 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,301 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,8ac5fbdd63b313ce252838a189c73438,1230530000480 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,302 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,36946541ed9a62f419cf7238d32a6a38,1230448587552 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,302 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,58221f5f3a79b7dd5e23c339c507a7c6,1230487059978 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,303 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,f9f5b2d035349cf4292ed7dd7e9823f0,1230505507313 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,303 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,4b30e2a8b7949ed62e097824969f8bca,1230448501152 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:32,304 INFO org.apache.hadoop.hbase.master.RegionManager: assigning region content,0c4b7c2825f674510a511233e9e0c539,1230530122456 to server XX.XX.XX.34:60020
> 2008-12-29 12:19:35,080 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,3ea6e20cf48a32fc063912ccae0e960e,1230416307426 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,07730cecbb5b13364cbe17af5778f983,1230448573272 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,3ea6e20cf48a32fc063912ccae0e960e,1230416307426, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,e8c750a5983137306e6b3a84eb1fd6e6,1230424193882 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,07730cecbb5b13364cbe17af5778f983,1230448573272, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,29be0a07dece4e9bb318c4d253807287,1230356680860 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,e8c750a5983137306e6b3a84eb1fd6e6,1230424193882, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,23893c82d792ff5c9d3ff99123bc1773,1230390013767 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,29be0a07dece4e9bb318c4d253807287,1230356680860, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,e76475012b59500d25000aa7658bc279,1230498762282 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,23893c82d792ff5c9d3ff99123bc1773,1230390013767, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: result_domain,cn.com.sg.www,1230498785950 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,e76475012b59500d25000aa7658bc279,1230498762282, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,c7b7e7a475765e16d0bbf1e51c95e85d,1230519854914 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of result_domain,cn.com.sg.www,1230498785950, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,ec0ec4d005ee7da14cbd1182339f8cf4,1230416299322 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,c7b7e7a475765e16d0bbf1e51c95e85d,1230519854914, false
> 2008-12-29 12:19:35,081 INFO org.apache.hadoop.hbase.master.ServerManager: Received MSG_REPORT_CLOSE: content,e16b2ea9f6da8b32fa96ab3490e3d849,1230424086678 from XX.XX.XX.51:60020
> 2008-12-29 12:19:35,081 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,ec0ec4d005ee7da14cbd1182339f8cf4,1230416299322, false
> 2008-12-29 12:19:35,082 DEBUG org.apache.hadoop.hbase.master.HMaster: Main processing loop: ProcessRegionClose of content,e16b2ea9f6da8b32fa96ab3490e3d849,1230424086678, false
> 2008-12-29 12:19:35,083 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000, call regionServerReport(address: XX.XX.XX.51:60020, startcode: 1230571104777, load: (requests=0, regions=64, usedHeap=99, maxHeap=1777), [Lorg.
> apache.hadoop.hbase.HMsg;@3e2dce4e, [Lorg.apache.hadoop.hbase.HRegionInfo;@314382c6) from XX.XX.XX.51:55371: error: java.io.IOException: java.lang.IllegalStateException: Cannot set a region to be closed if it was not already marked as clo
> sing. Region: content,ec0ec4d005ee7da14cbd1182339f8cf4,1230416299322
> java.io.IOException: java.lang.IllegalStateException: Cannot set a region to be closed if it was not already marked as closing. Region: content,ec0ec4d005ee7da14cbd1182339f8cf4,1230416299322
> at org.apache.hadoop.hbase.master.RegionManager$RegionState.setClosed(RegionManager.java:1108)
> at org.apache.hadoop.hbase.master.RegionManager.setClosed(RegionManager.java:816)
> at org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.java:377)
> at org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell(ServerManager.java:327)
> at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:240)
> at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:570)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
> at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.