You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by David Alves <dr...@criticalsoftware.com> on 2008/04/21 18:50:13 UTC

Concurrent Modification Exceptions in logs

Hi Guys 
	
	My NPE problem on online table lookup seemed to go away (at least until
now), I think the cause was different dfs.replication values for hadoop
and hbase (thanks st.ack for pointing it out), now I'm just struggling
with region offline exceptions :).
	I'm seeing some CMEs in the logs they occurred while I still had bad
dfs.replication settings between hadoop and hbase but still thought you
should know.

Regards
David Alves

Trace:
2008-04-21 13:20:46,443 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: Processing message
(Retry: 0)
java.io.IOException: java.io.IOException:
java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at
org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.java:350)
	at
org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell(ServerManager.java:299)
	at
org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:217)
	at
org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:560)
	at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
	at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
	at
org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteExceptionHandler.java:48)
	at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:388)
	at java.lang.Thread.run(Thread.java:619)

Re: Concurrent Modification Exceptions in logs

Posted by stack <st...@duboce.net>.
CMEs are from the new region balancing code.  I made HBASE-599 to cover 
the fix for it.  Thanks for reporting the issue.
St.Ack

David Alves wrote:
> Hi Guys 
> 	
> 	My NPE problem on online table lookup seemed to go away (at least until
> now), I think the cause was different dfs.replication values for hadoop
> and hbase (thanks st.ack for pointing it out), now I'm just struggling
> with region offline exceptions :).
> 	I'm seeing some CMEs in the logs they occurred while I still had bad
> dfs.replication settings between hadoop and hbase but still thought you
> should know.
>
> Regards
> David Alves
>
> Trace:
> 2008-04-21 13:20:46,443 WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: Processing message
> (Retry: 0)
> java.io.IOException: java.io.IOException:
> java.util.ConcurrentModificationException
> 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
> 	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
> 	at
> org.apache.hadoop.hbase.master.ServerManager.processMsgs(ServerManager.java:350)
> 	at
> org.apache.hadoop.hbase.master.ServerManager.processRegionServerAllsWell(ServerManager.java:299)
> 	at
> org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:217)
> 	at
> org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:560)
> 	at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> 	at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> 	at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> 	at
> org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
> 	at
> org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteExceptionHandler.java:48)
> 	at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:388)
> 	at java.lang.Thread.run(Thread.java:619)
>
>
>
>
>
>