You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2009/01/18 02:42:59 UTC
[jira] Created: (HBASE-1133) Master does not reload META on IOE
Master does not reload META on IOE
----------------------------------
Key: HBASE-1133
URL: https://issues.apache.org/jira/browse/HBASE-1133
Project: Hadoop HBase
Issue Type: Bug
Reporter: Andrew Purtell
This never recalibrates:
2009-01-18 01:35:30,906 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan one META region: {regionname: .META.,,1, startKey: <>, server: 10.30.94.35:60020}
java.io.IOException: java.io.IOException: All datanodes 10.30.94.34:50010 are bad. Aborting...
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2443)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:1995)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2159)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:95)
at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1133) Master does not reload META on IOE
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664905#action_12664905 ]
stack commented on HBASE-1133:
------------------------------
Usually means no datanodes running? Is that your case? How'd it happen?
> Master does not reload META on IOE
> ----------------------------------
>
> Key: HBASE-1133
> URL: https://issues.apache.org/jira/browse/HBASE-1133
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
>
> This never recalibrates:
> 2009-01-18 01:35:30,906 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan one META region: {regionname: .META.,,1, startKey: <>, server: 10.30.94.35:60020}
> java.io.IOException: java.io.IOException: All datanodes 10.30.94.34:50010 are bad. Aborting...
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2443)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:1995)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2159)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:95)
> at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1133) Master does not reload META on IOE
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664920#action_12664920 ]
stack commented on HBASE-1133:
------------------------------
I think we'd need HBASE-1084
> Master does not reload META on IOE
> ----------------------------------
>
> Key: HBASE-1133
> URL: https://issues.apache.org/jira/browse/HBASE-1133
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
>
> This never recalibrates:
> 2009-01-18 01:35:30,906 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan one META region: {regionname: .META.,,1, startKey: <>, server: 10.30.94.35:60020}
> java.io.IOException: java.io.IOException: All datanodes 10.30.94.34:50010 are bad. Aborting...
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2443)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:1995)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2159)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:95)
> at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1133) Master does not reload META on IOE
Posted by "Andrew Purtell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664910#action_12664910 ]
Andrew Purtell commented on HBASE-1133:
---------------------------------------
Needed to up FD limit again. Meantime, datanodes had trouble and some were restarted. After, all datanodes were up and reporting to namenode within time limits again, but MetaScanner was unhappy, throwing the IOE above. Is there anything that can be done on IOE in MetaScanner like this? If not, never mind.
> Master does not reload META on IOE
> ----------------------------------
>
> Key: HBASE-1133
> URL: https://issues.apache.org/jira/browse/HBASE-1133
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: Andrew Purtell
>
> This never recalibrates:
> 2009-01-18 01:35:30,906 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan one META region: {regionname: .META.,,1, startKey: <>, server: 10.30.94.35:60020}
> java.io.IOException: java.io.IOException: All datanodes 10.30.94.34:50010 are bad. Aborting...
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2443)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1600(DFSClient.java:1995)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2159)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:95)
> at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
> ...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.