You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Sanford Rockowitz <ro...@minsoft.com> on 2010/12/12 07:41:32 UTC

exceptions copying files into HDFS

Folks,

I'm a Hadoop newbie, and I hope this is an appropriate place to post 
this question.

I'm trying to work through the initial examples.  When I try to copy 
files into HDFS, hadoop throws exceptions.   I imagine it's something in 
my configuration, but I'm at a loss to figure out what.

I'm running on openSuSE 11.3, using Oracle Java 1.6.0_23.  The problem 
occurs whether I use 32 bit or 64 bit Java.   The problem occurs in both 
vanilla Apache hadoop 0.20.2 and Cloudera's 0.20.2+737.

Following are the console output, the datanode log file, and the 
relevant configuration files.

Thanks in advance for any pointers.

Sanford

=== CONSOLE ===

rock@ritter:~/programs/hadoop-0.20.2+737> hadoop fs -put conf input
10/12/11 21:04:41 INFO hdfs.DFSClient: Exception in 
createBlockOutputStream java.io.EOFException
10/12/11 21:04:41 INFO hdfs.DFSClient: Abandoning block 
blk_1699203955671139323_1010
10/12/11 21:04:41 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010
10/12/11 21:04:41 WARN hdfs.DFSClient: DataStreamer Exception: 
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
/user/rock/input/fair-scheduler.xml could only be replicated to 0 nodes, 
instead of 1
         at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415)
         at 
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
         at java.security.AccessController.doPrivileged(Native Method)
         at javax.security.auth.Subject.doAs(Subject.java:396)
         at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)

         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
         at $Proxy0.addBlock(Unknown Source)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
         at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
         at $Proxy0.addBlock(Unknown Source)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483)

10/12/11 21:04:41 WARN hdfs.DFSClient: Error Recovery for block 
blk_1699203955671139323_1010 bad datanode[0] nodes == null
10/12/11 21:04:41 WARN hdfs.DFSClient: Could not get block locations. 
Source file "/user/rock/input/fair-scheduler.xml" - Aborting...
put: java.io.IOException: File /user/rock/input/fair-scheduler.xml could 
only be replicated to 0 nodes, instead of 1
10/12/11 21:04:41 ERROR hdfs.DFSClient: Exception closing file 
/user/rock/input/fair-scheduler.xml : 
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
/user/rock/input/fair-scheduler.xml could only be replicated to 0 nodes, 
instead of 1
         at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415)
         at 
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
         at java.security.AccessController.doPrivileged(Native Method)
         at javax.security.auth.Subject.doAs(Subject.java:396)
         at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
/user/rock/input/fair-scheduler.xml could only be replicated to 0 nodes, 
instead of 1
         at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415)
         at 
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
         at java.security.AccessController.doPrivileged(Native Method)
         at javax.security.auth.Subject.doAs(Subject.java:396)
         at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)

         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
         at $Proxy0.addBlock(Unknown Source)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
         at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
         at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
         at $Proxy0.addBlock(Unknown Source)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288)
         at 
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483)
rock@ritter:~/programs/hadoop-0.20.2+737>


=== DATANODE LOG ===

And here's the the corresponding contents of the datanode log:
10-12-11 21:02:37,541 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG:   host = ritter.minsoft.com/127.0.0.2
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 0.20.2+737
STARTUP_MSG:   build = git://bruno-desktop/ on branch  -r 
98c55c28258aa6f42250569bd7fa431ac657bdbd; compiled by 'bruno' on Mon Oct 
11 09:37:19 PDT 2010
************************************************************/
2010-12-11 21:02:42,046 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: Registered 
FSDatasetStatusMBean
2010-12-11 21:02:42,047 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 50010
2010-12-11 21:02:42,049 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 
1048576 bytes/s
2010-12-11 21:02:42,085 INFO org.mortbay.log: Logging to 
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
org.mortbay.log.Slf4jLog
2010-12-11 21:02:42,124 INFO org.apache.hadoop.http.HttpServer: Added 
global filtersafety 
(class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: Port 
returned by webServer.getConnectors()[0].getLocalPort() before open() is 
-1. Opening the listener on 50075
2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: 
listener.getLocalPort() returned 50075 
webServer.getConnectors()[0].getLocalPort() returned 50075
2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: Jetty 
bound to port 50075
2010-12-11 21:02:42,130 INFO org.mortbay.log: jetty-6.1.14
2010-12-11 21:02:47,772 INFO org.mortbay.log: Started 
SelectChannelConnector@0.0.0.0:50075
2010-12-11 21:02:47,782 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: 
Initializing JVM Metrics with processName=DataNode, sessionId=null
2010-12-11 21:02:47,797 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: 
Initializing RPC Metrics with hostName=DataNode, port=50020
2010-12-11 21:02:47,798 INFO 
org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC 
Metrics with hostName=DataNode, port=50020
2010-12-11 21:02:47,800 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = 
DatanodeRegistration(ritter.minsoft.com:50010, 
storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
ipcPort=50020)
2010-12-11 21:02:47,813 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: 
DatanodeRegistration(127.0.0.1:50010, 
storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
ipcPort=50020)In DataNode.run, data = 
FSDataset{dirpath='/tmp/hadoop-rock/dfs/data/current'}
2010-12-11 21:02:47,816 INFO org.apache.hadoop.ipc.Server: IPC Server 
Responder: starting
2010-12-11 21:02:47,818 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: using 
BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec
2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
listener on 50020: starting
2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
handler 1 on 50020: starting
2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
handler 2 on 50020: starting
2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
handler 0 on 50020: starting
2010-12-11 21:02:47,827 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 0 blocks 
got processed in 6 msecs
2010-12-11 21:02:47,827 INFO 
org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic block 
scanner.
2010-12-11 21:04:41,371 ERROR 
org.apache.hadoop.hdfs.server.datanode.DataNode: 
DatanodeRegistration(127.0.0.1:50010, 
storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
ipcPort=50020):DataXceiver
java.net.SocketException: Operation not supported
         at sun.nio.ch.Net.getIntOption0(Native Method)
         at sun.nio.ch.Net.getIntOption(Net.java:181)
         at 
sun.nio.ch.SocketChannelImpl$1.getInt(SocketChannelImpl.java:419)
         at sun.nio.ch.SocketOptsImpl.getInt(SocketOptsImpl.java:60)
         at 
sun.nio.ch.SocketOptsImpl.receiveBufferSize(SocketOptsImpl.java:142)
         at 
sun.nio.ch.SocketOptsImpl$IP$TCP.receiveBufferSize(SocketOptsImpl.java:286)
         at 
sun.nio.ch.OptionAdaptor.getReceiveBufferSize(OptionAdaptor.java:148)
         at 
sun.nio.ch.SocketAdaptor.getReceiveBufferSize(SocketAdaptor.java:336)
         at 
org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:255)
         at 
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:122)

=== CONFIG FILES ===

rock@ritter:~/programs/hadoop-0.20.2+737/conf> cat core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost</value>
<!-- default port 8020 -->
</property>
</configuration>


rock@ritter:~/programs/hadoop-0.20.2+737/conf> cat hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

Re: exceptions copying files into HDFS

Posted by Sanford Rockowitz <ro...@minsoft.com>.
As I posted the problem about getting HDFS to work in pseudo-distributed 
mode, I should post the solution as well.  Apparently, the Java 
environment (JAVA_HOME, etc) was not set up  properly on the daemons, 
which in hindsight explains the exceptions in the Java NIO Socket code.  
I moved the definitions of JAVA_HOME, HADOOP_INSTALL, and PATH from 
.profile to .bashrc to ensure they get set for each shell, and the 
problems resolved.

Sanford


On 12/11/2010 10:41 PM, Sanford Rockowitz wrote:
> Folks,
>
> I'm a Hadoop newbie, and I hope this is an appropriate place to post 
> this question.
>
> I'm trying to work through the initial examples.  When I try to copy 
> files into HDFS, hadoop throws exceptions.   I imagine it's something 
> in my configuration, but I'm at a loss to figure out what.
>
> I'm running on openSuSE 11.3, using Oracle Java 1.6.0_23.  The problem 
> occurs whether I use 32 bit or 64 bit Java.   The problem occurs in 
> both vanilla Apache hadoop 0.20.2 and Cloudera's 0.20.2+737.
>
> Following are the console output, the datanode log file, and the 
> relevant configuration files.
>
> Thanks in advance for any pointers.
>
> Sanford
>
> === CONSOLE ===
>
> rock@ritter:~/programs/hadoop-0.20.2+737> hadoop fs -put conf input
> 10/12/11 21:04:41 INFO hdfs.DFSClient: Exception in 
> createBlockOutputStream java.io.EOFException
> 10/12/11 21:04:41 INFO hdfs.DFSClient: Abandoning block 
> blk_1699203955671139323_1010
> 10/12/11 21:04:41 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010
> 10/12/11 21:04:41 WARN hdfs.DFSClient: DataStreamer Exception: 
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
> /user/rock/input/fair-scheduler.xml could only be replicated to 0 
> nodes, instead of 1
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415)
>         at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>
>         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>         at $Proxy0.addBlock(Unknown Source)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>         at $Proxy0.addBlock(Unknown Source)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483)
>
> 10/12/11 21:04:41 WARN hdfs.DFSClient: Error Recovery for block 
> blk_1699203955671139323_1010 bad datanode[0] nodes == null
> 10/12/11 21:04:41 WARN hdfs.DFSClient: Could not get block locations. 
> Source file "/user/rock/input/fair-scheduler.xml" - Aborting...
> put: java.io.IOException: File /user/rock/input/fair-scheduler.xml 
> could only be replicated to 0 nodes, instead of 1
> 10/12/11 21:04:41 ERROR hdfs.DFSClient: Exception closing file 
> /user/rock/input/fair-scheduler.xml : 
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
> /user/rock/input/fair-scheduler.xml could only be replicated to 0 
> nodes, instead of 1
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415)
>         at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
> /user/rock/input/fair-scheduler.xml could only be replicated to 0 
> nodes, instead of 1
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415)
>         at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>
>         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>         at $Proxy0.addBlock(Unknown Source)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>         at $Proxy0.addBlock(Unknown Source)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483)
> rock@ritter:~/programs/hadoop-0.20.2+737>
>
>
> === DATANODE LOG ===
>
> And here's the the corresponding contents of the datanode log:
> 10-12-11 21:02:37,541 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting DataNode
> STARTUP_MSG:   host = ritter.minsoft.com/127.0.0.2
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 0.20.2+737
> STARTUP_MSG:   build = git://bruno-desktop/ on branch  -r 
> 98c55c28258aa6f42250569bd7fa431ac657bdbd; compiled by 'bruno' on Mon 
> Oct 11 09:37:19 PDT 2010
> ************************************************************/
> 2010-12-11 21:02:42,046 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Registered 
> FSDatasetStatusMBean
> 2010-12-11 21:02:42,047 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 
> 50010
> 2010-12-11 21:02:42,049 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 
> 1048576 bytes/s
> 2010-12-11 21:02:42,085 INFO org.mortbay.log: Logging to 
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
> org.mortbay.log.Slf4jLog
> 2010-12-11 21:02:42,124 INFO org.apache.hadoop.http.HttpServer: Added 
> global filtersafety 
> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
> 2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: Port 
> returned by webServer.getConnectors()[0].getLocalPort() before open() 
> is -1. Opening the listener on 50075
> 2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: 
> listener.getLocalPort() returned 50075 
> webServer.getConnectors()[0].getLocalPort() returned 50075
> 2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: Jetty 
> bound to port 50075
> 2010-12-11 21:02:42,130 INFO org.mortbay.log: jetty-6.1.14
> 2010-12-11 21:02:47,772 INFO org.mortbay.log: Started 
> SelectChannelConnector@0.0.0.0:50075
> 2010-12-11 21:02:47,782 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: 
> Initializing JVM Metrics with processName=DataNode, sessionId=null
> 2010-12-11 21:02:47,797 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: 
> Initializing RPC Metrics with hostName=DataNode, port=50020
> 2010-12-11 21:02:47,798 INFO 
> org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC 
> Metrics with hostName=DataNode, port=50020
> 2010-12-11 21:02:47,800 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = 
> DatanodeRegistration(ritter.minsoft.com:50010, 
> storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
> ipcPort=50020)
> 2010-12-11 21:02:47,813 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: 
> DatanodeRegistration(127.0.0.1:50010, 
> storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
> ipcPort=50020)In DataNode.run, data = 
> FSDataset{dirpath='/tmp/hadoop-rock/dfs/data/current'}
> 2010-12-11 21:02:47,816 INFO org.apache.hadoop.ipc.Server: IPC Server 
> Responder: starting
> 2010-12-11 21:02:47,818 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: using 
> BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> listener on 50020: starting
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> handler 1 on 50020: starting
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> handler 2 on 50020: starting
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> handler 0 on 50020: starting
> 2010-12-11 21:02:47,827 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 0 
> blocks got processed in 6 msecs
> 2010-12-11 21:02:47,827 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic 
> block scanner.
> 2010-12-11 21:04:41,371 ERROR 
> org.apache.hadoop.hdfs.server.datanode.DataNode: 
> DatanodeRegistration(127.0.0.1:50010, 
> storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
> ipcPort=50020):DataXceiver
> java.net.SocketException: Operation not supported
>         at sun.nio.ch.Net.getIntOption0(Native Method)
>         at sun.nio.ch.Net.getIntOption(Net.java:181)
>         at 
> sun.nio.ch.SocketChannelImpl$1.getInt(SocketChannelImpl.java:419)
>         at sun.nio.ch.SocketOptsImpl.getInt(SocketOptsImpl.java:60)
>         at 
> sun.nio.ch.SocketOptsImpl.receiveBufferSize(SocketOptsImpl.java:142)
>         at 
> sun.nio.ch.SocketOptsImpl$IP$TCP.receiveBufferSize(SocketOptsImpl.java:286)
>         at 
> sun.nio.ch.OptionAdaptor.getReceiveBufferSize(OptionAdaptor.java:148)
>         at 
> sun.nio.ch.SocketAdaptor.getReceiveBufferSize(SocketAdaptor.java:336)
>         at 
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:255)
>         at 
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:122)
>
> === CONFIG FILES ===
>
> rock@ritter:~/programs/hadoop-0.20.2+737/conf> cat core-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost</value>
> <!-- default port 8020 -->
> </property>
> </configuration>
>
>
> rock@ritter:~/programs/hadoop-0.20.2+737/conf> cat hdfs-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> </configuration>
>
>
>

Re: exceptions copying files into HDFS

Posted by Adarsh Sharma <ad...@orkash.com>.
Sanford Rockowitz wrote:
> Folks,
>
> I'm a Hadoop newbie, and I hope this is an appropriate place to post 
> this question.
>
> I'm trying to work through the initial examples.  When I try to copy 
> files into HDFS, hadoop throws exceptions.   I imagine it's something 
> in my configuration, but I'm at a loss to figure out what.
>
> I'm running on openSuSE 11.3, using Oracle Java 1.6.0_23.  The problem 
> occurs whether I use 32 bit or 64 bit Java.   The problem occurs in 
> both vanilla Apache hadoop 0.20.2 and Cloudera's 0.20.2+737.
>
> Following are the console output, the datanode log file, and the 
> relevant configuration files.
>
> Thanks in advance for any pointers.
>
> Sanford
>
> === CONSOLE ===
>
> rock@ritter:~/programs/hadoop-0.20.2+737> hadoop fs -put conf input
> 10/12/11 21:04:41 INFO hdfs.DFSClient: Exception in 
> createBlockOutputStream java.io.EOFException
> 10/12/11 21:04:41 INFO hdfs.DFSClient: Abandoning block 
> blk_1699203955671139323_1010
> 10/12/11 21:04:41 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010
> 10/12/11 21:04:41 WARN hdfs.DFSClient: DataStreamer Exception: 
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
> /user/rock/input/fair-scheduler.xml could only be replicated to 0 
> nodes, instead of 1
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415) 
>
>         at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588) 
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063) 
>
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>
>         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>         at $Proxy0.addBlock(Unknown Source)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) 
>
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) 
>
>         at $Proxy0.addBlock(Unknown Source)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166) 
>
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036) 
>
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288) 
>
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483) 
>
>
> 10/12/11 21:04:41 WARN hdfs.DFSClient: Error Recovery for block 
> blk_1699203955671139323_1010 bad datanode[0] nodes == null
> 10/12/11 21:04:41 WARN hdfs.DFSClient: Could not get block locations. 
> Source file "/user/rock/input/fair-scheduler.xml" - Aborting...
> put: java.io.IOException: File /user/rock/input/fair-scheduler.xml 
> could only be replicated to 0 nodes, instead of 1
> 10/12/11 21:04:41 ERROR hdfs.DFSClient: Exception closing file 
> /user/rock/input/fair-scheduler.xml : 
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
> /user/rock/input/fair-scheduler.xml could only be replicated to 0 
> nodes, instead of 1
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415) 
>
>         at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588) 
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063) 
>
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
> /user/rock/input/fair-scheduler.xml could only be replicated to 0 
> nodes, instead of 1
>         at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1415) 
>
>         at 
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:588) 
>
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:528)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1319)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1315)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063) 
>
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1313)
>
>         at org.apache.hadoop.ipc.Client.call(Client.java:1054)
>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
>         at $Proxy0.addBlock(Unknown Source)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) 
>
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
>
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) 
>
>         at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) 
>
>         at $Proxy0.addBlock(Unknown Source)
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3166) 
>
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3036) 
>
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$1900(DFSClient.java:2288) 
>
>         at 
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2483) 
>
> rock@ritter:~/programs/hadoop-0.20.2+737>
>
>
> === DATANODE LOG ===
>
> And here's the the corresponding contents of the datanode log:
> 10-12-11 21:02:37,541 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting DataNode
> STARTUP_MSG:   host = ritter.minsoft.com/127.0.0.2
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 0.20.2+737
> STARTUP_MSG:   build = git://bruno-desktop/ on branch  -r 
> 98c55c28258aa6f42250569bd7fa431ac657bdbd; compiled by 'bruno' on Mon 
> Oct 11 09:37:19 PDT 2010
> ************************************************************/
> 2010-12-11 21:02:42,046 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Registered 
> FSDatasetStatusMBean
> 2010-12-11 21:02:42,047 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at 
> 50010
> 2010-12-11 21:02:42,049 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 
> 1048576 bytes/s
> 2010-12-11 21:02:42,085 INFO org.mortbay.log: Logging to 
> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
> org.mortbay.log.Slf4jLog
> 2010-12-11 21:02:42,124 INFO org.apache.hadoop.http.HttpServer: Added 
> global filtersafety 
> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
> 2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: Port 
> returned by webServer.getConnectors()[0].getLocalPort() before open() 
> is -1. Opening the listener on 50075
> 2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: 
> listener.getLocalPort() returned 50075 
> webServer.getConnectors()[0].getLocalPort() returned 50075
> 2010-12-11 21:02:42,130 INFO org.apache.hadoop.http.HttpServer: Jetty 
> bound to port 50075
> 2010-12-11 21:02:42,130 INFO org.mortbay.log: jetty-6.1.14
> 2010-12-11 21:02:47,772 INFO org.mortbay.log: Started 
> SelectChannelConnector@0.0.0.0:50075
> 2010-12-11 21:02:47,782 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: 
> Initializing JVM Metrics with processName=DataNode, sessionId=null
> 2010-12-11 21:02:47,797 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: 
> Initializing RPC Metrics with hostName=DataNode, port=50020
> 2010-12-11 21:02:47,798 INFO 
> org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC 
> Metrics with hostName=DataNode, port=50020
> 2010-12-11 21:02:47,800 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: dnRegistration = 
> DatanodeRegistration(ritter.minsoft.com:50010, 
> storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
> ipcPort=50020)
> 2010-12-11 21:02:47,813 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: 
> DatanodeRegistration(127.0.0.1:50010, 
> storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
> ipcPort=50020)In DataNode.run, data = 
> FSDataset{dirpath='/tmp/hadoop-rock/dfs/data/current'}
> 2010-12-11 21:02:47,816 INFO org.apache.hadoop.ipc.Server: IPC Server 
> Responder: starting
> 2010-12-11 21:02:47,818 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: using 
> BLOCKREPORT_INTERVAL of 3600000msec Initial delay: 0msec
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> listener on 50020: starting
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> handler 1 on 50020: starting
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> handler 2 on 50020: starting
> 2010-12-11 21:02:47,819 INFO org.apache.hadoop.ipc.Server: IPC Server 
> handler 0 on 50020: starting
> 2010-12-11 21:02:47,827 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: BlockReport of 0 
> blocks got processed in 6 msecs
> 2010-12-11 21:02:47,827 INFO 
> org.apache.hadoop.hdfs.server.datanode.DataNode: Starting Periodic 
> block scanner.
> 2010-12-11 21:04:41,371 ERROR 
> org.apache.hadoop.hdfs.server.datanode.DataNode: 
> DatanodeRegistration(127.0.0.1:50010, 
> storageID=DS-1618752214-127.0.0.2-50010-1292091159510, infoPort=50075, 
> ipcPort=50020):DataXceiver
> java.net.SocketException: Operation not supported
>         at sun.nio.ch.Net.getIntOption0(Native Method)
>         at sun.nio.ch.Net.getIntOption(Net.java:181)
>         at 
> sun.nio.ch.SocketChannelImpl$1.getInt(SocketChannelImpl.java:419)
>         at sun.nio.ch.SocketOptsImpl.getInt(SocketOptsImpl.java:60)
>         at 
> sun.nio.ch.SocketOptsImpl.receiveBufferSize(SocketOptsImpl.java:142)
>         at 
> sun.nio.ch.SocketOptsImpl$IP$TCP.receiveBufferSize(SocketOptsImpl.java:286) 
>
>         at 
> sun.nio.ch.OptionAdaptor.getReceiveBufferSize(OptionAdaptor.java:148)
>         at 
> sun.nio.ch.SocketAdaptor.getReceiveBufferSize(SocketAdaptor.java:336)
>         at 
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:255) 
>
>         at 
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:122) 
>
>
> === CONFIG FILES ===
>
> rock@ritter:~/programs/hadoop-0.20.2+737/conf> cat core-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost</value>
> <!-- default port 8020 -->
> </property>
> </configuration>
>
>
> rock@ritter:~/programs/hadoop-0.20.2+737/conf> cat hdfs-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <!-- Put site-specific property overrides in this file. -->
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> </configuration>
>
Simply Check through ssh that your slaves are connecting to each other.

ssh from 1 slave to another.

Best Regards

Adarsh Sharma

Re: exceptions copying files into HDFS

Posted by li ping <li...@gmail.com>.
That's right.

You have to make sure the datanode is running.
If you are using the virtual machine, like Virtual-box, sometime, you should
wait for a moment until the datanode is active. seems like the performance
issue, the datanode in vm will be active after several mins.

On Sun, Dec 12, 2010 at 2:48 PM, Varadharajan Mukundan <srinathsmn@gmail.com
> wrote:

> Hi,
>
> > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> > /user/rock/input/fair-scheduler.xml could only be replicated to 0 nodes,
> > instead of 1
>
> I think none of your datanodes are actually running. why not use jps
> and make sure whether they are running. Also check the datanode logs
> in the nodes.
>
> --
> Thanks,
> M. Varadharajan
>
> ------------------------------------------------
>
> "Experience is what you get when you didn't get what you wanted"
>                -By Prof. Randy Pausch in "The Last Lecture"
>
> My Journal :- www.thinkasgeek.wordpress.com
>



-- 
-----李平

Re: exceptions copying files into HDFS

Posted by rahul patodi <pa...@gmail.com>.
Sanford,.
I have read ur previous posts, also blog URL given by me also contain
configuration for running hadoop in pseudo distributed mode
Also the exception you are getting is because your data node is down
I would suggest please start from scratch
*To be more specific* if you need quick install tutorial:
for hadoop:
http://hadoop-tutorial.blogspot.com/2010/11/running-hadoop-in-pseudo-distributed.html
for cloudera:
http://cloudera-tutorial.blogspot.com/2010/11/running-cloudera-in-pseudo-distributed.html

On Sun, Dec 12, 2010 at 11:12 PM, Sanford Rockowitz
<ro...@minsoft.com>wrote:

> Rahul,
>
> I should have been more explicit.  I am simply trying to run in
> pseudo-distributed mode.   For further comments, see my previous post to
> Varadharajan.
>
> Thanks,
> Sanford
>
>
> On 12/12/2010 2:24 AM, rahul patodi wrote:
>
>> you can follow this tutorial:
>>
>>
>> http://hadoop-tutorial.blogspot.com/2010/11/running-hadoop-in-distributed-mode.html
>>
>> http://cloudera-tutorial.blogspot.com/2010/11/running-cloudera-in-distributed-mode.html
>> also, before running any job please ensure all the required processes are
>> running on the correct node
>> like on master:
>> Namenode, jobtracker, secondarynamenode(if you are not running secondary
>> name node on another system)
>>
>> on slave:
>> datanode, tasktracker
>>
>>
>> On Sun, Dec 12, 2010 at 2:46 PM, Varadharajan Mukundan<
>> srinathsmn@gmail.com
>>
>>> wrote:
>>> HI,
>>>
>>>  jps reports DataNode, NameNode, and SecondayNameNode as running:
>>>>
>>>> rock@ritter:/tmp/hadoop-rock>  jps
>>>> 31177 Jps
>>>> 29909 DataNode
>>>> 29751 NameNode
>>>> 30052 SecondaryNameNode
>>>>
>>> In master node, the output of the "JPS" will contain a tasktracker,
>>> jobtracker, namenode, secondary namenode, datanode(optional, depending on
>>> your config) and your slaves will have tasktracker, datanodes in their
>>> jps
>>> output. If you need more help on configuring hadoop, i recommend you to
>>> take
>>> a look at
>>>
>>>
>>> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
>>>
>>>
>>>
>>>
>>>  Here is the contents of the Hadoop node tree.  The only thing that looks
>>>> like a log file are the dncp_block_verification.log.curr files, and
>>>> those
>>>> are empty.
>>>> Note the presence of the in_use.lock files, which suggests that this
>>>> node
>>>>
>>> is
>>>
>>>> indeed being used.
>>>>
>>>
>>> The logs will be in the "logs" directory in $HADOOP_HOME (hadoop home
>>> directory), are you looking for logs in this directory?
>>>
>>>
>>> --
>>> Thanks,
>>> M. Varadharajan
>>>
>>> ------------------------------------------------
>>>
>>> "Experience is what you get when you didn't get what you wanted"
>>>               -By Prof. Randy Pausch in "The Last Lecture"
>>>
>>> My Journal :- www.thinkasgeek.wordpress.com
>>>
>>>
>>
>>
>


-- 
*Regards*,
Rahul Patodi
Associate Software Engineer,
Impetus Infotech (India) Pvt Ltd,
www.impetus.com
Mob:09907074413

Re: exceptions copying files into HDFS

Posted by Sanford Rockowitz <ro...@minsoft.com>.
Rahul,

I should have been more explicit.  I am simply trying to run in 
pseudo-distributed mode.   For further comments, see my previous post to 
Varadharajan.

Thanks,
Sanford

On 12/12/2010 2:24 AM, rahul patodi wrote:
> you can follow this tutorial:
>
> http://hadoop-tutorial.blogspot.com/2010/11/running-hadoop-in-distributed-mode.html
> http://cloudera-tutorial.blogspot.com/2010/11/running-cloudera-in-distributed-mode.html
> also, before running any job please ensure all the required processes are
> running on the correct node
> like on master:
> Namenode, jobtracker, secondarynamenode(if you are not running secondary
> name node on another system)
>
> on slave:
> datanode, tasktracker
>
>
> On Sun, Dec 12, 2010 at 2:46 PM, Varadharajan Mukundan<srinathsmn@gmail.com
>> wrote:
>> HI,
>>
>>> jps reports DataNode, NameNode, and SecondayNameNode as running:
>>>
>>> rock@ritter:/tmp/hadoop-rock>  jps
>>> 31177 Jps
>>> 29909 DataNode
>>> 29751 NameNode
>>> 30052 SecondaryNameNode
>> In master node, the output of the "JPS" will contain a tasktracker,
>> jobtracker, namenode, secondary namenode, datanode(optional, depending on
>> your config) and your slaves will have tasktracker, datanodes in their jps
>> output. If you need more help on configuring hadoop, i recommend you to
>> take
>> a look at
>>
>> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
>>
>>
>>
>>
>>> Here is the contents of the Hadoop node tree.  The only thing that looks
>>> like a log file are the dncp_block_verification.log.curr files, and those
>>> are empty.
>>> Note the presence of the in_use.lock files, which suggests that this node
>> is
>>> indeed being used.
>>
>> The logs will be in the "logs" directory in $HADOOP_HOME (hadoop home
>> directory), are you looking for logs in this directory?
>>
>>
>> --
>> Thanks,
>> M. Varadharajan
>>
>> ------------------------------------------------
>>
>> "Experience is what you get when you didn't get what you wanted"
>>                -By Prof. Randy Pausch in "The Last Lecture"
>>
>> My Journal :- www.thinkasgeek.wordpress.com
>>
>
>

Re: exceptions copying files into HDFS

Posted by rahul patodi <pa...@gmail.com>.
you can follow this tutorial:

http://hadoop-tutorial.blogspot.com/2010/11/running-hadoop-in-distributed-mode.html
http://cloudera-tutorial.blogspot.com/2010/11/running-cloudera-in-distributed-mode.html
also, before running any job please ensure all the required processes are
running on the correct node
like on master:
Namenode, jobtracker, secondarynamenode(if you are not running secondary
name node on another system)

on slave:
datanode, tasktracker


On Sun, Dec 12, 2010 at 2:46 PM, Varadharajan Mukundan <srinathsmn@gmail.com
> wrote:

> HI,
>
> > jps reports DataNode, NameNode, and SecondayNameNode as running:
> >
> > rock@ritter:/tmp/hadoop-rock> jps
> > 31177 Jps
> > 29909 DataNode
> > 29751 NameNode
> > 30052 SecondaryNameNode
>
> In master node, the output of the "JPS" will contain a tasktracker,
> jobtracker, namenode, secondary namenode, datanode(optional, depending on
> your config) and your slaves will have tasktracker, datanodes in their jps
> output. If you need more help on configuring hadoop, i recommend you to
> take
> a look at
>
> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
>
>
>
>
> > Here is the contents of the Hadoop node tree.  The only thing that looks
> > like a log file are the dncp_block_verification.log.curr files, and those
> > are empty.
> > Note the presence of the in_use.lock files, which suggests that this node
> is
> > indeed being used.
>
>
> The logs will be in the "logs" directory in $HADOOP_HOME (hadoop home
> directory), are you looking for logs in this directory?
>
>
> --
> Thanks,
> M. Varadharajan
>
> ------------------------------------------------
>
> "Experience is what you get when you didn't get what you wanted"
>               -By Prof. Randy Pausch in "The Last Lecture"
>
> My Journal :- www.thinkasgeek.wordpress.com
>



-- 
-Thanks and Regards,
Rahul Patodi
Associate Software Engineer,
Impetus Infotech (India) Private Limited,
www.impetus.com
Mob:09907074413

Re: exceptions copying files into HDFS

Posted by Sanford Rockowitz <ro...@minsoft.com>.
Varadharajan,

I should have been more explicit in pointing out that I am trying to run 
in pseudo-distributed mode.  The dfs-replication value in hdfs-site.xml 
is 1.

I had looked at Michael Noll's tutorial, and the only differences I saw 
from the Apache basic tutorial for pseudo-distributed mode are:
  - running Hadoop on user hadoop rather than the logged on user
  - explicit port settings and hadoop directory name settings in the 
configuration files
  - the files being copied into HDFS for the example

I have changed the configuration files to use Noll's ports and explicit 
directory name.   The result is unchanged.

Since all I am trying to do run the command:
   hadoop fs -put conf conf input

it seemed to me that for testing purposes all I need to do was start the 
hdfs daemons using the command:
   start-dfs.sh

JobTracker and TaskTracker should be irrelevant for my testcase.   
However, I have started all the daemons with the command
   start-all.sh

The result is unchanged.

In reading the logs it seems to me that the key error is the datanode 
log, where I find
    SocketException: operation not supported
on SocketAdapater.getReceiveBufferSize() calls within 
DataXceiver.writeBlock()

Thanks for your comments.

Sanford





On 12/12/2010 1:16 AM, Varadharajan Mukundan wrote:
> HI,
>
>> jps reports DataNode, NameNode, and SecondayNameNode as running:
>>
>> rock@ritter:/tmp/hadoop-rock>  jps
>> 31177 Jps
>> 29909 DataNode
>> 29751 NameNode
>> 30052 SecondaryNameNode
> In master node, the output of the "JPS" will contain a tasktracker,
> jobtracker, namenode, secondary namenode, datanode(optional, depending on
> your config) and your slaves will have tasktracker, datanodes in their jps
> output. If you need more help on configuring hadoop, i recommend you to take
> a look at
> http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
>
>
>
>
>> Here is the contents of the Hadoop node tree.  The only thing that looks
>> like a log file are the dncp_block_verification.log.curr files, and those
>> are empty.
>> Note the presence of the in_use.lock files, which suggests that this node
> is
>> indeed being used.
>
> The logs will be in the "logs" directory in $HADOOP_HOME (hadoop home
> directory), are you looking for logs in this directory?
>
>

Re: exceptions copying files into HDFS

Posted by Varadharajan Mukundan <sr...@gmail.com>.
HI,

> jps reports DataNode, NameNode, and SecondayNameNode as running:
>
> rock@ritter:/tmp/hadoop-rock> jps
> 31177 Jps
> 29909 DataNode
> 29751 NameNode
> 30052 SecondaryNameNode

In master node, the output of the "JPS" will contain a tasktracker,
jobtracker, namenode, secondary namenode, datanode(optional, depending on
your config) and your slaves will have tasktracker, datanodes in their jps
output. If you need more help on configuring hadoop, i recommend you to take
a look at
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/




> Here is the contents of the Hadoop node tree.  The only thing that looks
> like a log file are the dncp_block_verification.log.curr files, and those
> are empty.
> Note the presence of the in_use.lock files, which suggests that this node
is
> indeed being used.


The logs will be in the "logs" directory in $HADOOP_HOME (hadoop home
directory), are you looking for logs in this directory?


-- 
Thanks,
M. Varadharajan

------------------------------------------------

"Experience is what you get when you didn't get what you wanted"
               -By Prof. Randy Pausch in "The Last Lecture"

My Journal :- www.thinkasgeek.wordpress.com

Re: exceptions copying files into HDFS

Posted by Sanford Rockowitz <ro...@minsoft.com>.
On 12/11/2010 10:48 PM, Varadharajan Mukundan wrote:
> Hi,
>
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
>> /user/rock/input/fair-scheduler.xml could only be replicated to 0 nodes,
>> instead of 1
> I think none of your datanodes are actually running. why not use jps
> and make sure whether they are running. Also check the datanode logs
> in the nodes.
>


jps reports DataNode, NameNode, and SecondayNameNode as running:

rock@ritter:/tmp/hadoop-rock> jps
31177 Jps
29909 DataNode
29751 NameNode
30052 SecondaryNameNode
rock@ritter:/tmp/hadoop-rock>


Here is the contents of the Hadoop node tree.  The only thing that looks 
like a log file are the dncp_block_verification.log.curr files, and 
those are empty.
Note the presence of the in_use.lock files, which suggests that this 
node is indeed being used.



rock@ritter:/tmp/hadoop-rock> ls -lR
.:
total 4
drwxr-xr-x 5 rock users 4096 2010-12-11 10:12 dfs

./dfs:
total 12
drwxr-xr-x 6 rock users 4096 2010-12-11 21:02 data
drwxr-xr-x 5 rock users 4096 2010-12-11 21:02 name
drwxr-xr-x 4 rock users 4096 2010-12-11 23:08 namesecondary

./dfs/data:
total 20
drwxr-xr-x 2 rock users 4096 2010-12-11 10:12 blocksBeingWritten
drwxr-xr-x 2 rock users 4096 2010-12-11 10:12 current
drwxr-xr-x 2 rock users 4096 2010-12-11 10:12 detach
-rw-r--r-- 1 rock users    0 2010-12-11 21:02 in_use.lock
-rw-r--r-- 1 rock users  157 2010-12-11 10:12 storage
drwxr-xr-x 2 rock users 4096 2010-12-11 21:02 tmp

./dfs/data/blocksBeingWritten:
total 0

./dfs/data/current:
total 4
-rw-r--r-- 1 rock users   0 2010-12-11 10:12 
dncp_block_verification.log.curr
-rw-r--r-- 1 rock users 154 2010-12-11 21:02 VERSION

./dfs/data/detach:
total 0

./dfs/data/tmp:
total 0

./dfs/name:
total 12
drwxr-xr-x 2 rock users 4096 2010-12-11 23:08 current
drwxr-xr-x 2 rock users 4096 2010-12-11 10:12 image
-rw-r--r-- 1 rock users    0 2010-12-11 21:02 in_use.lock
drwxr-xr-x 2 rock users 4096 2010-12-11 20:17 previous.checkpoint

./dfs/name/current:
total 16
-rw-r--r-- 1 rock users   4 2010-12-11 23:08 edits
-rw-r--r-- 1 rock users 424 2010-12-11 23:08 fsimage
-rw-r--r-- 1 rock users   8 2010-12-11 23:08 fstime
-rw-r--r-- 1 rock users 100 2010-12-11 23:08 VERSION

./dfs/name/image:
total 4
-rw-r--r-- 1 rock users 157 2010-12-11 23:08 fsimage

./dfs/name/previous.checkpoint:
total 16
-rw-r--r-- 1 rock users   4 2010-12-11 20:17 edits
-rw-r--r-- 1 rock users 538 2010-12-11 20:17 fsimage
-rw-r--r-- 1 rock users   8 2010-12-11 20:17 fstime
-rw-r--r-- 1 rock users 100 2010-12-11 20:17 VERSION

./dfs/namesecondary:
total 8
drwxr-xr-x 2 rock users 4096 2010-12-11 23:08 current
drwxr-xr-x 2 rock users 4096 2010-12-11 10:17 image
-rw-r--r-- 1 rock users    0 2010-12-11 21:02 in_use.lock

./dfs/namesecondary/current:
total 16
-rw-r--r-- 1 rock users   4 2010-12-11 23:08 edits
-rw-r--r-- 1 rock users 424 2010-12-11 23:08 fsimage
-rw-r--r-- 1 rock users   8 2010-12-11 23:08 fstime
-rw-r--r-- 1 rock users 100 2010-12-11 23:08 VERSION

./dfs/namesecondary/image:
total 4
-rw-r--r-- 1 rock users 157 2010-12-11 23:08 fsimage
rock@ritter:/tmp/hadoop-rock>

Re: exceptions copying files into HDFS

Posted by Varadharajan Mukundan <sr...@gmail.com>.
Hi,

> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File
> /user/rock/input/fair-scheduler.xml could only be replicated to 0 nodes,
> instead of 1

I think none of your datanodes are actually running. why not use jps
and make sure whether they are running. Also check the datanode logs
in the nodes.

-- 
Thanks,
M. Varadharajan

------------------------------------------------

"Experience is what you get when you didn't get what you wanted"
               -By Prof. Randy Pausch in "The Last Lecture"

My Journal :- www.thinkasgeek.wordpress.com