You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Robin East <ro...@xense.co.uk> on 2013/06/26 22:53:45 UTC
Exception in createBlockOutputStream - poss firewall issue
I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
The java code looks like this (using hadoop 1.1.2):
private static void createHdfsFile() throws IOException {
Configuration conf = new Configuration();
conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
FileSystem hdfs = FileSystem.get(conf);
System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
os.close();
}
Any idea how I can get this to work?
HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
er information
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Abandoning block blk_4933973859208379842_1028
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1107)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Dunno on top of my head if its present in your version (grep sources
perhaps), but there are properties dfs.client.use.datanode.hostname,
dfs.datanode.use.datanode.hostname, dfs.datanode.dns.interface and
dfs.client.local.interfaces available to tune something on this layer.
See http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
for their description (page is 2.x release hdfs-defaults).
On Wed, Jul 3, 2013 at 1:38 AM, Robin East <ro...@xense.co.uk> wrote:
> Hi John, exactly what I was thinking, however I haven't found a way to do
> that. If I ever have time I'll trawl through the code, however I've managed
> to avoid the issue by placing both machines inside the firewall.
>
> Regards
> Robin
>
> Sent from my iPhone
>
> On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
>
> I don’t know the answer… but if it is possible to make the DNs report a
> domain-name instead of an IP quad it may help.
>
> John
>
>
>
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
>
>
> Ok I should have added that external address of the cluster is NATed to the
> internal address. The internal address yyy.yyy.yyy.yyy is not a routable
> address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx
> but I guess the data node must advertise itself using the internal address
> (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in
> secure environments).
>
>
>
> Robin
>
> Sent from my iPad
>
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010
> and 50020 by default. Please open up these ports, aside of the NN's RPC
> ports, to be able to read/write data.
>
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>
> I have a single node hadoop cluster setup behind a firewall and am trying to
> create files using a java program outside the firewall and get the exception
> below. The java program works fine inside the firewall. The ip address for
> the single cluster is xxx.xxx.xxx.xxx however it appears that in the
> createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
>
> --
> Harsh J
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Dunno on top of my head if its present in your version (grep sources
perhaps), but there are properties dfs.client.use.datanode.hostname,
dfs.datanode.use.datanode.hostname, dfs.datanode.dns.interface and
dfs.client.local.interfaces available to tune something on this layer.
See http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
for their description (page is 2.x release hdfs-defaults).
On Wed, Jul 3, 2013 at 1:38 AM, Robin East <ro...@xense.co.uk> wrote:
> Hi John, exactly what I was thinking, however I haven't found a way to do
> that. If I ever have time I'll trawl through the code, however I've managed
> to avoid the issue by placing both machines inside the firewall.
>
> Regards
> Robin
>
> Sent from my iPhone
>
> On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
>
> I don’t know the answer… but if it is possible to make the DNs report a
> domain-name instead of an IP quad it may help.
>
> John
>
>
>
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
>
>
> Ok I should have added that external address of the cluster is NATed to the
> internal address. The internal address yyy.yyy.yyy.yyy is not a routable
> address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx
> but I guess the data node must advertise itself using the internal address
> (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in
> secure environments).
>
>
>
> Robin
>
> Sent from my iPad
>
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010
> and 50020 by default. Please open up these ports, aside of the NN's RPC
> ports, to be able to read/write data.
>
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>
> I have a single node hadoop cluster setup behind a firewall and am trying to
> create files using a java program outside the firewall and get the exception
> below. The java program works fine inside the firewall. The ip address for
> the single cluster is xxx.xxx.xxx.xxx however it appears that in the
> createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
>
> --
> Harsh J
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Dunno on top of my head if its present in your version (grep sources
perhaps), but there are properties dfs.client.use.datanode.hostname,
dfs.datanode.use.datanode.hostname, dfs.datanode.dns.interface and
dfs.client.local.interfaces available to tune something on this layer.
See http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
for their description (page is 2.x release hdfs-defaults).
On Wed, Jul 3, 2013 at 1:38 AM, Robin East <ro...@xense.co.uk> wrote:
> Hi John, exactly what I was thinking, however I haven't found a way to do
> that. If I ever have time I'll trawl through the code, however I've managed
> to avoid the issue by placing both machines inside the firewall.
>
> Regards
> Robin
>
> Sent from my iPhone
>
> On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
>
> I don’t know the answer… but if it is possible to make the DNs report a
> domain-name instead of an IP quad it may help.
>
> John
>
>
>
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
>
>
> Ok I should have added that external address of the cluster is NATed to the
> internal address. The internal address yyy.yyy.yyy.yyy is not a routable
> address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx
> but I guess the data node must advertise itself using the internal address
> (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in
> secure environments).
>
>
>
> Robin
>
> Sent from my iPad
>
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010
> and 50020 by default. Please open up these ports, aside of the NN's RPC
> ports, to be able to read/write data.
>
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>
> I have a single node hadoop cluster setup behind a firewall and am trying to
> create files using a java program outside the firewall and get the exception
> below. The java program works fine inside the firewall. The ip address for
> the single cluster is xxx.xxx.xxx.xxx however it appears that in the
> createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
>
> --
> Harsh J
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Dunno on top of my head if its present in your version (grep sources
perhaps), but there are properties dfs.client.use.datanode.hostname,
dfs.datanode.use.datanode.hostname, dfs.datanode.dns.interface and
dfs.client.local.interfaces available to tune something on this layer.
See http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
for their description (page is 2.x release hdfs-defaults).
On Wed, Jul 3, 2013 at 1:38 AM, Robin East <ro...@xense.co.uk> wrote:
> Hi John, exactly what I was thinking, however I haven't found a way to do
> that. If I ever have time I'll trawl through the code, however I've managed
> to avoid the issue by placing both machines inside the firewall.
>
> Regards
> Robin
>
> Sent from my iPhone
>
> On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
>
> I don’t know the answer… but if it is possible to make the DNs report a
> domain-name instead of an IP quad it may help.
>
> John
>
>
>
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
>
>
> Ok I should have added that external address of the cluster is NATed to the
> internal address. The internal address yyy.yyy.yyy.yyy is not a routable
> address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx
> but I guess the data node must advertise itself using the internal address
> (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in
> secure environments).
>
>
>
> Robin
>
> Sent from my iPad
>
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010
> and 50020 by default. Please open up these ports, aside of the NN's RPC
> ports, to be able to read/write data.
>
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>
> I have a single node hadoop cluster setup behind a firewall and am trying to
> create files using a java program outside the firewall and get the exception
> below. The java program works fine inside the firewall. The ip address for
> the single cluster is xxx.xxx.xxx.xxx however it appears that in the
> createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
>
> --
> Harsh J
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Hi John, exactly what I was thinking, however I haven't found a way to do that. If I ever have time I'll trawl through the code, however I've managed to avoid the issue by placing both machines inside the firewall.
Regards
Robin
Sent from my iPhone
On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
> I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
> John
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
> Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
>
> Robin
>
> Sent from my iPad
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
> --
> Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Hi John, exactly what I was thinking, however I haven't found a way to do that. If I ever have time I'll trawl through the code, however I've managed to avoid the issue by placing both machines inside the firewall.
Regards
Robin
Sent from my iPhone
On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
> I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
> John
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
> Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
>
> Robin
>
> Sent from my iPad
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
> --
> Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Hi John, exactly what I was thinking, however I haven't found a way to do that. If I ever have time I'll trawl through the code, however I've managed to avoid the issue by placing both machines inside the firewall.
Regards
Robin
Sent from my iPhone
On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
> I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
> John
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
> Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
>
> Robin
>
> Sent from my iPad
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
> --
> Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Hi John, exactly what I was thinking, however I haven't found a way to do that. If I ever have time I'll trawl through the code, however I've managed to avoid the issue by placing both machines inside the firewall.
Regards
Robin
Sent from my iPhone
On 2 Jul 2013, at 19:48, John Lilley <jo...@redpoint.net> wrote:
> I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
> John
>
>
> From: Robin East [mailto:robin.east@xense.co.uk]
> Sent: Thursday, June 27, 2013 12:18 AM
> To: user@hadoop.apache.org
> Subject: Re: Exception in createBlockOutputStream - poss firewall issue
>
> Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
>
> Robin
>
> Sent from my iPad
>
> On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
>
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
>
> --
> Harsh J
RE: Exception in createBlockOutputStream - poss firewall issue
Posted by John Lilley <jo...@redpoint.net>.
I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
John
From: Robin East [mailto:robin.east@xense.co.uk]
Sent: Thursday, June 27, 2013 12:18 AM
To: user@hadoop.apache.org
Subject: Re: Exception in createBlockOutputStream - poss firewall issue
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com>> wrote:
Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk>> wrote:
I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
The java code looks like this (using hadoop 1.1.2):
private static void createHdfsFile() throws IOException {
Configuration conf = new Configuration();
conf.set("fs.default.name<http://fs.default.name>", "hdfs://"+hdfsHost+":9000");
FileSystem hdfs = FileSystem.get(conf);
System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
os.close();
}
Any idea how I can get this to work?
HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
er information
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Abandoning block blk_4933973859208379842_1028
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1107)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
--
Harsh J
RE: Exception in createBlockOutputStream - poss firewall issue
Posted by John Lilley <jo...@redpoint.net>.
I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
John
From: Robin East [mailto:robin.east@xense.co.uk]
Sent: Thursday, June 27, 2013 12:18 AM
To: user@hadoop.apache.org
Subject: Re: Exception in createBlockOutputStream - poss firewall issue
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com>> wrote:
Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk>> wrote:
I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
The java code looks like this (using hadoop 1.1.2):
private static void createHdfsFile() throws IOException {
Configuration conf = new Configuration();
conf.set("fs.default.name<http://fs.default.name>", "hdfs://"+hdfsHost+":9000");
FileSystem hdfs = FileSystem.get(conf);
System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
os.close();
}
Any idea how I can get this to work?
HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
er information
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Abandoning block blk_4933973859208379842_1028
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1107)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
--
Harsh J
RE: Exception in createBlockOutputStream - poss firewall issue
Posted by John Lilley <jo...@redpoint.net>.
I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
John
From: Robin East [mailto:robin.east@xense.co.uk]
Sent: Thursday, June 27, 2013 12:18 AM
To: user@hadoop.apache.org
Subject: Re: Exception in createBlockOutputStream - poss firewall issue
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com>> wrote:
Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk>> wrote:
I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
The java code looks like this (using hadoop 1.1.2):
private static void createHdfsFile() throws IOException {
Configuration conf = new Configuration();
conf.set("fs.default.name<http://fs.default.name>", "hdfs://"+hdfsHost+":9000");
FileSystem hdfs = FileSystem.get(conf);
System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
os.close();
}
Any idea how I can get this to work?
HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
er information
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Abandoning block blk_4933973859208379842_1028
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1107)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
--
Harsh J
RE: Exception in createBlockOutputStream - poss firewall issue
Posted by John Lilley <jo...@redpoint.net>.
I don’t know the answer… but if it is possible to make the DNs report a domain-name instead of an IP quad it may help.
John
From: Robin East [mailto:robin.east@xense.co.uk]
Sent: Thursday, June 27, 2013 12:18 AM
To: user@hadoop.apache.org
Subject: Re: Exception in createBlockOutputStream - poss firewall issue
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com>> wrote:
Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk>> wrote:
I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
The java code looks like this (using hadoop 1.1.2):
private static void createHdfsFile() throws IOException {
Configuration conf = new Configuration();
conf.set("fs.default.name<http://fs.default.name>", "hdfs://"+hdfsHost+":9000");
FileSystem hdfs = FileSystem.get(conf);
System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
os.close();
}
Any idea how I can get this to work?
HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
er information
Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Abandoning block blk_4933973859208379842_1028
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1107)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
>> The java code looks like this (using hadoop 1.1.2):
>>
>> private static void createHdfsFile() throws IOException {
>> Configuration conf = new Configuration();
>> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
>> FileSystem hdfs = FileSystem.get(conf);
>> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
>> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>>
>> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
>> os.close();
>> }
>>
>> Any idea how I can get this to work?
>>
>>
>> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
>> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
>> er information
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Abandoning block blk_4933973859208379842_1028
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
>> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
>> could only be replicated to 0 nodes, instead of 1
>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
>> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>>
>> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
>> at $Proxy1.addBlock(Unknown Source)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>> at java.lang.reflect.Method.invoke(Unknown Source)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
>> at $Proxy1.addBlock(Unknown Source)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
> --
> Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
>> The java code looks like this (using hadoop 1.1.2):
>>
>> private static void createHdfsFile() throws IOException {
>> Configuration conf = new Configuration();
>> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
>> FileSystem hdfs = FileSystem.get(conf);
>> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
>> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>>
>> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
>> os.close();
>> }
>>
>> Any idea how I can get this to work?
>>
>>
>> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
>> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
>> er information
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Abandoning block blk_4933973859208379842_1028
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
>> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
>> could only be replicated to 0 nodes, instead of 1
>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
>> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>>
>> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
>> at $Proxy1.addBlock(Unknown Source)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>> at java.lang.reflect.Method.invoke(Unknown Source)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
>> at $Proxy1.addBlock(Unknown Source)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
> --
> Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
>> The java code looks like this (using hadoop 1.1.2):
>>
>> private static void createHdfsFile() throws IOException {
>> Configuration conf = new Configuration();
>> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
>> FileSystem hdfs = FileSystem.get(conf);
>> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
>> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>>
>> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
>> os.close();
>> }
>>
>> Any idea how I can get this to work?
>>
>>
>> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
>> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
>> er information
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Abandoning block blk_4933973859208379842_1028
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
>> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
>> could only be replicated to 0 nodes, instead of 1
>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
>> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>>
>> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
>> at $Proxy1.addBlock(Unknown Source)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>> at java.lang.reflect.Method.invoke(Unknown Source)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
>> at $Proxy1.addBlock(Unknown Source)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
> --
> Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Robin East <ro...@xense.co.uk>.
Ok I should have added that external address of the cluster is NATed to the internal address. The internal address yyy.yyy.yyy.yyy is not a routable address for the client. The client can reach the namenode on xxx.xxx.xxx.xxx but I guess the data node must advertise itself using the internal address (yyy.yyy.yyy.yyy). What is the way round this ( not an uncommon problem in secure environments).
Robin
Sent from my iPad
On 27 Jun 2013, at 03:41, Harsh J <ha...@cloudera.com> wrote:
> Clients will read/write data to the DNs directly. DNs serve on port 50010 and 50020 by default. Please open up these ports, aside of the NN's RPC ports, to be able to read/write data.
>
>
> On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
>> I have a single node hadoop cluster setup behind a firewall and am trying to create files using a java program outside the firewall and get the exception below. The java program works fine inside the firewall. The ip address for the single cluster is xxx.xxx.xxx.xxx however it appears that in the createBlockOutputStream the client things the data node is at ip yyy.yyy.yyy.yyy (the internal address of the cluster) which is not accessible.
>> The java code looks like this (using hadoop 1.1.2):
>>
>> private static void createHdfsFile() throws IOException {
>> Configuration conf = new Configuration();
>> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
>> FileSystem hdfs = FileSystem.get(conf);
>> System.out.println("HDFS Working Directory: " + hdfs.getWorkingDirectory().toString());
>> FSDataOutputStream os = hdfs.create(new Path("/user/hadoop/test2.txt"));
>>
>> os.writeChars("Example text\n for a hadoop write call\n\ntesting\n");
>> os.close();
>> }
>>
>> Any idea how I can get this to work?
>>
>>
>> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream createBlockOutputStream
>> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010 java.net.ConnectException: Connection timed out: no furth
>> er information
>> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Abandoning block blk_4933973859208379842_1028
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream nextBlockOutputStream
>> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
>> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
>> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/test2.txt
>> could only be replicated to 0 nodes, instead of 1
>> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
>> at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:601)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:415)
>> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>>
>> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
>> at $Proxy1.addBlock(Unknown Source)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>> at java.lang.reflect.Method.invoke(Unknown Source)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
>> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
>> at $Proxy1.addBlock(Unknown Source)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
>> at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
>
> --
> Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Clients will read/write data to the DNs directly. DNs serve on port 50010
and 50020 by default. Please open up these ports, aside of the NN's RPC
ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying
> to create files using a java program outside the firewall and get the
> exception below. The java program works fine inside the firewall. The ip
> address for the single cluster is xxx.xxx.xxx.xxx however it appears that
> in the createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Clients will read/write data to the DNs directly. DNs serve on port 50010
and 50020 by default. Please open up these ports, aside of the NN's RPC
ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying
> to create files using a java program outside the firewall and get the
> exception below. The java program works fine inside the firewall. The ip
> address for the single cluster is xxx.xxx.xxx.xxx however it appears that
> in the createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Clients will read/write data to the DNs directly. DNs serve on port 50010
and 50020 by default. Please open up these ports, aside of the NN's RPC
ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying
> to create files using a java program outside the firewall and get the
> exception below. The java program works fine inside the firewall. The ip
> address for the single cluster is xxx.xxx.xxx.xxx however it appears that
> in the createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
--
Harsh J
Re: Exception in createBlockOutputStream - poss firewall issue
Posted by Harsh J <ha...@cloudera.com>.
Clients will read/write data to the DNs directly. DNs serve on port 50010
and 50020 by default. Please open up these ports, aside of the NN's RPC
ports, to be able to read/write data.
On Thu, Jun 27, 2013 at 2:23 AM, Robin East <ro...@xense.co.uk> wrote:
> I have a single node hadoop cluster setup behind a firewall and am trying
> to create files using a java program outside the firewall and get the
> exception below. The java program works fine inside the firewall. The ip
> address for the single cluster is xxx.xxx.xxx.xxx however it appears that
> in the createBlockOutputStream the client things the data node is at ip
> yyy.yyy.yyy.yyy (the internal address of the cluster) which is not
> accessible.
> The java code looks like this (using hadoop 1.1.2):
>
> private static void createHdfsFile() throws IOException {
> Configuration conf = new Configuration();
> conf.set("fs.default.name", "hdfs://"+hdfsHost+":9000");
> FileSystem hdfs = FileSystem.get(conf);
> System.out.println("HDFS Working Directory: " +
> hdfs.getWorkingDirectory().toString());
> FSDataOutputStream os = hdfs.create(new
> Path("/user/hadoop/test2.txt"));
>
> os.writeChars("Example text\n for a hadoop write
> call\n\ntesting\n");
> os.close();
> }
>
> Any idea how I can get this to work?
>
>
> HDFS Working Directory: hdfs://xxx.xxx.xxx.xxx:9000/user/zzzzz
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> createBlockOutputStream
> INFO: Exception in createBlockOutputStream yyy.yyy.yyy.yyy:50010
> java.net.ConnectException: Connection timed out: no furth
> er information
> Jun 26, 2013 8:08:52 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Abandoning block blk_4933973859208379842_1028
> Jun 26, 2013 8:08:53 PM org.apache.hadoop.hdfs.DFSClient$DFSOutputStream
> nextBlockOutputStream
> INFO: Excluding datanode yyy.yyy.yyy.yyy:50010
> Jun 26, 2013 8:08:53 PM
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer run
> WARNING: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException: File /user/hadoop/test2.txt
> could only be replicated to 0 nodes, instead of 1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
> at $Proxy1.addBlock(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
> at $Proxy1.addBlock(Unknown Source)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
>
>
--
Harsh J