You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Steve Lewis <lo...@gmail.com> on 2011/09/24 02:19:06 UTC
I need help talking to HDFS over a firewall
I have a small piece of code which opens hdfs. When I run the code my a
machine riunning Windows 7 from work it connects perfectly
String host = "myhost"
int port = 9000;
String connectString = "hdfs://" + host + ":" + port + "/";
Configuration config = new Configuration();
config.set("fs.default.name",connectString);
m_DFS = FileSystem.get(config);
When I run over a VPN from home it used to work but has stopped . The Hadoop
cluster at work may have been reconfigured
My windows firewall is turned off
telnet myhost 9000 works from the command line
http://hyhost:50075/browseDirectory.jsp?dir=/<http://glados1:50075/browseDirectory.jsp?dir=/>
shows me the contents of HDFS
Our IT department insists is a firewall of some issue on my end. They also
insist that once I am connected through the VPN nothing is blocked on their
end.
Is there any way this could be a cluster or hadoop issue. Again all of my
code talking to the cluster runs fine at work and nothing works from home
--
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com
RE: I need help talking to HDFS over a firewall
Posted by Aaron Baff <Aa...@telescope.tv>.
Are you sure you have the right port number? As you say, if it's been reconfigured, could they have changed the port the NN runs on? Also, could they have changed the hostname of the NN? Instead of connecting to the NN you actually are trying to connect to one of the datanodes?
--Aaron
-----Original Message-----
From: Steve Lewis [mailto:lordjoe2000@gmail.com]
Sent: Friday, September 23, 2011 5:19 PM
To: common-user
Subject: I need help talking to HDFS over a firewall
I have a small piece of code which opens hdfs. When I run the code my a
machine riunning Windows 7 from work it connects perfectly
String host = "myhost"
int port = 9000;
String connectString = "hdfs://" + host + ":" + port + "/";
Configuration config = new Configuration();
config.set("fs.default.name",connectString);
m_DFS = FileSystem.get(config);
When I run over a VPN from home it used to work but has stopped . The Hadoop
cluster at work may have been reconfigured
My windows firewall is turned off
telnet myhost 9000 works from the command line
http://hyhost:50075/browseDirectory.jsp?dir=/<http://glados1:50075/browseDirectory.jsp?dir=/>
shows me the contents of HDFS
Our IT department insists is a firewall of some issue on my end. They also
insist that once I am connected through the VPN nothing is blocked on their
end.
Is there any way this could be a cluster or hadoop issue. Again all of my
code talking to the cluster runs fine at work and nothing works from home
--
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com
RE: I need help talking to HDFS over a firewall
Posted by "Dhodapkar, Chinmay" <ch...@qualcomm.com>.
Have you tried using the host ip addr instead of hostname? This seems a little weird...
If you are going to face firewall issues in the future, you may want to consider using Hoop to access hdfs using REST api. (http://www.cloudera.com/blog/2011/07/hoop-hadoop-hdfs-over-http/)
-chinmay
-----Original Message-----
From: Steve Lewis [mailto:lordjoe2000@gmail.com]
Sent: Friday, September 23, 2011 5:19 PM
To: common-user
Subject: I need help talking to HDFS over a firewall
I have a small piece of code which opens hdfs. When I run the code my a
machine riunning Windows 7 from work it connects perfectly
String host = "myhost"
int port = 9000;
String connectString = "hdfs://" + host + ":" + port + "/";
Configuration config = new Configuration();
config.set("fs.default.name",connectString);
m_DFS = FileSystem.get(config);
When I run over a VPN from home it used to work but has stopped . The Hadoop
cluster at work may have been reconfigured
My windows firewall is turned off
telnet myhost 9000 works from the command line
http://hyhost:50075/browseDirectory.jsp?dir=/<http://glados1:50075/browseDirectory.jsp?dir=/>
shows me the contents of HDFS
Our IT department insists is a firewall of some issue on my end. They also
insist that once I am connected through the VPN nothing is blocked on their
end.
Is there any way this could be a cluster or hadoop issue. Again all of my
code talking to the cluster runs fine at work and nothing works from home
--
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com