You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Jian Fang <ji...@gmail.com> on 2013/09/10 00:39:11 UTC
HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Hi,
I need to use the web services in application master, for example,
curl
http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
the following errors.
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 500 Connection refused</title>
</head>
<body><h2>HTTP ERROR 500</h2>
<p>Problem accessing
/proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
<pre> Connection refused</pre></p><h3>Caused
by:</h3><pre>java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:249)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at
org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at
org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
at
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
at
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
Since there was no jetty logs for the calls, I used tcpdump to capture the
packets to see what was going on.
Here is the TCP session:
GET /ws/v1/mapreduce/info HTTP/1.1
Accept: application/json
User-Agent: Java/1.6.0_43
Cookie: proxy-user=dr.who
Host: 10.140.14.159:39975
HTTP/1.1 302 Found
Cache-Control: no-cache
Expires: Mon, 09 Sep 2013 22:00:35 GMT
Date: Mon, 09 Sep 2013 22:00:35 GMT
Pragma: no-cache
Content-Type: text/plain; charset=utf-8
Location:
http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
Content-Length: 0
Server: Jetty(6.1.26)
Looked like the application master did not redirect the web service call
correctly (10.140.14.159 is the slave node where the application master was
running and 9026 is my resource manager http port).
There was not firewall between machines and it worked if I use the
following command to call the application master web services directly.
curl http://10.140.14.159:39975/ws/v1/mapreduce/info
Anything wrong here?
Thanks,
John
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Just in case someone run into the same problem and need help. One
workaround is to set "yarn.web-proxy.address" in yarn-site.xml explicitly
and start the proxy server as a separate process by running the following
command.
./yarn-daemon.sh start proxyserver
On Tue, Sep 10, 2013 at 1:14 PM, Jian Fang <ji...@gmail.com>wrote:
> Ok, seems there is a jira for this issue.
>
> https://issues.apache.org/jira/browse/YARN-800
>
>
> On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
>
>> Hi,
>>
>> I need to use the web services in application master, for example,
>>
>> curl
>> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>>
>> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
>> the following errors.
>>
>> <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500 Connection refused</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing
>> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
>> <pre> Connection refused</pre></p><h3>Caused
>> by:</h3><pre>java.net.ConnectException: Connection refused
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>> at
>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at java.net.Socket.<init>(Socket.java:375)
>> at java.net.Socket.<init>(Socket.java:249)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>> at
>> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>
>> Since there was no jetty logs for the calls, I used tcpdump to capture
>> the packets to see what was going on.
>>
>> Here is the TCP session:
>>
>> GET /ws/v1/mapreduce/info HTTP/1.1
>> Accept: application/json
>> User-Agent: Java/1.6.0_43
>> Cookie: proxy-user=dr.who
>> Host: 10.140.14.159:39975
>>
>> HTTP/1.1 302 Found
>> Cache-Control: no-cache
>> Expires: Mon, 09 Sep 2013 22:00:35 GMT
>> Date: Mon, 09 Sep 2013 22:00:35 GMT
>> Pragma: no-cache
>> Content-Type: text/plain; charset=utf-8
>> Location:
>> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>> Content-Length: 0
>> Server: Jetty(6.1.26)
>>
>> Looked like the application master did not redirect the web service call
>> correctly (10.140.14.159 is the slave node where the application master was
>> running and 9026 is my resource manager http port).
>>
>> There was not firewall between machines and it worked if I use the
>> following command to call the application master web services directly.
>>
>> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>>
>> Anything wrong here?
>>
>> Thanks,
>>
>> John
>>
>
>
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Just in case someone run into the same problem and need help. One
workaround is to set "yarn.web-proxy.address" in yarn-site.xml explicitly
and start the proxy server as a separate process by running the following
command.
./yarn-daemon.sh start proxyserver
On Tue, Sep 10, 2013 at 1:14 PM, Jian Fang <ji...@gmail.com>wrote:
> Ok, seems there is a jira for this issue.
>
> https://issues.apache.org/jira/browse/YARN-800
>
>
> On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
>
>> Hi,
>>
>> I need to use the web services in application master, for example,
>>
>> curl
>> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>>
>> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
>> the following errors.
>>
>> <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500 Connection refused</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing
>> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
>> <pre> Connection refused</pre></p><h3>Caused
>> by:</h3><pre>java.net.ConnectException: Connection refused
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>> at
>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at java.net.Socket.<init>(Socket.java:375)
>> at java.net.Socket.<init>(Socket.java:249)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>> at
>> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>
>> Since there was no jetty logs for the calls, I used tcpdump to capture
>> the packets to see what was going on.
>>
>> Here is the TCP session:
>>
>> GET /ws/v1/mapreduce/info HTTP/1.1
>> Accept: application/json
>> User-Agent: Java/1.6.0_43
>> Cookie: proxy-user=dr.who
>> Host: 10.140.14.159:39975
>>
>> HTTP/1.1 302 Found
>> Cache-Control: no-cache
>> Expires: Mon, 09 Sep 2013 22:00:35 GMT
>> Date: Mon, 09 Sep 2013 22:00:35 GMT
>> Pragma: no-cache
>> Content-Type: text/plain; charset=utf-8
>> Location:
>> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>> Content-Length: 0
>> Server: Jetty(6.1.26)
>>
>> Looked like the application master did not redirect the web service call
>> correctly (10.140.14.159 is the slave node where the application master was
>> running and 9026 is my resource manager http port).
>>
>> There was not firewall between machines and it worked if I use the
>> following command to call the application master web services directly.
>>
>> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>>
>> Anything wrong here?
>>
>> Thanks,
>>
>> John
>>
>
>
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Just in case someone run into the same problem and need help. One
workaround is to set "yarn.web-proxy.address" in yarn-site.xml explicitly
and start the proxy server as a separate process by running the following
command.
./yarn-daemon.sh start proxyserver
On Tue, Sep 10, 2013 at 1:14 PM, Jian Fang <ji...@gmail.com>wrote:
> Ok, seems there is a jira for this issue.
>
> https://issues.apache.org/jira/browse/YARN-800
>
>
> On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
>
>> Hi,
>>
>> I need to use the web services in application master, for example,
>>
>> curl
>> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>>
>> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
>> the following errors.
>>
>> <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500 Connection refused</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing
>> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
>> <pre> Connection refused</pre></p><h3>Caused
>> by:</h3><pre>java.net.ConnectException: Connection refused
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>> at
>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at java.net.Socket.<init>(Socket.java:375)
>> at java.net.Socket.<init>(Socket.java:249)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>> at
>> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>
>> Since there was no jetty logs for the calls, I used tcpdump to capture
>> the packets to see what was going on.
>>
>> Here is the TCP session:
>>
>> GET /ws/v1/mapreduce/info HTTP/1.1
>> Accept: application/json
>> User-Agent: Java/1.6.0_43
>> Cookie: proxy-user=dr.who
>> Host: 10.140.14.159:39975
>>
>> HTTP/1.1 302 Found
>> Cache-Control: no-cache
>> Expires: Mon, 09 Sep 2013 22:00:35 GMT
>> Date: Mon, 09 Sep 2013 22:00:35 GMT
>> Pragma: no-cache
>> Content-Type: text/plain; charset=utf-8
>> Location:
>> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>> Content-Length: 0
>> Server: Jetty(6.1.26)
>>
>> Looked like the application master did not redirect the web service call
>> correctly (10.140.14.159 is the slave node where the application master was
>> running and 9026 is my resource manager http port).
>>
>> There was not firewall between machines and it worked if I use the
>> following command to call the application master web services directly.
>>
>> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>>
>> Anything wrong here?
>>
>> Thanks,
>>
>> John
>>
>
>
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Just in case someone run into the same problem and need help. One
workaround is to set "yarn.web-proxy.address" in yarn-site.xml explicitly
and start the proxy server as a separate process by running the following
command.
./yarn-daemon.sh start proxyserver
On Tue, Sep 10, 2013 at 1:14 PM, Jian Fang <ji...@gmail.com>wrote:
> Ok, seems there is a jira for this issue.
>
> https://issues.apache.org/jira/browse/YARN-800
>
>
> On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
>
>> Hi,
>>
>> I need to use the web services in application master, for example,
>>
>> curl
>> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>>
>> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
>> the following errors.
>>
>> <html>
>> <head>
>> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
>> <title>Error 500 Connection refused</title>
>> </head>
>> <body><h2>HTTP ERROR 500</h2>
>> <p>Problem accessing
>> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
>> <pre> Connection refused</pre></p><h3>Caused
>> by:</h3><pre>java.net.ConnectException: Connection refused
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
>> at
>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>> at java.net.Socket.connect(Socket.java:529)
>> at java.net.Socket.connect(Socket.java:478)
>> at java.net.Socket.<init>(Socket.java:375)
>> at java.net.Socket.<init>(Socket.java:249)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
>> at
>> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
>> at
>> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
>> at
>> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
>> at
>> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
>> at
>> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>
>> Since there was no jetty logs for the calls, I used tcpdump to capture
>> the packets to see what was going on.
>>
>> Here is the TCP session:
>>
>> GET /ws/v1/mapreduce/info HTTP/1.1
>> Accept: application/json
>> User-Agent: Java/1.6.0_43
>> Cookie: proxy-user=dr.who
>> Host: 10.140.14.159:39975
>>
>> HTTP/1.1 302 Found
>> Cache-Control: no-cache
>> Expires: Mon, 09 Sep 2013 22:00:35 GMT
>> Date: Mon, 09 Sep 2013 22:00:35 GMT
>> Pragma: no-cache
>> Content-Type: text/plain; charset=utf-8
>> Location:
>> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>> Content-Length: 0
>> Server: Jetty(6.1.26)
>>
>> Looked like the application master did not redirect the web service call
>> correctly (10.140.14.159 is the slave node where the application master was
>> running and 9026 is my resource manager http port).
>>
>> There was not firewall between machines and it worked if I use the
>> following command to call the application master web services directly.
>>
>> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>>
>> Anything wrong here?
>>
>> Thanks,
>>
>> John
>>
>
>
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Ok, seems there is a jira for this issue.
https://issues.apache.org/jira/browse/YARN-800
On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
> Hi,
>
> I need to use the web services in application master, for example,
>
> curl
> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>
> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
> the following errors.
>
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 500 Connection refused</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing
> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
> <pre> Connection refused</pre></p><h3>Caused
> by:</h3><pre>java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
> at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at java.net.Socket.<init>(Socket.java:375)
> at java.net.Socket.<init>(Socket.java:249)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
> at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>
> Since there was no jetty logs for the calls, I used tcpdump to capture the
> packets to see what was going on.
>
> Here is the TCP session:
>
> GET /ws/v1/mapreduce/info HTTP/1.1
> Accept: application/json
> User-Agent: Java/1.6.0_43
> Cookie: proxy-user=dr.who
> Host: 10.140.14.159:39975
>
> HTTP/1.1 302 Found
> Cache-Control: no-cache
> Expires: Mon, 09 Sep 2013 22:00:35 GMT
> Date: Mon, 09 Sep 2013 22:00:35 GMT
> Pragma: no-cache
> Content-Type: text/plain; charset=utf-8
> Location:
> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
> Content-Length: 0
> Server: Jetty(6.1.26)
>
> Looked like the application master did not redirect the web service call
> correctly (10.140.14.159 is the slave node where the application master was
> running and 9026 is my resource manager http port).
>
> There was not firewall between machines and it worked if I use the
> following command to call the application master web services directly.
>
> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>
> Anything wrong here?
>
> Thanks,
>
> John
>
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Ok, seems there is a jira for this issue.
https://issues.apache.org/jira/browse/YARN-800
On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
> Hi,
>
> I need to use the web services in application master, for example,
>
> curl
> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>
> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
> the following errors.
>
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 500 Connection refused</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing
> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
> <pre> Connection refused</pre></p><h3>Caused
> by:</h3><pre>java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
> at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at java.net.Socket.<init>(Socket.java:375)
> at java.net.Socket.<init>(Socket.java:249)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
> at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>
> Since there was no jetty logs for the calls, I used tcpdump to capture the
> packets to see what was going on.
>
> Here is the TCP session:
>
> GET /ws/v1/mapreduce/info HTTP/1.1
> Accept: application/json
> User-Agent: Java/1.6.0_43
> Cookie: proxy-user=dr.who
> Host: 10.140.14.159:39975
>
> HTTP/1.1 302 Found
> Cache-Control: no-cache
> Expires: Mon, 09 Sep 2013 22:00:35 GMT
> Date: Mon, 09 Sep 2013 22:00:35 GMT
> Pragma: no-cache
> Content-Type: text/plain; charset=utf-8
> Location:
> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
> Content-Length: 0
> Server: Jetty(6.1.26)
>
> Looked like the application master did not redirect the web service call
> correctly (10.140.14.159 is the slave node where the application master was
> running and 9026 is my resource manager http port).
>
> There was not firewall between machines and it worked if I use the
> following command to call the application master web services directly.
>
> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>
> Anything wrong here?
>
> Thanks,
>
> John
>
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Ok, seems there is a jira for this issue.
https://issues.apache.org/jira/browse/YARN-800
On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
> Hi,
>
> I need to use the web services in application master, for example,
>
> curl
> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>
> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
> the following errors.
>
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 500 Connection refused</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing
> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
> <pre> Connection refused</pre></p><h3>Caused
> by:</h3><pre>java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
> at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at java.net.Socket.<init>(Socket.java:375)
> at java.net.Socket.<init>(Socket.java:249)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
> at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>
> Since there was no jetty logs for the calls, I used tcpdump to capture the
> packets to see what was going on.
>
> Here is the TCP session:
>
> GET /ws/v1/mapreduce/info HTTP/1.1
> Accept: application/json
> User-Agent: Java/1.6.0_43
> Cookie: proxy-user=dr.who
> Host: 10.140.14.159:39975
>
> HTTP/1.1 302 Found
> Cache-Control: no-cache
> Expires: Mon, 09 Sep 2013 22:00:35 GMT
> Date: Mon, 09 Sep 2013 22:00:35 GMT
> Pragma: no-cache
> Content-Type: text/plain; charset=utf-8
> Location:
> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
> Content-Length: 0
> Server: Jetty(6.1.26)
>
> Looked like the application master did not redirect the web service call
> correctly (10.140.14.159 is the slave node where the application master was
> running and 9026 is my resource manager http port).
>
> There was not firewall between machines and it worked if I use the
> following command to call the application master web services directly.
>
> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>
> Anything wrong here?
>
> Thanks,
>
> John
>
Re: HTTP ERROR 500 when call application master proxy URL in Hadoop 2.1.0-beta
Posted by Jian Fang <ji...@gmail.com>.
Ok, seems there is a jira for this issue.
https://issues.apache.org/jira/browse/YARN-800
On Mon, Sep 9, 2013 at 3:39 PM, Jian Fang <ji...@gmail.com>wrote:
> Hi,
>
> I need to use the web services in application master, for example,
>
> curl
> http://10.6.179.230:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
>
> This used to work fine, but after I upgraded to Hadoop 2.1.0-beta, I got
> the following errors.
>
> <html>
> <head>
> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
> <title>Error 500 Connection refused</title>
> </head>
> <body><h2>HTTP ERROR 500</h2>
> <p>Problem accessing
> /proxy/application_1378761541170_0003/ws/v1/mapreduce/info. Reason:
> <pre> Connection refused</pre></p><h3>Caused
> by:</h3><pre>java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
> at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at java.net.Socket.connect(Socket.java:529)
> at java.net.Socket.connect(Socket.java:478)
> at java.net.Socket.<init>(Socket.java:375)
> at java.net.Socket.<init>(Socket.java:249)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
> at
> org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
> at
> org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
> at
> org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
> at
> org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.proxyLink(WebAppProxyServlet.java:185)
> at
> org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet.doGet(WebAppProxyServlet.java:334)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>
> Since there was no jetty logs for the calls, I used tcpdump to capture the
> packets to see what was going on.
>
> Here is the TCP session:
>
> GET /ws/v1/mapreduce/info HTTP/1.1
> Accept: application/json
> User-Agent: Java/1.6.0_43
> Cookie: proxy-user=dr.who
> Host: 10.140.14.159:39975
>
> HTTP/1.1 302 Found
> Cache-Control: no-cache
> Expires: Mon, 09 Sep 2013 22:00:35 GMT
> Date: Mon, 09 Sep 2013 22:00:35 GMT
> Pragma: no-cache
> Content-Type: text/plain; charset=utf-8
> Location:
> http://10.140.14.159:9026/proxy/application_1378761541170_0003/ws/v1/mapreduce/info
> Content-Length: 0
> Server: Jetty(6.1.26)
>
> Looked like the application master did not redirect the web service call
> correctly (10.140.14.159 is the slave node where the application master was
> running and 9026 is my resource manager http port).
>
> There was not firewall between machines and it worked if I use the
> following command to call the application master web services directly.
>
> curl http://10.140.14.159:39975/ws/v1/mapreduce/info
>
> Anything wrong here?
>
> Thanks,
>
> John
>