Is it possible to disable AM page proxy in Yarn client mode?

[Rex Xiong <by...@gmail.com>]

In Yarn client mode, Spark driver URL will be redirected to Yarn web proxy
server, but I don't want to use this dynamic name, is it possible to still
use <host>:<port> as standalone mode?

Re: Is it possible to disable AM page proxy in Yarn client mode?

[Steve Loughran <st...@hortonworks.com>]

the reason that redirect is there is for security reasons; in a kerberos enabled cluster the RM proxy does the authentication, then forwards the requests to the running application. There's no obvious way to disable it in the spark application master, and I wouldn't recommend doing this anyway, as it only gets you into a situation where your code works until you flip the security bit on.

The Spark Web UI installs a new filter, (AmIpFilter), which 302's all HTTP requests coming in from anywhere other than the host running the RM Proxy.

1. if you make requests from that host (curl, browser, whatever), then they go through without the redirect.
2. If you don't have an RM proxy (why not?) then you can configure the spark AM to treat your client IP address as the proxy -and again, no redirect.

YARN-2031 covers the ongoing work to have that proxy/IP filter handle REST API properly. Currently it only handles GET operations & assumes a human visiting the application in a web browser

> On 3 Aug 2015, at 01:52, Rex Xiong <by...@gmail.com> wrote:
> 
> In Yarn client mode, Spark driver URL will be redirected to Yarn web proxy server, but I don't want to use this dynamic name, is it possible to still use <host>:<port> as standalone mode?


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org