[jira] [Comment Edited] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix

["Oliver Koeth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944778#comment-15944778 ] 

Oliver Koeth edited comment on SPARK-20044 at 3/28/17 8:41 AM:
---------------------------------------------------------------

Added a pull request. Works for me with the following configuration:
spark.ui.reverseProxy=true
spark.ui.reverseProxyUrl=/path/to/spark/

nginx front-end proxy setup:
<pre>
server {
    listen 9000;
    set $SPARK_MASTER http://<myhost>:8080;

    # redirect master UI path without terminating slash,
    # so that relative URLs are resolved correctly
    location ~ ^(?<prefix>/path/to/spark$) {
        return 302 $scheme://$host:$server_port$prefix/;
    }

    # split spark UI path into prefix and local path within master UI
    location ~ ^(?<prefix>/path/to/spark)(?<local_path>/.*) {
        # strip prefix when forwarding request
        rewrite ^ $local_path break;
        # forward to spark master UI
        proxy_pass $SPARK_MASTER;
        # fix host (implicit) and add prefix on redirects
        proxy_redirect $SPARK_MASTER $prefix;
    }
}
</pre>


was (Author: okoethibm):
Added a pull request. Works for me with the following configuration:
spark.ui.reverseProxy=true
spark.ui.reverseProxyUrl=/path/to/spark/

nginx front-end proxy setup:
server {
    listen 9000;
    set $SPARK_MASTER http://<myhost>:8080;

    # redirect master UI path without terminating slash,
    # so that relative URLs are resolved correctly
    location ~ ^(?<prefix>/path/to/spark$) {
        return 302 $scheme://$host:$server_port$prefix/;
    }

    # split spark UI path into prefix and local path within master UI
    location ~ ^(?<prefix>/path/to/spark)(?<local_path>/.*) {
        # strip prefix when forwarding request
        rewrite ^ $local_path break;
        # forward to spark master UI
        proxy_pass $SPARK_MASTER;
        # fix host (implicit) and add prefix on redirects
        proxy_redirect $SPARK_MASTER $prefix;
    }
}

> Support Spark UI behind front-end reverse proxy using a path prefix
> -------------------------------------------------------------------
>
>                 Key: SPARK-20044
>                 URL: https://issues.apache.org/jira/browse/SPARK-20044
>             Project: Spark
>          Issue Type: Improvement
>          Components: Web UI
>    Affects Versions: 2.1.0
>            Reporter: Oliver Koeth
>            Priority: Minor
>              Labels: reverse-proxy, sso
>
> Purpose: allow to run the Spark web UI behind a reverse proxy with URLs prefixed by a context root, like www.mydomain.com/spark. In particular, this allows to access multiple Spark clusters through the same virtual host, only distinguishing them by context root, like www.mydomain.com/cluster1, www.mydomain.com/cluster2, and it allows to run the Spark UI in a common cookie domain (for SSO) with other services.
> [SPARK-15487] introduced some support for front-end reverse proxies by allowing all Spark UI requests to be routed through the master UI as a single endpoint and also added a spark.ui.reverseProxyUrl setting to define a another proxy sitting in front of Spark. However, as noted in the comments on [SPARK-15487], this mechanism does not currently work if the reverseProxyUrl includes a context root like the examples above: Most links generated by the Spark UI result in full path URLs (like /proxy/app-"id"/...) that do not account for a path prefix (context root) and work only if the Spark UI "owns" the entire virtual host. In fact, the only place in the UI where the reverseProxyUrl seems to be used is the back-link from the worker UI to the master UI.
> The discussion on [SPARK-15487] proposes to open a new issue for the problem, but that does not seem to have happened, so this issue aims to address the remaining shortcomings of spark.ui.reverseProxyUrl
> The problem can be partially worked around by doing content rewrite in a front-end proxy and prefixing src="/..." or href="/..." links with a context root. However, detecting and patching URLs in HTML output is not a robust approach and breaks down for URLs included in custom REST responses. E.g. the "allexecutors" REST call used from the Spark 2.1.0 application/executors page returns links for log viewing that direct to the worker UI and do not work in this scenario.
> This issue proposes to honor spark.ui.reverseProxyUrl throughout Spark UI URL generation. Experiments indicate that most of this can simply be achieved by using/prepending spark.ui.reverseProxyUrl to the existing spark.ui.proxyBase system property. Beyond that, the places that require adaption are
> - worker and application links in the master web UI
> - webui URLs returned by REST interfaces
> Note: It seems that returned redirect location headers do not need to be adapted, since URL rewriting for these is commonly done in front-end proxies and has a well-defined interface



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org