You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Bjorn Jonsson (JIRA)" <ji...@apache.org> on 2017/05/17 02:09:04 UTC
[jira] [Updated] (SPARK-20772) Add support for query parameters in
redirects on Yarn
[ https://issues.apache.org/jira/browse/SPARK-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bjorn Jonsson updated SPARK-20772:
----------------------------------
Description:
Spark uses rewrites of query parameters to paths (http://<fqdn>:4040/jobs/job?id=0 --> http://<fqdn>:4040/jobs/job/?id=0). This works fine in local or standalone mode, but does not work on Yarn (with the org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter filter), where the query parameter is dropped.
The repro steps are:
- Start up the spark-shell and run a job
- Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
- A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
Going directly through the RM proxy works (does not cause query parameters to be dropped).
was:
Spark uses rewrites of query parameters to paths (http://<fqdn>:4040/jobs/job?id=0 --> http://<fqdn>:4040/jobs/job/?id=0). This works fine in local or standalone mode, but does not work on Yarn (with the org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter filter), where the query parameter is dropped.
The repro steps are:
- Start up the spark-shell in yarn client or cluster mode and run a job
- Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
- A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
Going directly through the RM proxy works (does not cause query parameters to be dropped).
> Add support for query parameters in redirects on Yarn
> -----------------------------------------------------
>
> Key: SPARK-20772
> URL: https://issues.apache.org/jira/browse/SPARK-20772
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 2.1.0
> Reporter: Bjorn Jonsson
> Priority: Minor
>
> Spark uses rewrites of query parameters to paths (http://<fqdn>:4040/jobs/job?id=0 --> http://<fqdn>:4040/jobs/job/?id=0). This works fine in local or standalone mode, but does not work on Yarn (with the org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter filter), where the query parameter is dropped.
> The repro steps are:
> - Start up the spark-shell and run a job
> - Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
> - A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
> Going directly through the RM proxy works (does not cause query parameters to be dropped).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org