[jira] [Comment Edited] (MESOS-1509) Use Content-Disposition filename (if available) when downloading HTTP URIs

["Yennick Trevels (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15721914#comment-15721914 ] 

Yennick Trevels edited comment on MESOS-1509 at 12/5/16 10:43 AM:
------------------------------------------------------------------

I can confirm that Mesos currently disregards the "Content-Disposition" header.

*My scenario*
I tried submitting a spark application to DC/OS where the Jar was hosted on S3. This required a S3 presigned (Signature V4) url, of which the url parameters can get pretty long.
sample presigned S3 url: 
{code}
https://s3.eu-central-1.amazonaws.com/bucketA/spark-app.jar?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=XXXXXXXXXXXXXXXXXXXXXXXXXX&X-Amz-SignedHeaders=host&X-Amz-Date=20161205T095356Z&X-Amz-Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
{code}

When trying to run the Spark app on DC/OS it immediately failed with the following error:
{code}
Failed to fetch 'https://s3.eu-central-1.amazonaws.com/bucketA/spark-app.jar?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=XXXXXXXXXXXXXXXXXXXXXXXXXX&X-Amz-SignedHeaders=host&X-Amz-Date=20161205T095356Z&X-Amz-Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX': Error downloading resource: File name too long
{code}

Then I tried uploading the file to S3 with a Content-Disposition header: 
{code}
attachment; filename='spark-app-alt.jar'; filename*=UTF-8''spark%2Dapp%2Dalt%2Ejar
{code}
The content disposition header is properly displayed in the AWS Console (so nothing wrong there) and when downloading the file via a browser with a presigned S3 url it gets properly saved with the filename specified in the content-disposition header.
However, when the Mesos Fetcher tries to download the file, it still fails with the "file name too long" error.


was (Author: slevinbe):
I can confirm that Mesos currently disregards the "Content-Disposition" header.

*My scenario*
I tried submitting a spark application to DC/OS where the Jar was hosted on S3. This required a S3 presigned (Signature V4) url, of which the url parameters can get pretty long.
sample presigned S3 url: 
```
https://s3.eu-central-1.amazonaws.com/bucketA/spark-app.jar?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=XXXXXXXXXXXXXXXXXXXXXXXXXX&X-Amz-SignedHeaders=host&X-Amz-Date=20161205T095356Z&X-Amz-Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
```

When trying to run the Spark app on DC/OS it immediately failed with the following error:
```
Failed to fetch 'https://s3.eu-central-1.amazonaws.com/bucketA/spark-app.jar?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=3600&X-Amz-Credential=XXXXXXXXXXXXXXXXXXXXXXXXXX&X-Amz-SignedHeaders=host&X-Amz-Date=20161205T095356Z&X-Amz-Signature=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX': Error downloading resource: File name too long
```

Then I tried uploading the file to S3 with a Content-Disposition header: 
```
attachment; filename='spark-app-alt.jar'; filename*=UTF-8''spark%2Dapp%2Dalt%2Ejar
```
The content disposition header is properly displayed in the AWS Console (so nothing wrong there) and when downloading the file via a browser with a presigned S3 url it gets properly saved with the filename specified in the content-disposition header.
However, when the Mesos Fetcher tries to download the file, it still fails with the "file name too long" error.

> Use Content-Disposition filename (if available) when downloading HTTP URIs
> --------------------------------------------------------------------------
>
>                 Key: MESOS-1509
>                 URL: https://issues.apache.org/jira/browse/MESOS-1509
>             Project: Mesos
>          Issue Type: Improvement
>          Components: agent
>    Affects Versions: 0.18.0, 0.19.0, 0.20.0, 0.21.0
>         Environment: Linux (but should be irrelevant)
>            Reporter: Bjoern Metzdorf
>            Priority: Minor
>
> Currently the slave stores downloaded HTTP URIs in filenames that are made up from the part after the last "/" in the URI (in src/launcher/fetcher.cpp:122):
> {code}
> path =  path::join(directory, path.substr(path.find_last_of("/") + 1));
> {code}
> The problem is that the query string is included in the filename and a URI like {{http://my.web.server/dynamic/resource.tar.gz?a=b}} results in a downloaded file named {{resource.tar.gz?a=b}}.
> The curl maintainers faced the same problem and added this:
> {code}
> -J, --remote-header-name
>               (HTTP) This option tells the -O, --remote-name option to use the server-specified Content-Disposition filename instead of extracting a filename from the URL.
> {code}
> Maybe Mesos could do the same if a Content-Disposition header exists.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)