[jira] [Created] (MESOS-9348) URL-encoded HDFS artifacts can't be fetched through the cache.

["James Peach (JIRA)" <ji...@apache.org>]

James Peach created MESOS-9348:
----------------------------------

             Summary: URL-encoded HDFS artifacts can't be fetched through the cache.
                 Key: MESOS-9348
                 URL: https://issues.apache.org/jira/browse/MESOS-9348
             Project: Mesos
          Issue Type: Bug
          Components: fetcher
            Reporter: James Peach


The {{hdfs dfs}} command always does a URI decode on the target output file. This means that the output file gets stored in the fetcher cache under the wrong filename and we can never retrieve it.

Here's an example of how the command behaves:
{noformat}
[/tmp]# hdfs dfs -copyToLocal hdfs:///artifacts/8c/99/4b/8c994b489674589a58805e2e695e98674b9dd793411579f0fbaea3459f94f86e/connector/%5BRELEASE%5D/connector-%5BRELEASE%5D.jar $(pwd)/%5B-jpeach-%5D.jar

[/tmp]# ls -l *jpeach*
-rw-r--r-- 1 root root 7285799 Oct 22 23:29 [-jpeach-].jar
{noformat}

Here's how this plays out in the fetcher:
{noformat}
W1022 23:22:13.649587 3186459 fetcher.cpp:395] Copying instead of extracting resource from URI with 'extract' flag, because it does not seem to be an archive: hdfs:///artifacts/8c/99/4b/8c994b489674589a58805e2e695e98674b9dd793411579f0fbaea3459f94f86e/connector/%5BRELEASE%5D/connector-%5BRELEASE%5D.jar
cp: cannot stat `/srv/mesos/fetch/jarvis/c67-connector-_ASE%5D.jar': No such file or directory
E1022 23:22:13.652987 3186459 fetcher.cpp:613] EXIT with status 1: Failed to fetch 'hdfs:///artifacts/8c/99/4b/8c994b489674589a58805e2e695e98674b9dd793411579f0fbaea3459f94f86e/connector/%5BRELEASE%5D/connector-%5BRELEASE%5D.jar': cp failed with status: 256
...
# ls -latr /srv/mesos/fetch
...
-rw-r--r-- 1 jarvis jarvis   7285799 Oct 22 23:22 c67-connector-_ASE].jar
{noformat}

The fetcher has downloaded the artifact into the cache, but can't copy it into the sandbox because it was downloaded to the wrong filename.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)