[jira] [Commented] (KNOX-256) Oozie output/input Dir configuration rewrite rules

["Mate Juhasz (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KNOX-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17014950#comment-17014950 ] 

Mate Juhasz commented on KNOX-256:
----------------------------------

Hi! This issue, or very similar is still present. Knox writes incorrect values for "${nameNode}" if this variable is substituted in another place,  "/hdfs:/..." instead of "hdfs://..."

> Oozie output/input Dir configuration rewrite rules
> --------------------------------------------------
>
>                 Key: KNOX-256
>                 URL: https://issues.apache.org/jira/browse/KNOX-256
>             Project: Apache Knox
>          Issue Type: Bug
>    Affects Versions: 0.3.0
>            Reporter: Vladimir Tkhir
>            Priority: Major
>         Attachments: README, oozie-examples.jar, workflow-configuration.xml, workflow.xml
>
>
> Steps to reproduce:
> Save attached files to /tmp/oozie folder
> # 0. Optionally cleanup the test directory in case a previous example was run without cleaning up.
> curl -i -k -u guest:guest-password -X DELETE \
>     'https://localhost:8443/gateway/sandbox/webhdfs/v1/user/guest/example?op=DELETE&recursive=true'
> # 1. Create the inode for workflow definition file in /user/guest/example
> curl -i -k -u guest:guest-password -X PUT \
>     'https://localhost:8443/gateway/sandbox/webhdfs/v1/user/guest/example/workflow.xml?op=CREATE'
> # 2. Upload the workflow definition file. 
> curl -i -k -u guest:guest-password -T /tmp/oozie/workflow.xml -X PUT \
>     '{Value Location header from command above}'
> # 3. Create the inode for oozie-examples.jar in /user/guest/example/lib
> curl -i -k -u guest:guest-password -X PUT \
>     'https://localhost:8443/gateway/sandbox/webhdfs/v1/user/guest/example/lib/oozie-examples.jar?op=CREATE'
> # 4. Upload oozie-examples.jar to /user/guest/example/lib.
> curl -i -k -u guest:guest-password -T /tmp/oozie/oozie-examples.jar -X PUT \
>     '{Value Location header from command above}'
> # 5. Create the inode for a sample input file readme in /user/guest/example/input.
> curl -i -k -u guest:guest-password -X PUT \
>     'https://localhost:8443/gateway/sandbox/webhdfs/v1/user/guest/example/input/README?op=CREATE'
> # 6. Upload readme.txt to /user/guest/example/input.
> curl -i -k -u guest:guest-password -T /tmp/oozie/README -X PUT \
>     '{Value of Location header from command above}'
> # 7. Submit the job via Oozie
> # Take note of the Job ID in the JSON response as this will be used in the next step.
> curl -ikv -u guest:guest-password -H Content-Type:application/xml -T /tmp/oozie/workflow-configuration.xml \
>     -X POST 'https://localhost:8443/gateway/sandbox/oozie/v1/jobs?action=start'
> Job started, but failed with exception message: java.lang.IllegalArgumentException: Pathname /user/guest/Oozie1SysTest/output-data/hdfs:/hdp:8020/map-reduce from hdfs://hdp:8020/user/guest/Oozie1SysTest/output-data/hdfs:/hdp:8020/map-reduce is not a valid DFS filename.
> Knox interprets outputDir parameter value as a absolute path and rewrites it to /hdp:8020/map-reduce. 
> mapred.output.dir parameter in workflow definition uses outputDir parameter in its value: /user/${wf:user()}/${examplesRoot}/output-data/${outputDir} and resulting parameter value is incorrect.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)