[jira] [Commented] (OOZIE-1241) DistributedCache unqualification prevents oozie from reading applications from non 'default' filesystems

["jdnurmi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590122#comment-13590122 ] 

jdnurmi commented on OOZIE-1241:
--------------------------------

>From fairly limited testing, this does seem to allow oozie on apache hadoop to interact with application.xml's and similar on S3;

To be clear, it was this email:
http://mail-archives.apache.org/mod_mbox/oozie-user/201212.mbox/browser 
and a class path update hinted at by:
https://groups.google.com/a/cloudera.org/forum/?fromgroups=#!topic/cdh-user/rY99hCM2V_A 

that allowed S3 to be used as a source for oozie workflows.

I have not experienced any obvious issues with this patch, but will update this ticket if I notice any.
                
> DistributedCache unqualification prevents oozie from reading applications from non 'default' filesystems
> --------------------------------------------------------------------------------------------------------
>
>                 Key: OOZIE-1241
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1241
>             Project: Oozie
>          Issue Type: Bug
>          Components: action
>    Affects Versions: 3.3.1
>         Environment: Sun Java6, oozie 3.3.0 (and 3.3.1), ubuntu lucid (+/-)
>            Reporter: jdnurmi
>            Priority: Minor
>         Attachments: urifix.patch
>
>
> For example, a workflow config similar to:
> oozie.wf.application.path=s3n://foo/bar/baz
> Pretty much regardless of the workflow itself will yield something akin to
> "JA008: File does not exist: /bar/baz/oozie/mapred-job-launcher.jar"
> Chasing this down a bit, I ran across:
> http://mail-archives.apache.org/mod_mbox/oozie-user/201212.mbox/browser
> which seemed like a cogent analysis of the problem I was facing.
> The 1-line patch to be attached "fixes the glitch", and passes all tests on 3.3.0 (confirming 3.3.1), but I can't swear it doesn't introduce further problems.
> (There is an additional subtle quirk, in that a Path with an anchor that has been ToURI'd puts the anchor in the URI's /path/ as opposed to its fragment - causing it to be URL encoded later on in the process.  By using new URI(path.toString()), the fragment is properly parsed).
> I am still waiting for a few workflows to run to see if this fixes "all" s3 access, but wanted to collect feedback early in case this is a bad path to travel down for other reasons.  It at least appears to solve the case of your workflow.xml being in S3.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira