[jira] [Commented] (OOZIE-1695) Generic HCFS supported integration path

["Robert Kanter (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900949#comment-13900949 ] 

Robert Kanter commented on OOZIE-1695:
--------------------------------------

We can easily rename the name-node element in the workflow schemas, we'd have to add a new version for all schemas though.  While I agree that the {{<name-node>}} element isn't exactly accurate for non-HDFS filesystems, I'm not sure what generic thing we'd replace it with.  A similar issue is the {{<job-tracker>}} element, which you can either put the job tracker (Hadoop 1) or the Resource Manager (Hadoop 2), even though its not exactly accurate either; I'm not sure what we'd replace this with either.  I'm all for improving this area, but we need to make sure its not too confusing for the majority of users.  

On the documentation in general: As most people will be using HDFS, we need to be carful generalizing the documentation mentioning HDFS or the NameNode to not be too confusing to the majority of users.  

On the documentation for adding file system plugins to Oozie: The oozie-site property for the filesystem whitelist is documented here: http://oozie.apache.org/docs/4.0.0/AG_Install.html#Filesystem_Configuration
We can add something similar to what we say below that in the "Adding HCatalog jars to Oozie war:" section about adding other FileSystem jars to Oozie (it's the same process).  It may make sense to add a general note about adding plugin jars (of any kind) to Oozie in this manner somewhere in the documentation, and then linking to it from both of these places.  

> Generic HCFS supported integration path 
> ----------------------------------------
>
>                 Key: OOZIE-1695
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1695
>             Project: Oozie
>          Issue Type: Task
>            Reporter: jay vyas
>
> The dizzying OOZIE-426 JIRA indicates that at the moment it is not clear wether oozie does, or doesn't, support any HCFS file system. 
> - the good news is, after some digging : It does !
> So we have two tasks, mostly documentation i guess, but possibly some modifications in code/comments would be nice as well to clarify things further for developers:
> - So now we need to all agree on and document the "right" way to add file system plugins into Oozie.  Hopefully we can do so using semantics which is not dependant on HDFS/S3 to avoid confusion.  
> - In addition, Some clarity on why and how the "nameNode" parameter   is enforced as part of the XML schema for java tasks should also be clarified.  Clearly that is a bug since oozie supports non-HDFS deployments.  
> Specifically, it appears that in the http://oozie.apache.org/docs/3.2.0-incubating/WorkflowFunctionalSpec.html  "The java action has to be configured with the job-tracker, name-node, main Java class, JVM options and arguments.".....



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)