[jira] [Commented] (OOZIE-2339) Provide an API for writing jobs based on the XSD schemas

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16259751#comment-16259751 ] 

Hadoop QA commented on OOZIE-2339:
----------------------------------

Testing JIRA OOZIE-2339

Cleaning local git workspace

----------------------------

{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.    {color:green}+1{color} the patch does not introduce any @author tags
.    {color:red}-1{color} the patch contains 12 line(s) with tabs
.    {color:green}+1{color} the patch does not introduce any trailing spaces
.    {color:red}-1{color} the patch contains 6 line(s) longer than 132 characters
.    {color:green}+1{color} the patch does adds/modifies 57 testcase(s)
{color:green}+1 RAT{color}
.    {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
.    {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
.    {color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:red}-1 COMPILE{color}
.    {color:green}+1{color} HEAD compiles
.    {color:red}-1{color} patch does not compile
.    {color:red}-1{color} the patch seems to introduce 1 new javac warning(s)
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.    {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
.    {color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.    {color:red}-1{color} distro tarball fails with the patch

----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/246/

> Provide an API for writing jobs based on the XSD schemas
> --------------------------------------------------------
>
>                 Key: OOZIE-2339
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2339
>             Project: Oozie
>          Issue Type: New Feature
>          Components: client
>    Affects Versions: 4.3.0
>            Reporter: Robert Kanter
>            Assignee: Andras Piros
>             Fix For: 5.0.0
>
>         Attachments: OOZIE-2339.001.patch, OOZIE-2339.002.patch, OOZIE-2339.003.patch
>
>
> Users often complain about the XML they have to write for Oozie jobs.  It would be nice if they could write them in something like Java, but we don't want to have to maintain a separate Java API for this.  I was looking around and saw that JAXB might be the right thing here.  From what I can tell, it lets you create Java classes from XSD schemas.  So, we should be able to auto-generate a Java API for writing Oozie jobs, without having to really maintain it.
> We should investigate if this is feasible and, if so, implement it.
> Some useful looking links:
> * [JAXB overview|https://en.wikipedia.org/wiki/Java_Architecture_for_XML_Binding]
> * [JAXB description|https://jaxb.java.net/2.2.11/docs/ch03.html]
> * [Maven JAXB plugin|https://java.net/projects/maven-jaxb2-plugin/pages/Home]
> * [Apache Falcon|https://falcon.apache.org]
> Key features:
> * must have:
> ** inside an {{oozie-jobs-api}} artifact
> ** able to create workflow / coordinator / bundle definitions programmatically
> ** synchronizing each and every XSD change on rebuild
> ** can write {{workflow.xml}}, {{coordinator.xml}}, {{bundle.xml}}, and {{jobs.properties}} artifacts of every XSD version
> ** cloneability of workflow etc. {{Object}} s
> ** perform cross checks, e.g. that the workflow graph is a DAG
> ** only latest XSD versions should be supported as must have
> * nice to have:
> ** XSD version(s) can be provided. When not provided, latest ones are considered as valid
> ** implement a [*fluent API*|https://en.wikipedia.org/wiki/Fluent_interface]
> ** have a Python / Jython REPL to make it easy to experiment
> ** create documentation about usage
> ** can read {{workflow.xml}}, {{coordinator.xml}}, {{bundle.xml}}, and {{jobs.properties}} artifacts of every XSD version
> ** can convert between XSD versions
> ** support XSD change on the fly (within REPL)
> ** support HDFS reads / writes
> ** support dry run on an Oozie server to perform checks



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)