[jira] [Comment Edited] (FALCON-1) Create packaging and scripts to install and try out Apache Falcon

["Srikanth Sundarrajan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FALCON-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703430#comment-13703430 ] 

Srikanth Sundarrajan edited comment on FALCON-1 at 7/9/13 4:12 PM:
-------------------------------------------------------------------

The patch includes the following changes

1. Download oozie tar ball from apache release, apply patch specific to OOZIE-674 and apply oozie el extensions required for FALCON, bundle them as a custom version - 3.2.2 (no such oozie version exists in the oozie release listing) and add them to local repo on the host
2. Added a jetty:plugin to bring up oozie (from step 1), hadoop and falcon inside the same jvm
3. Migrated the unit tests that depend on oozie/hadoop instance to integration test run via maven failsafe plugin
4. Maven assembly for packaging falcon (this doesn't include oozie)
5. Packaging script for bundling custom oozie for external deployment

*Building (with unit-tests, checkstyle, findbugs & integration-tests) for default hadoop version*
{code}
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=144m" && mvn clean install
{code}

*Building (with unit-tests, checkstyle, findbugs & integration-tests) for custom hadoop version (ex)*
{code}
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=144m" && mvn clean install -Dhadoop.version=0.20.2-cdh3u5
{code}

*Building running falcon with embedded hadoop & oozie for test*
{code}
cd <<project-home>>
mvn clean package -DskipTests -DskipCheck=true
rm -rf target/store
rm -rf webapp/target/webapps/oozie/data
cd webapp
mvn jetty:run
{code}

*Creating package for falcon*
{code}
mvn clean assembly:assembly
{code}

*Creating a custom package of oozie for deployment along with falcon*
{code}
<<projecthome>>/build-tools/src/bin/package.sh
{code}

                
      was (Author: sriksun):
    The patch includes the following changes

1. Download oozie tar ball from apache release, apply patch specific to OOZIE-674 and apply oozie el extensions required for FALCON, bundle them as a custom version - 3.2.2 (no such oozie version exists in the oozie release listing) and add them to local repo on the host
2. Added a jetty:plugin to bring up oozie (from step 1), hadoop and falcon inside the same jvm
3. Migrated the unit tests that depend on oozie/hadoop instance to integration test run via maven failsafe plugin
4. Maven assembly for packaging falcon (this doesn't include oozie)
5. Packaging script for bundling custom oozie for external deployment

*Building (with unit-tests, checkstyle, findbugs & integration-tests) for default hadoop version *
{code}
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=144m" && mvn clean install
{code}

*Building (with unit-tests, checkstyle, findbugs & integration-tests) for custom hadoop version (ex) *
{code}
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=144m" && mvn clean install -Dhadoop.version=0.20.2-cdh3u5
{code}

*Building running falcon with embedded hadoop & oozie for test *
{code}
cd <<project-home>>
mvn clean package -DskipTests -DskipCheck=true
rm -rf target/store
rm -rf webapp/target/webapps/oozie/data
cd webapp
mvn jetty:run
{code}

*Creating package for falcon *
{code}
mvn clean assembly:assembly
{code}

*Creating a custom package of oozie for deployment along with falcon*
{code}
<<projecthome>>/build-tools/src/bin/package.sh
{code}

                  
> Create packaging and scripts to install and try out Apache Falcon 
> ------------------------------------------------------------------
>
>                 Key: FALCON-1
>                 URL: https://issues.apache.org/jira/browse/FALCON-1
>             Project: Falcon
>          Issue Type: Improvement
>    Affects Versions: 0.3
>            Reporter: Srikanth Sundarrajan
>            Assignee: Srikanth Sundarrajan
>            Priority: Critical
>              Labels: general
>             Fix For: 0.3
>
>         Attachments: FALCON-1.patch
>
>
> To get Falcon up and running, it currently require us to bring up hadoop, oozie instances and also make changes to oozie-site to make necessary EL extension to oozie. The idea is to minimize these manual steps and to reduce friction while trying out Falcon.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira