Re: Review Request 16511: Process update for wf changes

[Srikanth Sundarrajan <sr...@hotmail.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16511/#review31027
-----------------------------------------------------------

Ship it!


+1

- Srikanth Sundarrajan


On Dec. 31, 2013, 10:06 a.m., shwethags wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16511/
> -----------------------------------------------------------
> 
> (Updated Dec. 31, 2013, 10:06 a.m.)
> 
> 
> Review request for Falcon.
> 
> 
> Bugs: FALCON-206
>     https://issues.apache.org/jira/browse/FALCON-206
> 
> 
> Repository: falcon-git
> 
> 
> Description
> -------
> 
> At process schedule, process mapper creates a checksum file that contains the list of user wf/lib files and their checksum. The process mapper also copies the user wf and lib to staging directory(where parent wf and coord xml are stored). The parent workflow references the user workflow copied to staging dir. 
> Process update updates in oozie only if entity is updated or user wf/lib is updated.
> With this change, when user wf/lib is updated, the currently running instances will not fail. This will also make adding new input to process easier (which includes updating process and workflow.xml)
> Limitations: user oozie workflow should use only relative paths for libs/pig scripts/hive scripts
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/falcon/entity/ClusterHelper.java e332aba 
>   common/src/main/java/org/apache/falcon/entity/EntityUtil.java ba80cac 
>   common/src/main/java/org/apache/falcon/update/UpdateHelper.java fc69933 
>   common/src/main/resources/log4j.xml 734d17c 
>   common/src/main/resources/startup.properties 5473f5d 
>   common/src/test/java/org/apache/falcon/entity/parser/FeedUpdateTest.java f39f300 
>   common/src/test/java/org/apache/falcon/entity/parser/ProcessEntityParserTest.java e656772 
>   common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java 42bcee0 
>   common/src/test/resources/config/process/process-0.1.xml bb5cd35 
>   common/src/test/resources/config/process/process-0.2.xml c4cd83e 
>   common/src/test/resources/config/process/process-table.xml 1d6a8f0 
>   feed/src/main/java/org/apache/falcon/workflow/OozieFeedWorkflowBuilder.java 7b9095f 
>   oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java c47ec01 
>   oozie/src/main/java/org/apache/falcon/workflow/OozieWorkflowBuilder.java 1978c53 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieHouseKeepingService.java 7e2f8a4 
>   oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 1ed1ff7 
>   prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java 7ec6cd1 
>   prism/src/main/resources/log4j.xml ac1d9e4 
>   process/src/main/java/org/apache/falcon/converter/OozieProcessMapper.java 8749f07 
>   process/src/main/java/org/apache/falcon/workflow/OozieProcessWorkflowBuilder.java 1329733 
>   process/src/test/java/org/apache/falcon/converter/OozieProcessMapperTest.java 7d5f4d1 
>   replication/src/main/java/org/apache/falcon/replication/CustomReplicator.java ba01c80 
>   replication/src/main/java/org/apache/falcon/replication/FeedReplicator.java 17227bf 
>   replication/src/main/java/org/apache/falcon/replication/FilteredCopyListing.java 9c97a44 
>   src/bin/service-start.sh 430bb1a 
>   src/conf/log4j.xml 0b28ddd 
>   src/conf/startup.properties 69613f6 
>   webapp/src/main/resources/log4j.xml 6790576 
>   webapp/src/test/java/org/apache/falcon/logging/LogMoverIT.java 24c2959 
>   webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java cb2fcbb 
>   webapp/src/test/java/org/apache/falcon/resource/TestContext.java d224f90 
>   webapp/src/test/java/org/apache/falcon/util/OozieTestUtils.java 690fb6b 
> 
> Diff: https://reviews.apache.org/r/16511/diff/
> 
> 
> Testing
> -------
> 
> UT, tested with embedded deployment
> 
> 
> Thanks,
> 
> shwethags
> 
>