You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by sh...@inmobi.com on 2014/12/16 11:42:49 UTC
Review Request 29092: FALCON-943 process update copying user lib is
very slow
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/
-----------------------------------------------------------
Review request for Falcon.
Bugs: FALCON-943
https://issues.apache.org/jira/browse/FALCON-943
Repository: falcon-git
Description
-------
Addresses the following:
1. Removed copying user workflow/lib to staging directory
2. Removed update at specific time
3. Copying falcon jars to parent workflow lib copies only specific falcon jars
4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
Diffs
-----
client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
common/src/main/resources/startup.properties 433c2a8
common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
docs/src/site/twiki/FalconCLI.twiki 202af63
docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
oozie/src/main/resources/action/post-process.xml 440a131
oozie/src/main/resources/action/pre-process.xml 070c42b
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
src/conf/startup.properties 2db4b1e
src/main/assemblies/distributed-package.xml 7320312
src/main/assemblies/standalone-package.xml f9d6009
src/main/examples/data/generate.sh 74ec3ba
webapp/src/main/java/org/apache/falcon/Debug.java c606074
webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
Diff: https://reviews.apache.org/r/29092/diff/
Testing
-------
UTs, end to end test for process, retention and replication
Thanks,
shwethags
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by sh...@inmobi.com.
> On Dec. 18, 2014, 9:40 a.m., Suhas Vasu wrote:
> > common/src/main/resources/startup.properties, line 53
> > <https://reviews.apache.org/r/29092/diff/1/?file=792947#file792947line53>
> >
> > "falcon-oozie-adaptorOEB" ? typo ?
yes, typo. will fix it
> On Dec. 18, 2014, 9:40 a.m., Suhas Vasu wrote:
> > docs/src/site/twiki/FalconDocumentation.twiki, line 211
> > <https://reviews.apache.org/r/29092/diff/1/?file=792950#file792950line211>
> >
> > We first set end time and then schedule new process/feed definition.
> > I feel we should change this sequence in documentation to avoid confusion to new users.
Yes, I thought it was other was around. Will change
- shwethags
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65475
-----------------------------------------------------------
On Dec. 16, 2014, 10:42 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 16, 2014, 10:42 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by Suhas Vasu <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65475
-----------------------------------------------------------
common/src/main/resources/startup.properties
<https://reviews.apache.org/r/29092/#comment108661>
"falcon-oozie-adaptorOEB" ? typo ?
docs/src/site/twiki/FalconDocumentation.twiki
<https://reviews.apache.org/r/29092/#comment108662>
We first set end time and then schedule new process/feed definition.
I feel we should change this sequence in documentation to avoid confusion to new users.
oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java
<https://reviews.apache.org/r/29092/#comment108665>
I feel it would be better if we rename jar to jarname or jarfile
oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java
<https://reviews.apache.org/r/29092/#comment108666>
can we reuse this function above (line 56) ?
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java
<https://reviews.apache.org/r/29092/#comment108669>
Feel StringUtils.isEmpty would be better
- Suhas Vasu
On Dec. 16, 2014, 10:42 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 16, 2014, 10:42 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by sh...@gmail.com.
> On Dec. 21, 2014, 4:19 p.m., Srikanth Sundarrajan wrote:
> > oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java, line 76
> > <https://reviews.apache.org/r/29092/diff/2/?file=795936#file795936line76>
> >
> > This is a new property. Wouldn't this result in NPE if the config is missed ? How do we ensure that necessary config steps are listed before migrating to the version with this change.
>
> shwethags wrote:
> Yes it does. During the upgrade, the guildeline is that they should take startup properties and make any changes required
>
> Srikanth Sundarrajan wrote:
> Should we setup a RELEASE_NOTES doc and start appending changes such as these, to ensure smoother rollouts?
If taking startup properties and adding more properties is not the usual procedure, I would rather hard-code it than make it as an extra step. This would change only with any changes in falcon and not user configurable
- shwethags
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65743
-----------------------------------------------------------
On Dec. 18, 2014, 10:32 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2014, 10:32 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by sh...@gmail.com.
> On Dec. 21, 2014, 4:19 p.m., Srikanth Sundarrajan wrote:
> > oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java, line 1200
> > <https://reviews.apache.org/r/29092/diff/2/?file=795946#file795946line1200>
> >
> > Does it make sense to return this effective time and surface it up all the way back to the user ? It seems like it might be useful, but not fully convinced myself.
If we use coord update in future for entity update, we will not be able to support effective time. Hence removed it altogether
> On Dec. 21, 2014, 4:19 p.m., Srikanth Sundarrajan wrote:
> > oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java, line 122
> > <https://reviews.apache.org/r/29092/diff/2/?file=795944#file795944line122>
> >
> > Are we cleaning up automatically previous installation artefacts ?
> >
> > Generally it might be useful to log when files are deleted.
> >
> > This JIRA seems to be slipping in many changes albeit unrelated.
The working path of the cluster that contains the third party jars has just the jar names without any version and its difficult to debug without jar versions. This just cleans up the jar without version. I can rollback this and address it in another jira
> On Dec. 21, 2014, 4:19 p.m., Srikanth Sundarrajan wrote:
> > oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java, line 76
> > <https://reviews.apache.org/r/29092/diff/2/?file=795936#file795936line76>
> >
> > This is a new property. Wouldn't this result in NPE if the config is missed ? How do we ensure that necessary config steps are listed before migrating to the version with this change.
Yes it does. During the upgrade, the guildeline is that they should take startup properties and make any changes required
> On Dec. 21, 2014, 4:19 p.m., Srikanth Sundarrajan wrote:
> > common/src/main/java/org/apache/falcon/entity/ProcessHelper.java, line 86
> > <https://reviews.apache.org/r/29092/diff/2/?file=795924#file795924line86>
> >
> > They dont seem to be related to rollback of the strict process versioning. Why are they being removed ?
Yes, they are related to copying user workflow and lib to staging directory. Without that, it will just be process workflow and lib path and hence these functions are not required
> On Dec. 21, 2014, 4:19 p.m., Srikanth Sundarrajan wrote:
> > common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java, line 263
> > <https://reviews.apache.org/r/29092/diff/2/?file=795923#file795923line263>
> >
> > is this moved to FeedHelper ? If not, can you move it there? Seems like an useful func.
Its already in Feedhelper
- shwethags
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65743
-----------------------------------------------------------
On Dec. 18, 2014, 10:32 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2014, 10:32 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by Srikanth Sundarrajan <sr...@hotmail.com>.
> On Dec. 21, 2014, 4:19 p.m., Srikanth Sundarrajan wrote:
> > oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java, line 76
> > <https://reviews.apache.org/r/29092/diff/2/?file=795936#file795936line76>
> >
> > This is a new property. Wouldn't this result in NPE if the config is missed ? How do we ensure that necessary config steps are listed before migrating to the version with this change.
>
> shwethags wrote:
> Yes it does. During the upgrade, the guildeline is that they should take startup properties and make any changes required
Should we setup a RELEASE_NOTES doc and start appending changes such as these, to ensure smoother rollouts?
- Srikanth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65743
-----------------------------------------------------------
On Dec. 18, 2014, 10:32 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2014, 10:32 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by Srikanth Sundarrajan <sr...@hotmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65743
-----------------------------------------------------------
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java
<https://reviews.apache.org/r/29092/#comment108957>
is this moved to FeedHelper ? If not, can you move it there? Seems like an useful func.
common/src/main/java/org/apache/falcon/entity/ProcessHelper.java
<https://reviews.apache.org/r/29092/#comment108959>
They dont seem to be related to rollback of the strict process versioning. Why are they being removed ?
oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java
<https://reviews.apache.org/r/29092/#comment108961>
This is a new property. Wouldn't this result in NPE if the config is missed ? How do we ensure that necessary config steps are listed before migrating to the version with this change.
oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java
<https://reviews.apache.org/r/29092/#comment108962>
Are we cleaning up automatically previous installation artefacts ?
Generally it might be useful to log when files are deleted.
This JIRA seems to be slipping in many changes albeit unrelated.
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java
<https://reviews.apache.org/r/29092/#comment108964>
Does it make sense to return this effective time and surface it up all the way back to the user ? It seems like it might be useful, but not fully convinced myself.
- Srikanth Sundarrajan
On Dec. 18, 2014, 10:32 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2014, 10:32 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by sh...@inmobi.com.
> On Dec. 19, 2014, 3:09 a.m., Ajay Yadava wrote:
> > oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java, line 187
> > <https://reviews.apache.org/r/29092/diff/1/?file=792965#file792965line187>
> >
> > Wouldn't it be better if we use created/modified time here as well? In case of a report we might have the numbers changed but the size might not change.
We don't record the size at file level and instead at root directory for the feed instance. I don't think there is a way to get modified time from hadoop filesystem. Of course, we can figure out a way to handle it. But since I haven't touched anything related to late data in this patch, lets track that separately in another jira
- shwethags
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65587
-----------------------------------------------------------
On Dec. 18, 2014, 10:32 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2014, 10:32 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by Ajay Yadava <aj...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65587
-----------------------------------------------------------
oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java
<https://reviews.apache.org/r/29092/#comment108834>
Wouldn't it be better if we use created/modified time here as well? In case of a report we might have the numbers changed but the size might not change.
- Ajay Yadava
On Dec. 18, 2014, 10:32 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2014, 10:32 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by Suhas Vasu <su...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65615
-----------------------------------------------------------
Ship it!
Ship It!
- Suhas Vasu
On Dec. 18, 2014, 10:32 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 18, 2014, 10:32 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/main/resources/startup.properties 433c2a8
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/conf/startup.properties 2db4b1e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by Srikanth Sundarrajan <sr...@hotmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/#review65772
-----------------------------------------------------------
Ship it!
Ship It!
- Srikanth Sundarrajan
On Dec. 22, 2014, 11:30 a.m., shwethags wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29092/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2014, 11:30 a.m.)
>
>
> Review request for Falcon.
>
>
> Bugs: FALCON-943
> https://issues.apache.org/jira/browse/FALCON-943
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Addresses the following:
> 1. Removed copying user workflow/lib to staging directory
> 2. Removed update at specific time
> 3. Copying falcon jars to parent workflow lib copies only specific falcon jars
> 4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
>
>
> Diffs
> -----
>
> client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
> client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
> common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
> common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
> common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
> common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
> common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
> common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
> docs/src/site/twiki/FalconCLI.twiki 202af63
> docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
> docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
> oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
> oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
> oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
> oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
> oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
> oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
> oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
> oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
> oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
> oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
> oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
> oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
> oozie/src/main/resources/action/post-process.xml 440a131
> oozie/src/main/resources/action/pre-process.xml 070c42b
> oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
> prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
> prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
> rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
> rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
> src/main/assemblies/distributed-package.xml 7320312
> src/main/assemblies/standalone-package.xml f9d6009
> src/main/examples/data/generate.sh 74ec3ba
> webapp/src/main/java/org/apache/falcon/Debug.java c606074
> webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
> webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
> webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
> webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
>
> Diff: https://reviews.apache.org/r/29092/diff/
>
>
> Testing
> -------
>
> UTs, end to end test for process, retention and replication
>
>
> Thanks,
>
> shwethags
>
>
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by sh...@gmail.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/
-----------------------------------------------------------
(Updated Dec. 22, 2014, 11:30 a.m.)
Review request for Falcon.
Changes
-------
Removed renaming third part jars on cluster
Bugs: FALCON-943
https://issues.apache.org/jira/browse/FALCON-943
Repository: falcon-git
Description
-------
Addresses the following:
1. Removed copying user workflow/lib to staging directory
2. Removed update at specific time
3. Copying falcon jars to parent workflow lib copies only specific falcon jars
4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
Diffs (updated)
-----
client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
docs/src/site/twiki/FalconCLI.twiki 202af63
docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
oozie/src/main/resources/action/post-process.xml 440a131
oozie/src/main/resources/action/pre-process.xml 070c42b
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
src/main/assemblies/distributed-package.xml 7320312
src/main/assemblies/standalone-package.xml f9d6009
src/main/examples/data/generate.sh 74ec3ba
webapp/src/main/java/org/apache/falcon/Debug.java c606074
webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
Diff: https://reviews.apache.org/r/29092/diff/
Testing
-------
UTs, end to end test for process, retention and replication
Thanks,
shwethags
Re: Review Request 29092: FALCON-943 process update copying user lib
is very slow
Posted by sh...@inmobi.com.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29092/
-----------------------------------------------------------
(Updated Dec. 18, 2014, 10:32 a.m.)
Review request for Falcon.
Changes
-------
Incorporated review comments
Bugs: FALCON-943
https://issues.apache.org/jira/browse/FALCON-943
Repository: falcon-git
Description
-------
Addresses the following:
1. Removed copying user workflow/lib to staging directory
2. Removed update at specific time
3. Copying falcon jars to parent workflow lib copies only specific falcon jars
4. Libs copied at cluster submit(3rd party jars) have versions so that its easier to debug issues
Diffs (updated)
-----
client/src/main/java/org/apache/falcon/cli/FalconCLI.java 93776d3
client/src/main/java/org/apache/falcon/client/FalconClient.java 23c8943
common/src/main/java/org/apache/falcon/entity/EntityUtil.java 59e43fb
common/src/main/java/org/apache/falcon/entity/FileSystemStorage.java bbe274b
common/src/main/java/org/apache/falcon/entity/ProcessHelper.java 174f8f6
common/src/main/java/org/apache/falcon/update/UpdateHelper.java 5a86ae3
common/src/main/java/org/apache/falcon/workflow/engine/AbstractWorkflowEngine.java ae158f7
common/src/main/resources/startup.properties 433c2a8
common/src/test/java/org/apache/falcon/update/UpdateHelperTest.java ef0d769
docs/src/site/twiki/FalconCLI.twiki 202af63
docs/src/site/twiki/FalconDocumentation.twiki 0d2a692
docs/src/site/twiki/restapi/EntityUpdate.twiki f2c2e7e
oozie-el-extensions/src/main/conf/oozie-site.xml PRE-CREATION
oozie-el-extensions/src/test/resources/oozie-site.xml a106aa6
oozie/src/main/java/org/apache/falcon/oozie/OozieBundleBuilder.java c73401a
oozie/src/main/java/org/apache/falcon/oozie/OozieCoordinatorBuilder.java 2ceb91e
oozie/src/main/java/org/apache/falcon/oozie/OozieEntityBuilder.java e341fb8
oozie/src/main/java/org/apache/falcon/oozie/OozieOrchestrationWorkflowBuilder.java 771295f
oozie/src/main/java/org/apache/falcon/oozie/feed/FeedBundleBuilder.java 3347fbf
oozie/src/main/java/org/apache/falcon/oozie/process/HiveProcessWorkflowBuilder.java 1db4ca4
oozie/src/main/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilder.java 14668f0
oozie/src/main/java/org/apache/falcon/oozie/process/PigProcessWorkflowBuilder.java 6bd5dd8
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessBundleBuilder.java 8691ee5
oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionWorkflowBuilder.java d271695
oozie/src/main/java/org/apache/falcon/service/SharedLibraryHostingService.java 9567c5f
oozie/src/main/java/org/apache/falcon/workflow/LateDataHandler.java PRE-CREATION
oozie/src/main/java/org/apache/falcon/workflow/engine/OozieWorkflowEngine.java 745aaac
oozie/src/main/resources/action/post-process.xml 440a131
oozie/src/main/resources/action/pre-process.xml 070c42b
oozie/src/test/java/org/apache/falcon/oozie/process/OozieProcessWorkflowBuilderTest.java 3477258
prism/src/main/java/org/apache/falcon/resource/AbstractEntityManager.java b6e1cec
prism/src/main/java/org/apache/falcon/resource/proxy/SchedulableEntityManagerProxy.java fb9d9f3
rerun/src/main/java/org/apache/falcon/latedata/LateDataHandler.java d35abfa
rerun/src/main/java/org/apache/falcon/rerun/handler/LateRerunConsumer.java 9ba632e
src/conf/startup.properties 2db4b1e
src/main/assemblies/distributed-package.xml 7320312
src/main/assemblies/standalone-package.xml f9d6009
src/main/examples/data/generate.sh 74ec3ba
webapp/src/main/java/org/apache/falcon/Debug.java c606074
webapp/src/main/java/org/apache/falcon/resource/ConfigSyncService.java 98b8b3d
webapp/src/test/java/org/apache/falcon/cli/FalconCLIIT.java a3acbdb
webapp/src/test/java/org/apache/falcon/late/LateDataHandlerIT.java 1885bb7
webapp/src/test/java/org/apache/falcon/resource/EntityManagerJerseyIT.java f7e6bdb
Diff: https://reviews.apache.org/r/29092/diff/
Testing
-------
UTs, end to end test for process, retention and replication
Thanks,
shwethags