You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Mohammad Islam <mi...@yahoo.com> on 2012/11/25 05:10:33 UTC
Review Request: OOZIE-1086 Command to check the missing partitions directly
against HCatalog server
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/
-----------------------------------------------------------
Review request for oozie.
Description
-------
The same will be used in multiple scenarios:
1. As part of action materialization.
2. Recovery service
This addresses bug OOZIE-1086.
https://issues.apache.org/jira/browse/OOZIE-1086
Diffs
-----
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java 1411494
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1411608
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
Diff: https://reviews.apache.org/r/8210/diff/
Testing
-------
Thanks,
Mohammad Islam
Re: Review Request: OOZIE-1086 Command to check the missing partitions
directly against HCatalog server
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/#review13781
-----------------------------------------------------------
Ship it!
Okay my comments are covered then. Thanks! +1
- Mona Chitnis
On Nov. 27, 2012, 7:24 p.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8210/
> -----------------------------------------------------------
>
> (Updated Nov. 27, 2012, 7:24 p.m.)
>
>
> Review request for oozie.
>
>
> Description
> -------
>
> The same will be used in multiple scenarios:
> 1. As part of action materialization.
> 2. Recovery service
>
>
> This addresses bug OOZIE-1086.
> https://issues.apache.org/jira/browse/OOZIE-1086
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1414006
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionUpdatePushMissingDependency.java 1414006
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8210/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request: OOZIE-1086 Command to check the missing partitions
directly against HCatalog server
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/
-----------------------------------------------------------
(Updated Nov. 27, 2012, 7:24 p.m.)
Review request for oozie.
Changes
-------
Updated with Mona's comments.
Description
-------
The same will be used in multiple scenarios:
1. As part of action materialization.
2. Recovery service
This addresses bug OOZIE-1086.
https://issues.apache.org/jira/browse/OOZIE-1086
Diffs (updated)
-----
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1414006
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionUpdatePushMissingDependency.java 1414006
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
Diff: https://reviews.apache.org/r/8210/diff/
Testing
-------
Thanks,
Mohammad Islam
Re: Review Request: OOZIE-1086 Command to check the missing partitions
directly against HCatalog server
Posted by Mohammad Islam <mi...@yahoo.com>.
> On Nov. 27, 2012, 6:57 p.m., Mona Chitnis wrote:
> > No minor nitpicks. But two overall comments
> >
> > 1. A different HCat client is getting instantiated for each such inputCheck command through the MetaDataAccessorService's getHCatClient method. Should we use an instance pool instead?
> >
> > 2. Will this class become obsolete once we implement Schema Handler logic? Because then, a common place - ActionInputCheck should determine internally whether to go to HDFS Filesystem or MetaDataClientWrapper for querying. Thoughts?
1. As per Rohini, HCatClient internally use the caching. We don't need it explicitly.
2. We still need it. Had a chat with Rohini on this.
- Mohammad
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/#review13771
-----------------------------------------------------------
On Nov. 27, 2012, 1:45 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8210/
> -----------------------------------------------------------
>
> (Updated Nov. 27, 2012, 1:45 a.m.)
>
>
> Review request for oozie.
>
>
> Description
> -------
>
> The same will be used in multiple scenarios:
> 1. As part of action materialization.
> 2. Recovery service
>
>
> This addresses bug OOZIE-1086.
> https://issues.apache.org/jira/browse/OOZIE-1086
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1411608
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionUpdatePushMissingDependency.java 1411494
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8210/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request: OOZIE-1086 Command to check the missing partitions
directly against HCatalog server
Posted by Mona Chitnis <ch...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/#review13771
-----------------------------------------------------------
No minor nitpicks. But two overall comments
1. A different HCat client is getting instantiated for each such inputCheck command through the MetaDataAccessorService's getHCatClient method. Should we use an instance pool instead?
2. Will this class become obsolete once we implement Schema Handler logic? Because then, a common place - ActionInputCheck should determine internally whether to go to HDFS Filesystem or MetaDataClientWrapper for querying. Thoughts?
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
<https://reviews.apache.org/r/8210/#comment29448>
loadState() is supposed to check this beforehand, I believe its not required here
- Mona Chitnis
On Nov. 27, 2012, 1:45 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8210/
> -----------------------------------------------------------
>
> (Updated Nov. 27, 2012, 1:45 a.m.)
>
>
> Review request for oozie.
>
>
> Description
> -------
>
> The same will be used in multiple scenarios:
> 1. As part of action materialization.
> 2. Recovery service
>
>
> This addresses bug OOZIE-1086.
> https://issues.apache.org/jira/browse/OOZIE-1086
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1411608
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionUpdatePushMissingDependency.java 1411494
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8210/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request: OOZIE-1086 Command to check the missing partitions
directly against HCatalog server
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/#review13779
-----------------------------------------------------------
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java
<https://reviews.apache.org/r/8210/#comment29454>
correct. taking this out.
- Mohammad Islam
On Nov. 27, 2012, 1:45 a.m., Mohammad Islam wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8210/
> -----------------------------------------------------------
>
> (Updated Nov. 27, 2012, 1:45 a.m.)
>
>
> Review request for oozie.
>
>
> Description
> -------
>
> The same will be used in multiple scenarios:
> 1. As part of action materialization.
> 2. Recovery service
>
>
> This addresses bug OOZIE-1086.
> https://issues.apache.org/jira/browse/OOZIE-1086
>
>
> Diffs
> -----
>
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1411608
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionUpdatePushMissingDependency.java 1411494
> http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8210/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Mohammad Islam
>
>
Re: Review Request: OOZIE-1086 Command to check the missing partitions
directly against HCatalog server
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/
-----------------------------------------------------------
(Updated Nov. 27, 2012, 1:45 a.m.)
Review request for oozie.
Description
-------
The same will be used in multiple scenarios:
1. As part of action materialization.
2. Recovery service
This addresses bug OOZIE-1086.
https://issues.apache.org/jira/browse/OOZIE-1086
Diffs
-----
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1411608
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionUpdatePushMissingDependency.java 1411494
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
Diff: https://reviews.apache.org/r/8210/diff/
Testing
-------
Thanks,
Mohammad Islam
Re: Review Request: OOZIE-1086 Command to check the missing partitions
directly against HCatalog server
Posted by Mohammad Islam <mi...@yahoo.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8210/
-----------------------------------------------------------
(Updated Nov. 27, 2012, 1:45 a.m.)
Review request for oozie.
Changes
-------
Added License headers to new file
Description
-------
The same will be used in multiple scenarios:
1. As part of action materialization.
2. Recovery service
This addresses bug OOZIE-1086.
https://issues.apache.org/jira/browse/OOZIE-1086
Diffs (updated)
-----
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/command/coord/CoordPushDependencyCheckXCommand.java PRE-CREATION
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/main/java/org/apache/oozie/hcat/MetaDataClientWrapper.java 1411608
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordActionUpdatePushMissingDependency.java 1411494
http://svn.apache.org/repos/asf/oozie/branches/hcat-intre/core/src/test/java/org/apache/oozie/command/coord/TestCoordPushDependencyCheckXCommand.java PRE-CREATION
Diff: https://reviews.apache.org/r/8210/diff/
Testing
-------
Thanks,
Mohammad Islam