You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by Ryota Egashira <eg...@yahoo-inc.com> on 2014/10/31 18:43:39 UTC
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/
-----------------------------------------------------------
(Updated Oct. 31, 2014, 5:43 p.m.)
Review request for oozie.
Changes
-------
with HCatalog team help, got it work, getting delegation token from HCat as oozie server, and then delete partition using doAs(as user). verified in the test environment with kerberos
Bugs: OOZIE-1985
https://issues.apache.org/jira/browse/OOZIE-1985
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-1985
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 65338a3
core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
Diff: https://reviews.apache.org/r/25269/diff/
Testing
-------
Thanks,
Ryota Egashira
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/#review59601
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/ErrorCode.java
<https://reviews.apache.org/r/25269/#comment100895>
One code for "Failed to delete uri : {0}" should be good enough. User can make out hdfs or hcat from the uri.
core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java
<https://reviews.apache.org/r/25269/#comment100898>
FileSystem fs = getFileSystem(uri, conf, user);
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100903>
HCatClient client = getHCatClient(uri, conf, user);
client.dropPartitions(hcatUri.getDb(), hcatUri.getTable(), hcatUri.getPartitionMap(), true);
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100904>
Please change this method to getHCatClient(uri, conf, user) which only returns HCatClient;
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100905>
Inline this method into getHCatClient()
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100902>
Make this method as getHCatClient(URI uri, Configuration conf) which retuns HCatClient for Oozie server and add new code in getHCatClient(URI uri, Configuration conf, String user)
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment100901>
Need to these changes. XLog.getLog(HCatURIHandler.class) is required to get the latest log prefix information.
core/src/main/java/org/apache/oozie/dependency/URIHandler.java
<https://reviews.apache.org/r/25269/#comment100900>
* Delete a URI
* @param uri URI
- Rohini Palaniswamy
On Oct. 31, 2014, 5:43 p.m., Ryota Egashira wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25269/
> -----------------------------------------------------------
>
> (Updated Oct. 31, 2014, 5:43 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1985
> https://issues.apache.org/jira/browse/OOZIE-1985
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-1985
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
> core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
> core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
> core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
> core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 65338a3
> core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
>
> Diff: https://reviews.apache.org/r/25269/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ryota Egashira
>
>
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/#review62309
-----------------------------------------------------------
Ship it!
Ship It!
- Rohini Palaniswamy
On Nov. 19, 2014, 11:28 a.m., Ryota Egashira wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25269/
> -----------------------------------------------------------
>
> (Updated Nov. 19, 2014, 11:28 a.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1985
> https://issues.apache.org/jira/browse/OOZIE-1985
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-1985
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
> core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
> core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 3bb191e
> core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
> core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
> core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java ac023ca
> core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
>
> Diff: https://reviews.apache.org/r/25269/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ryota Egashira
>
>
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
Posted by Ryota Egashira <eg...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/
-----------------------------------------------------------
(Updated Nov. 19, 2014, 11:28 a.m.)
Review request for oozie.
Changes
-------
fixed comments. final version(hopefully)
Bugs: OOZIE-1985
https://issues.apache.org/jira/browse/OOZIE-1985
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-1985
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 3bb191e
core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java ac023ca
core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
Diff: https://reviews.apache.org/r/25269/diff/
Testing
-------
Thanks,
Ryota Egashira
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
Posted by Rohini Palaniswamy <ro...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/#review61538
-----------------------------------------------------------
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java
<https://reviews.apache.org/r/25269/#comment103233>
Add comment saying, read operations are allowed for all in hcat and so accessing as Oozie server itself. For write operations, doing doAs as user.
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java
<https://reviews.apache.org/r/25269/#comment103231>
keep the old method and overload with a new one.
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java
<https://reviews.apache.org/r/25269/#comment103230>
refresh
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java
<https://reviews.apache.org/r/25269/#comment103237>
Why do you need a transaction for read operation? Can you change to jpaexecutor service
- Rohini Palaniswamy
On Nov. 11, 2014, 6:26 p.m., Ryota Egashira wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25269/
> -----------------------------------------------------------
>
> (Updated Nov. 11, 2014, 6:26 p.m.)
>
>
> Review request for oozie.
>
>
> Bugs: OOZIE-1985
> https://issues.apache.org/jira/browse/OOZIE-1985
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> https://issues.apache.org/jira/browse/OOZIE-1985
>
>
> Diffs
> -----
>
> core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
> core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
> core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 3bb191e
> core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
> core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
> core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
> core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 65338a3
> core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
>
> Diff: https://reviews.apache.org/r/25269/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Ryota Egashira
>
>
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
Posted by Ryota Egashira <eg...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/
-----------------------------------------------------------
(Updated Nov. 11, 2014, 6:26 p.m.)
Review request for oozie.
Changes
-------
added context destroy in CoordRerun Command, and added read-only flag to URIHandler.getContext signature
Bugs: OOZIE-1985
https://issues.apache.org/jira/browse/OOZIE-1985
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-1985
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 3bb191e
core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 65338a3
core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
Diff: https://reviews.apache.org/r/25269/diff/
Testing
-------
Thanks,
Ryota Egashira
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
Posted by Ryota Egashira <eg...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/
-----------------------------------------------------------
(Updated Nov. 7, 2014, 12:20 a.m.)
Review request for oozie.
Changes
-------
add use of context, and HCatClient.cancelDelegationToken
Bugs: OOZIE-1985
https://issues.apache.org/jira/browse/OOZIE-1985
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-1985
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 65338a3
core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
Diff: https://reviews.apache.org/r/25269/diff/
Testing
-------
Thanks,
Ryota Egashira
Re: Review Request 25269: OOZIE-1985 support dropping hcat dataset in
coord rerun with cleanup option
Posted by Ryota Egashira <eg...@yahoo-inc.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25269/
-----------------------------------------------------------
(Updated Nov. 3, 2014, 11:54 p.m.)
Review request for oozie.
Changes
-------
fixed rohini's review comments
Bugs: OOZIE-1985
https://issues.apache.org/jira/browse/OOZIE-1985
Repository: oozie-git
Description
-------
https://issues.apache.org/jira/browse/OOZIE-1985
Diffs (updated)
-----
core/src/main/java/org/apache/oozie/ErrorCode.java 4afeb6c
core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java 098eb4b
core/src/main/java/org/apache/oozie/dependency/FSURIHandler.java a8f548a
core/src/main/java/org/apache/oozie/dependency/HCatURIHandler.java 629033b
core/src/main/java/org/apache/oozie/dependency/URIHandler.java 7280902
core/src/test/java/org/apache/oozie/command/coord/TestCoordRerunXCommand.java 65338a3
core/src/test/java/org/apache/oozie/dependency/TestHCatURIHandler.java 31a68bf
Diff: https://reviews.apache.org/r/25269/diff/
Testing
-------
Thanks,
Ryota Egashira