You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Dian Fu <di...@gmail.com> on 2015/12/21 10:39:03 UTC
Review Request 41602: Sqoop2: Impersonation is missing for kite
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41602/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-2752
https://issues.apache.org/jira/browse/SQOOP-2752
Repository: sqoop-sqoop2
Description
-------
As kite needs to access HDFS cluster if the dateset uri is "dataset:hdfs:...", so we should add impersonation support for Kite in the same way as SQOOP-2709.
Diffs
-----
common/src/main/java/org/apache/sqoop/error/code/KiteConnectorError.java 7db9904
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConstants.java f06300a
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 441fe30
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 3a0d626
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java a6551e6
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java d01e932
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 858042c
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java 204c978
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/security/SecurityUtils.java 0a42936
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/security/TestSecurityUtils.java 713c704
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteExtractor.java d93f9b5
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteFromInitializer.java 28c5bac
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteLoader.java ca0a5c7
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToDestroyer.java fb83f2b
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToInitializer.java 50daba0
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteUtils.java PRE-CREATION
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ConfigUtil.java e63bccf
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfig.java ee31f15
connector/connector-sdk/pom.xml 0cbbb04
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/security/SecurityUtils.java PRE-CREATION
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/security/TestSecurityUtils.java PRE-CREATION
test/src/test/java/org/apache/sqoop/integration/connector/kite/FromRDBMSToKiteTest.java 7b2aced
Diff: https://reviews.apache.org/r/41602/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 41602: Sqoop2: Impersonation is missing for kite
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41602/#review111962
-----------------------------------------------------------
Ship it!
Ship It!
- Jarek Cecho
On Dec. 28, 2015, 1:10 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41602/
> -----------------------------------------------------------
>
> (Updated Dec. 28, 2015, 1:10 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2752
> https://issues.apache.org/jira/browse/SQOOP-2752
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> As kite needs to access HDFS cluster if the dateset uri is "dataset:hdfs:...", so we should add impersonation support for Kite in the same way as SQOOP-2709.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/error/code/KiteConnectorError.java 7db9904
> connector/connector-kite/pom.xml 0792445
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteExtractor.java d93f9b5
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteFromInitializer.java 4502d59
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteLoader.java ca0a5c7
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToDestroyer.java fb83f2b
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToInitializer.java effab19
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteUtils.java PRE-CREATION
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ConfigUtil.java e63bccf
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfig.java ee31f15
> connector/connector-kite/src/main/resources/kite-connector-config.properties c134ac3
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteExtractor.java c49be92
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteFromInitializer.java 6df5d83
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteLoader.java c5aa1bd
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToDestroyer.java 00b8871
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToInitializer.java 5230ffe
> test/src/test/java/org/apache/sqoop/integration/connector/kite/FromRDBMSToKiteTest.java 7b2aced
>
> Diff: https://reviews.apache.org/r/41602/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 41602: Sqoop2: Impersonation is missing for kite
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41602/
-----------------------------------------------------------
(Updated Dec. 28, 2015, 1:10 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2752
https://issues.apache.org/jira/browse/SQOOP-2752
Repository: sqoop-sqoop2
Description
-------
As kite needs to access HDFS cluster if the dateset uri is "dataset:hdfs:...", so we should add impersonation support for Kite in the same way as SQOOP-2709.
Diffs (updated)
-----
common/src/main/java/org/apache/sqoop/error/code/KiteConnectorError.java 7db9904
connector/connector-kite/pom.xml 0792445
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteExtractor.java d93f9b5
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteFromInitializer.java 4502d59
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteLoader.java ca0a5c7
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToDestroyer.java fb83f2b
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToInitializer.java effab19
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteUtils.java PRE-CREATION
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ConfigUtil.java e63bccf
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfig.java ee31f15
connector/connector-kite/src/main/resources/kite-connector-config.properties c134ac3
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteExtractor.java c49be92
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteFromInitializer.java 6df5d83
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteLoader.java c5aa1bd
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToDestroyer.java 00b8871
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToInitializer.java 5230ffe
test/src/test/java/org/apache/sqoop/integration/connector/kite/FromRDBMSToKiteTest.java 7b2aced
Diff: https://reviews.apache.org/r/41602/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 41602: Sqoop2: Impersonation is missing for kite
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41602/#review111836
-----------------------------------------------------------
Overall looks good, but I do have one of high level suggestions:
I would prefer not to put hadoop as a depedency on connector-sdk. This would force every connector to synchronize with Hadoop dependencies (guava and such). Can we instead create a new module "connector-sdk-hadoop" that will contain Hadoop specific SDK classes and will in turn contain this Hadoop dependency (arguably if the connector needs something from Hadoop they should be fine with having hadoop jars on classpath). This way the connectors that doesn't need anything from hadoop don't end up having Hadoop's guava (and others) on the classpath. I would also do this as part of separate JIRA.
- Jarek Cecho
On Dec. 22, 2015, 4:55 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41602/
> -----------------------------------------------------------
>
> (Updated Dec. 22, 2015, 4:55 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2752
> https://issues.apache.org/jira/browse/SQOOP-2752
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> As kite needs to access HDFS cluster if the dateset uri is "dataset:hdfs:...", so we should add impersonation support for Kite in the same way as SQOOP-2709.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/error/code/KiteConnectorError.java 7db9904
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConstants.java f06300a
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 441fe30
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 3a0d626
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java a6551e6
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java d01e932
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 858042c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java 204c978
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/security/SecurityUtils.java 0a42936
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/security/TestSecurityUtils.java 713c704
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteExtractor.java d93f9b5
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteFromInitializer.java 28c5bac
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteLoader.java ca0a5c7
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToDestroyer.java fb83f2b
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToInitializer.java 50daba0
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteUtils.java PRE-CREATION
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ConfigUtil.java e63bccf
> connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfig.java ee31f15
> connector/connector-kite/src/main/resources/kite-connector-config.properties c134ac3
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteExtractor.java c49be92
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteFromInitializer.java 6df5d83
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteLoader.java c5aa1bd
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToDestroyer.java 00b8871
> connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToInitializer.java 5230ffe
> connector/connector-sdk/pom.xml 0cbbb04
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/security/SecurityUtils.java PRE-CREATION
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/security/TestSecurityUtils.java PRE-CREATION
> test/src/test/java/org/apache/sqoop/integration/connector/kite/FromRDBMSToKiteTest.java 7b2aced
>
> Diff: https://reviews.apache.org/r/41602/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 41602: Sqoop2: Impersonation is missing for kite
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41602/
-----------------------------------------------------------
(Updated Dec. 22, 2015, 4:55 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2752
https://issues.apache.org/jira/browse/SQOOP-2752
Repository: sqoop-sqoop2
Description
-------
As kite needs to access HDFS cluster if the dateset uri is "dataset:hdfs:...", so we should add impersonation support for Kite in the same way as SQOOP-2709.
Diffs (updated)
-----
common/src/main/java/org/apache/sqoop/error/code/KiteConnectorError.java 7db9904
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConstants.java f06300a
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 441fe30
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 3a0d626
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java a6551e6
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java d01e932
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 858042c
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java 204c978
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/security/SecurityUtils.java 0a42936
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/security/TestSecurityUtils.java 713c704
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteExtractor.java d93f9b5
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteFromInitializer.java 28c5bac
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteLoader.java ca0a5c7
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToDestroyer.java fb83f2b
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToInitializer.java 50daba0
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteUtils.java PRE-CREATION
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ConfigUtil.java e63bccf
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfig.java ee31f15
connector/connector-kite/src/main/resources/kite-connector-config.properties c134ac3
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteExtractor.java c49be92
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteFromInitializer.java 6df5d83
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteLoader.java c5aa1bd
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToDestroyer.java 00b8871
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToInitializer.java 5230ffe
connector/connector-sdk/pom.xml 0cbbb04
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/security/SecurityUtils.java PRE-CREATION
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/security/TestSecurityUtils.java PRE-CREATION
test/src/test/java/org/apache/sqoop/integration/connector/kite/FromRDBMSToKiteTest.java 7b2aced
Diff: https://reviews.apache.org/r/41602/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 41602: Sqoop2: Impersonation is missing for kite
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41602/
-----------------------------------------------------------
(Updated Dec. 22, 2015, 2:09 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2752
https://issues.apache.org/jira/browse/SQOOP-2752
Repository: sqoop-sqoop2
Description
-------
As kite needs to access HDFS cluster if the dateset uri is "dataset:hdfs:...", so we should add impersonation support for Kite in the same way as SQOOP-2709.
Diffs (updated)
-----
common/src/main/java/org/apache/sqoop/error/code/KiteConnectorError.java 7db9904
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConstants.java f06300a
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 441fe30
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 3a0d626
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java a6551e6
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java d01e932
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 858042c
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java 204c978
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/security/SecurityUtils.java 0a42936
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/security/TestSecurityUtils.java 713c704
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteExtractor.java d93f9b5
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteFromInitializer.java 28c5bac
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteLoader.java ca0a5c7
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToDestroyer.java fb83f2b
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteToInitializer.java 50daba0
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/KiteUtils.java PRE-CREATION
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/ConfigUtil.java e63bccf
connector/connector-kite/src/main/java/org/apache/sqoop/connector/kite/configuration/LinkConfig.java ee31f15
connector/connector-kite/src/main/resources/kite-connector-config.properties c134ac3
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteExtractor.java c49be92
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteFromInitializer.java 6df5d83
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteLoader.java c5aa1bd
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToDestroyer.java 00b8871
connector/connector-kite/src/test/java/org/apache/sqoop/connector/kite/TestKiteToInitializer.java 5230ffe
connector/connector-sdk/pom.xml 0cbbb04
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/security/SecurityUtils.java PRE-CREATION
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/security/TestSecurityUtils.java PRE-CREATION
test/src/test/java/org/apache/sqoop/integration/connector/kite/FromRDBMSToKiteTest.java 7b2aced
Diff: https://reviews.apache.org/r/41602/diff/
Testing
-------
Thanks,
Dian Fu