You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Abraham Elmahrek <ab...@cloudera.com> on 2014/10/23 23:04:04 UTC
Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-1620
https://issues.apache.org/jira/browse/SQOOP-1620
Repository: sqoop-sqoop2
Description
-------
commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Fri Oct 10 19:02:27 2014 -0700
SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
:100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
:100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
:100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
:100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
:100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
:100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
:000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
:000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
:100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
Diff: https://reviews.apache.org/r/27110/diff/
Testing
-------
mvn test and manually tested working order.
Thanks,
Abraham Elmahrek
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 25, 2014, 12:41 a.m., Veena Basavaraj wrote:
> > test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java, line 95
> > <https://reviews.apache.org/r/27110/diff/3/?file=733158#file733158line95>
> >
> > please please extract this logic to common method, when I refactor I notice this more
Are you referring to the statement where we modify configs?
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58456
-----------------------------------------------------------
On Oct. 25, 2014, 12:12 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2014, 12:12 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Veena Basavaraj <vb...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58456
-----------------------------------------------------------
Ship it!
Nice. please address the following
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
<https://reviews.apache.org/r/27110/#comment99475>
please please extract this logic to common method, when I refactor I notice this more
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
<https://reviews.apache.org/r/27110/#comment99476>
ditto, same as above
- Veena Basavaraj
On Oct. 24, 2014, 5:12 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 5:12 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 27, 2014, 4:39 p.m., Gwen Shapira wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java, line 34
> > <https://reviews.apache.org/r/27110/diff/4/?file=733994#file733994line34>
> >
> > WTF :)
> >
> > I'll trust you (and the unit tests) that this monster actually does what we need.
> >
> > It will probably get replaced with a small parser the first time we need to debug this.
Haha yeah. I don't know if this is cosure, but I've copy pasted this from somewhere on the interwebz.
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58615
-----------------------------------------------------------
On Oct. 27, 2014, 8:22 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 27, 2014, 8:22 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 27, 2014, 4:39 p.m., Gwen Shapira wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java, line 34
> > <https://reviews.apache.org/r/27110/diff/4/?file=733994#file733994line34>
> >
> > WTF :)
> >
> > I'll trust you (and the unit tests) that this monster actually does what we need.
> >
> > It will probably get replaced with a small parser the first time we need to debug this.
>
> Abraham Elmahrek wrote:
> Haha yeah. I don't know if this is cosure, but I've copy pasted this from somewhere on the interwebz.
kosher*
Also, it might be cool to add a connectivity test eventually.
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58615
-----------------------------------------------------------
On Oct. 27, 2014, 8:22 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 27, 2014, 8:22 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Gwen Shapira <gs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58615
-----------------------------------------------------------
Ship it!
Tests pass. [+1 LGTM](http://www.lgtm.in/p/O1M5aqWKn)
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
<https://reviews.apache.org/r/27110/#comment99677>
WTF :)
I'll trust you (and the unit tests) that this monster actually does what we need.
It will probably get replaced with a small parser the first time we need to debug this.
- Gwen Shapira
On Oct. 27, 2014, 8:22 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 27, 2014, 8:22 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/
-----------------------------------------------------------
(Updated Oct. 27, 2014, 8:22 a.m.)
Review request for Sqoop.
Bugs: SQOOP-1620
https://issues.apache.org/jira/browse/SQOOP-1620
Repository: sqoop-sqoop2
Description
-------
commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Fri Oct 10 19:02:27 2014 -0700
SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
:100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
:100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
:100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
:100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
:100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
:100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
:000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
:000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
:100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLinkConfig.java PRE-CREATION
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
Diff: https://reviews.apache.org/r/27110/diff/
Testing
-------
mvn test and manually tested working order.
Thanks,
Abraham Elmahrek
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Veena Basavaraj <vb...@cloudera.com>.
> On Oct. 24, 2014, 5:45 p.m., Veena Basavaraj wrote:
> > I wondered about this after I said ship it.
> >
> > Now that we added new configs to the hdfs, should not the version change and upgrade code needs to be updated? I was thinking this is when the upgrader api will be needed.
>
> Gwen Shapira wrote:
> In general, your observation is spot-on. However, the HDFS connector did not exist before this version, so there is nothing to upgrade in this specific case.
>
> Abraham Elmahrek wrote:
> My understanding is that the version will be the same as Sqoop. Did we change versioning to version connectors separately? Also, the upgrader API should automatically handle the additive case automatically.
>
> Veena Basavaraj wrote:
> sure this makes sense Gwen.
>
> Abe -> No we have not changed, in another related RB https://reviews.apache.org/r/27196/ I had this question to Jarcec and we seem to use the sqoop build version for the connector and the comment in thecode says all connectors built in with sqoop will use the current sqoop version. So if we add a new config after 1.99.4 release then we will need the upgrade code to check for the version.
I meant edit an existing config to be precise then we need custom logic.
- Veena
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58461
-----------------------------------------------------------
On Oct. 24, 2014, 5:12 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 5:12 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 25, 2014, 12:45 a.m., Veena Basavaraj wrote:
> > I wondered about this after I said ship it.
> >
> > Now that we added new configs to the hdfs, should not the version change and upgrade code needs to be updated? I was thinking this is when the upgrader api will be needed.
>
> Gwen Shapira wrote:
> In general, your observation is spot-on. However, the HDFS connector did not exist before this version, so there is nothing to upgrade in this specific case.
My understanding is that the version will be the same as Sqoop. Did we change versioning to version connectors separately? Also, the upgrader API should automatically handle the additive case automatically.
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58461
-----------------------------------------------------------
On Oct. 25, 2014, 12:12 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2014, 12:12 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Gwen Shapira <gs...@cloudera.com>.
> On Oct. 25, 2014, 12:45 a.m., Veena Basavaraj wrote:
> > I wondered about this after I said ship it.
> >
> > Now that we added new configs to the hdfs, should not the version change and upgrade code needs to be updated? I was thinking this is when the upgrader api will be needed.
In general, your observation is spot-on. However, the HDFS connector did not exist before this version, so there is nothing to upgrade in this specific case.
- Gwen
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58461
-----------------------------------------------------------
On Oct. 25, 2014, 12:12 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2014, 12:12 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Veena Basavaraj <vb...@cloudera.com>.
> On Oct. 24, 2014, 5:45 p.m., Veena Basavaraj wrote:
> > I wondered about this after I said ship it.
> >
> > Now that we added new configs to the hdfs, should not the version change and upgrade code needs to be updated? I was thinking this is when the upgrader api will be needed.
>
> Gwen Shapira wrote:
> In general, your observation is spot-on. However, the HDFS connector did not exist before this version, so there is nothing to upgrade in this specific case.
>
> Abraham Elmahrek wrote:
> My understanding is that the version will be the same as Sqoop. Did we change versioning to version connectors separately? Also, the upgrader API should automatically handle the additive case automatically.
sure this makes sense Gwen.
Abe -> No we have not changed, in another related RB https://reviews.apache.org/r/27196/ I had this question to Jarcec and we seem to use the sqoop build version for the connector and the comment in thecode says all connectors built in with sqoop will use the current sqoop version. So if we add a new config after 1.99.4 release then we will need the upgrade code to check for the version.
- Veena
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58461
-----------------------------------------------------------
On Oct. 24, 2014, 5:12 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 5:12 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Veena Basavaraj <vb...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58461
-----------------------------------------------------------
I wondered about this after I said ship it.
Now that we added new configs to the hdfs, should not the version change and upgrade code needs to be updated? I was thinking this is when the upgrader api will be needed.
- Veena Basavaraj
On Oct. 24, 2014, 5:12 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 5:12 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Veena Basavaraj <vb...@cloudera.com>.
> On Oct. 26, 2014, 3:39 p.m., Gwen Shapira wrote:
> > Hate to contradict Veena, but I don't think this is ready yet. For one thing, the integration tests this patch touches are failing on my machine, and the validator looks broken as well.
hehe, I did not run the patch, I proabably should have. I am glad you did not have to agree:)
> On Oct. 26, 2014, 3:39 p.m., Gwen Shapira wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java, line 34
> > <https://reviews.apache.org/r/27110/diff/3/?file=733151#file733151line34>
> >
> > I love the idea of verifying the URI, but something looks wrong here.
> >
> > 1. The regex assumes two ":" are mandatory, i.e. we'll always have a port number. But hdfs:///tmp and hdfs://ns1/tmp are both valid.
> > 2. It looks like we only allow a single character between the ":"? i.e. hdfs://a:8920 matches but hdfs://abcd:8020 does not. This is obviously broken.
> >
> > So, please add unit tests for this.
very good catch.
> On Oct. 26, 2014, 3:39 p.m., Gwen Shapira wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java, lines 41-43
> > <https://reviews.apache.org/r/27110/diff/3/?file=733151#file733151line41>
> >
> > Users will love it if we give an example for a legal URI in the error message. i.e: "... for example the URI hdfs://nameservice1:8020/sqoop is valid". It gives them something to copy paste.
love this comment too Gwen! I love making anything that makes users life happy.
- Veena
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58574
-----------------------------------------------------------
On Oct. 24, 2014, 5:12 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 5:12 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 26, 2014, 10:39 p.m., Gwen Shapira wrote:
> > Hate to contradict Veena, but I don't think this is ready yet. For one thing, the integration tests this patch touches are failing on my machine, and the validator looks broken as well.
>
> Veena Basavaraj wrote:
> hehe, I did not run the patch, I proabably should have. I am glad you did not have to agree:)
Indeed, the validator wasn't originally running. This is also why the integration tests are now failing. Fixed.
> On Oct. 26, 2014, 10:39 p.m., Gwen Shapira wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java, line 25
> > <https://reviews.apache.org/r/27110/diff/3/?file=733150#file733150line25>
> >
> > Not happy about a super generic name for a method that only handles URIs (and not compression codecs for example). Is the plan to use this for all configurations in the future? If so, please leave a comment to that effect (and perhaps double check if its even feasible).
Good point. Changing to configureURI.
It would be cool to have a single place where we provide properties for a Configuration object. It would make troubleshooting and testing easier. The codec is separate though... Sqoop loads the codec based on its own configuration and the filename. It's not yet used in the Configuration object other than producing a set of supported codecs (which we don't configure).
> On Oct. 26, 2014, 10:39 p.m., Gwen Shapira wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java, line 34
> > <https://reviews.apache.org/r/27110/diff/3/?file=733151#file733151line34>
> >
> > I love the idea of verifying the URI, but something looks wrong here.
> >
> > 1. The regex assumes two ":" are mandatory, i.e. we'll always have a port number. But hdfs:///tmp and hdfs://ns1/tmp are both valid.
> > 2. It looks like we only allow a single character between the ":"? i.e. hdfs://a:8920 matches but hdfs://abcd:8020 does not. This is obviously broken.
> >
> > So, please add unit tests for this.
>
> Veena Basavaraj wrote:
> very good catch.
Good point. For instance, file:/// should be supported as well. Maybe http://blog.dieweltistgarnichtso.net/constructing-a-regular-expression-that-matches-uris would work out for us.
> On Oct. 26, 2014, 10:39 p.m., Gwen Shapira wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java, lines 41-43
> > <https://reviews.apache.org/r/27110/diff/3/?file=733151#file733151line41>
> >
> > Users will love it if we give an example for a legal URI in the error message. i.e: "... for example the URI hdfs://nameservice1:8020/sqoop is valid". It gives them something to copy paste.
>
> Veena Basavaraj wrote:
> love this comment too Gwen! I love making anything that makes users life happy.
Good point!
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58574
-----------------------------------------------------------
On Oct. 25, 2014, 12:12 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2014, 12:12 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Gwen Shapira <gs...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58574
-----------------------------------------------------------
Hate to contradict Veena, but I don't think this is ready yet. For one thing, the integration tests this patch touches are failing on my machine, and the validator looks broken as well.
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
<https://reviews.apache.org/r/27110/#comment99632>
Not happy about a super generic name for a method that only handles URIs (and not compression codecs for example). Is the plan to use this for all configurations in the future? If so, please leave a comment to that effect (and perhaps double check if its even feasible).
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
<https://reviews.apache.org/r/27110/#comment99630>
I love the idea of verifying the URI, but something looks wrong here.
1. The regex assumes two ":" are mandatory, i.e. we'll always have a port number. But hdfs:///tmp and hdfs://ns1/tmp are both valid.
2. It looks like we only allow a single character between the ":"? i.e. hdfs://a:8920 matches but hdfs://abcd:8020 does not. This is obviously broken.
So, please add unit tests for this.
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
<https://reviews.apache.org/r/27110/#comment99631>
Users will love it if we give an example for a legal URI in the error message. i.e: "... for example the URI hdfs://nameservice1:8020/sqoop is valid". It gives them something to copy paste.
- Gwen Shapira
On Oct. 25, 2014, 12:12 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 25, 2014, 12:12 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/
-----------------------------------------------------------
(Updated Oct. 25, 2014, 12:12 a.m.)
Review request for Sqoop.
Bugs: SQOOP-1620
https://issues.apache.org/jira/browse/SQOOP-1620
Repository: sqoop-sqoop2
Description
-------
commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Fri Oct 10 19:02:27 2014 -0700
SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
:100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
:100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
:100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
:100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
:100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
:100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
:000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
:000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
:100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
Diff: https://reviews.apache.org/r/27110/diff/
Testing
-------
mvn test and manually tested working order.
Thanks,
Abraham Elmahrek
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Oct. 24, 2014, 3:22 a.m., Qian Xu wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java, line 28
> > <https://reviews.apache.org/r/27110/diff/2/?file=731285#file731285line28>
> >
> > Please change to `@ConfigClass(validator = {@Validator(LinkConfig.ConfigValidator.class)})`, otherwise it will not validate inputs.
Yes!
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58235
-----------------------------------------------------------
On Oct. 23, 2014, 9:35 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 9:35 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Qian Xu <sx...@googlemail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/#review58235
-----------------------------------------------------------
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
<https://reviews.apache.org/r/27110/#comment99188>
Please change to `@ConfigClass(validator = {@Validator(LinkConfig.ConfigValidator.class)})`, otherwise it will not validate inputs.
- Qian Xu
On Oct. 24, 2014, 5:35 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27110/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 5:35 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1620
> https://issues.apache.org/jira/browse/SQOOP-1620
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
> Author: Abraham Elmahrek <ab...@elmahrek.com>
> Date: Fri Oct 10 19:02:27 2014 -0700
>
> SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
>
> :100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
> :100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
> :100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
> :100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
> :100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
> :100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
> :000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
> :000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
> :100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
>
> Diff: https://reviews.apache.org/r/27110/diff/
>
>
> Testing
> -------
>
> mvn test and manually tested working order.
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 27110: SQOOP-1620: Sqoop2: FileSystem should be
configurable in HDFS connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27110/
-----------------------------------------------------------
(Updated Oct. 23, 2014, 9:35 p.m.)
Review request for Sqoop.
Changes
-------
Ran "mvn clean verify" and tested with file:///
Bugs: SQOOP-1620
https://issues.apache.org/jira/browse/SQOOP-1620
Repository: sqoop-sqoop2
Description
-------
commit ef7c1e58b115fec6c6d5a84345ca1e3c0cc93164
Author: Abraham Elmahrek <ab...@elmahrek.com>
Date: Fri Oct 10 19:02:27 2014 -0700
SQOOP-1620: Sqoop2: FileSystem should be configurable in HDFS connector
:100644 100644 cce0e29... 7ad66f8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java
:100644 100644 31b0a99... 0a2c98b... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 c7d35f7... 6d79db7... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java
:100644 100644 0752510... 4c6f566... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java
:100644 100644 682349c... 0753f9d... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 daa7fe2... 12bcd53... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java
:100644 100644 8bfd727... 3c85be8... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java
:100644 100644 e3d54b8... bce72b5... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java
:000000 100644 0000000... 0d77427... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:000000 100644 0000000... 54768ea... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java
:000000 100644 0000000... 29063a8... A connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java
:100644 100644 90bc8bc... 3d088d0... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 124c3df... 0a6369f... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:000000 100644 0000000... 846b565... A connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b1b3b16... f88424b... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 36f7443... b89424f... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsConnector.java cce0e29
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 31b0a99
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromDestroyer.java c7d35f7
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsFromInitializer.java 0752510
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 682349c
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsPartitioner.java daa7fe2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToDestroyer.java 8bfd727
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsToInitializer.java e3d54b8
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfig.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/LinkConfiguration.java PRE-CREATION
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 90bc8bc
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 124c3df
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java PRE-CREATION
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java 8429e15
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestPartitioner.java bef1984
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java b1b3b16
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java 36f7443
Diff: https://reviews.apache.org/r/27110/diff/
Testing
-------
mvn test and manually tested working order.
Thanks,
Abraham Elmahrek