You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Abraham Elmahrek <ab...@cloudera.com> on 2015/02/04 02:17:47 UTC
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/
-----------------------------------------------------------
(Updated Feb. 4, 2015, 1:17 a.m.)
Review request for Sqoop.
Changes
-------
Rebase
Bugs: SQOOP-1579
https://issues.apache.org/jira/browse/SQOOP-1579
Repository: sqoop-sqoop2
Description
-------
A question for the reviewer:
# Does the option "unescape" with default to "true" make sense? It did for me, but I can imagine from the users perspective that it might be confusing.
commit 74274f1d745215035eb766645b9e8dbb4c591139
Author: Abraham Elmahrek <ab...@apache.org>
Date: Mon Jan 26 19:21:15 2015 -0800
SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
:100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
:100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
:100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
:100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
:100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
:100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
:100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
:100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
:000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
:100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
:100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
:100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java fef8202
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java 0fac41a
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java f658d09
test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 1124cd3
test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java a65d862
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 0b530b9
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
Diff: https://reviews.apache.org/r/30303/diff/
Testing
-------
mvn clean integration-test
Thanks,
Abraham Elmahrek
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Feb. 21, 2015, 4:30 p.m., Jarek Cecho wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java, lines 53-66
> > <https://reviews.apache.org/r/30303/diff/6/?file=858209#file858209line53>
> >
> > At this point I would recommend keeping it as it is, but If I would be refactoring it, then I would create "form" (the old word) called "Format" (or something similar) and simply re-used that in both "From" and "To" parts. It seems simpler and more reusable then copy&pasting the same parameters around. Perhaps a subsequent JIRA?
Not sure I follow. I don't really mind refactoring this. Could you create a jira with more description?
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review73329
-----------------------------------------------------------
On Feb. 9, 2015, 6:13 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 6:13 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review73329
-----------------------------------------------------------
Ship it!
I believe that Stanley had some nit comment, I do have another one and then just a future recommendation. Otherwise I'm +1 :) Good job Abe!
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
<https://reviews.apache.org/r/30303/#comment119745>
At this point I would recommend keeping it as it is, but If I would be refactoring it, then I would create "form" (the old word) called "Format" (or something similar) and simply re-used that in both "From" and "To" parts. It seems simpler and more reusable then copy&pasting the same parameters around. Perhaps a subsequent JIRA?
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
<https://reviews.apache.org/r/30303/#comment119630>
Let's put the dropTable() into @After method rather then putting entire function into one long try-finally block?
- Jarek Cecho
On Feb. 9, 2015, 6:13 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 6:13 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Veena Basavaraj <vb...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review71631
-----------------------------------------------------------
Ship it!
I still hope we move the fromCSV and its counter part to CSVUtils, we have AvroUtils for the same reason, that handles the entire avro object.
- Veena Basavaraj
On Feb. 8, 2015, 10:13 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 8, 2015, 10:13 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Qian Xu <qi...@intel.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review72519
-----------------------------------------------------------
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
<https://reviews.apache.org/r/30303/#comment118579>
NIT:
return Boolean.TRUE == fromJobConfiguration.fromJobConfig.overrideNullValue;
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
<https://reviews.apache.org/r/30303/#comment118580>
NIT:
return Boolean.TRUE == toJobConfiguration.toJobConfig.overrideNullValue
- Qian Xu
On Feb. 9, 2015, 2:13 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 2:13 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Qian Xu <qi...@intel.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review74194
-----------------------------------------------------------
Ship it!
Ship It!
- Qian Xu
On Feb. 26, 2015, 3:18 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 26, 2015, 3:18 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 6754536
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 310eadc
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/
-----------------------------------------------------------
(Updated Feb. 25, 2015, 7:18 p.m.)
Review request for Sqoop.
Bugs: SQOOP-1579
https://issues.apache.org/jira/browse/SQOOP-1579
Repository: sqoop-sqoop2
Description
-------
commit 74274f1d745215035eb766645b9e8dbb4c591139
Author: Abraham Elmahrek <ab...@apache.org>
Date: Mon Jan 26 19:21:15 2015 -0800
SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
:100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
:100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
:100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
:100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
:100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
:100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
:100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
:100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
:000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
:100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
:100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
:100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 6754536
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 310eadc
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
Diff: https://reviews.apache.org/r/30303/diff/
Testing
-------
mvn clean integration-test
Thanks,
Abraham Elmahrek
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/
-----------------------------------------------------------
(Updated Feb. 9, 2015, 6:13 a.m.)
Review request for Sqoop.
Bugs: SQOOP-1579
https://issues.apache.org/jira/browse/SQOOP-1579
Repository: sqoop-sqoop2
Description
-------
commit 74274f1d745215035eb766645b9e8dbb4c591139
Author: Abraham Elmahrek <ab...@apache.org>
Date: Mon Jan 26 19:21:15 2015 -0800
SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
:100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
:100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
:100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
:100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
:100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
:100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
:100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
:100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
:000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
:100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
:100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
:100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
Diff: https://reviews.apache.org/r/30303/diff/
Testing
-------
mvn clean integration-test
Thanks,
Abraham Elmahrek
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/
-----------------------------------------------------------
(Updated Feb. 8, 2015, 7:48 p.m.)
Review request for Sqoop.
Changes
-------
Fix tests
Bugs: SQOOP-1579
https://issues.apache.org/jira/browse/SQOOP-1579
Repository: sqoop-sqoop2
Description (updated)
-------
commit 74274f1d745215035eb766645b9e8dbb4c591139
Author: Abraham Elmahrek <ab...@apache.org>
Date: Mon Jan 26 19:21:15 2015 -0800
SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
:100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
:100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
:100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
:100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
:100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
:100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
:100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
:100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
:000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
:100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
:100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
:100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
Diff: https://reviews.apache.org/r/30303/diff/
Testing
-------
mvn clean integration-test
Thanks,
Abraham Elmahrek
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Veena Basavaraj <vb...@cloudera.com>.
> On Feb. 7, 2015, 7:30 a.m., Veena Basavaraj wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java, line 119
> > <https://reviews.apache.org/r/30303/diff/4/?file=853438#file853438line119>
> >
> > I would prefer we keep these methods in CSVIDFUtils.
> >
> > SqoopIDFUtils was named so, because it handles the IDF column types.
> >
> > If we are extracting away the functionality of CSVIDF to be used in other connectors. then it is best to move thsoe methods into its own utils so connectors can use it and have correponding tests for it. there.
>
> Abraham Elmahrek wrote:
> Ultimately it would be nice to have our own IDF. Let's consider this a temporary step in the right direction and create an IDF after the release?
we have Sqoop AvroUtils for the avro stuff, so moving these 2 methods out to CSVUTils is better, dont think it is much effort to add a new class
> On Feb. 7, 2015, 7:30 a.m., Veena Basavaraj wrote:
> > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java, line 565
> > <https://reviews.apache.org/r/30303/diff/4/?file=853446#file853446line565>
> >
> > As comments above, lets not bloat the SQoopIDFUtils, please move the CSVIDF methods to its own class, since it is doing more than just the common column types we support
>
> Abraham Elmahrek wrote:
> A couple of thoughts on this:
> 1. I think it belongs in SqoopIDFUtils because most IDFs will need to go back and forth between CSV and Object form. CSV needs to be implemented by all IDFs.
> 2. If we want to change this, let's move it out in a different Jira. ``parseCSVString`` was originally part of SqoopIDFUtils any ways. https://github.com/apache/sqoop/blob/f7efa38005525e2fb22ce1af0323bf265d719c00/connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java#L518
I meant th fromCSV and fromObject
- Veena
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review71553
-----------------------------------------------------------
On Feb. 8, 2015, 10:13 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 8, 2015, 10:13 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Abraham Elmahrek <ab...@cloudera.com>.
> On Feb. 7, 2015, 3:30 p.m., Veena Basavaraj wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java, line 119
> > <https://reviews.apache.org/r/30303/diff/4/?file=853438#file853438line119>
> >
> > I would prefer we keep these methods in CSVIDFUtils.
> >
> > SqoopIDFUtils was named so, because it handles the IDF column types.
> >
> > If we are extracting away the functionality of CSVIDF to be used in other connectors. then it is best to move thsoe methods into its own utils so connectors can use it and have correponding tests for it. there.
Ultimately it would be nice to have our own IDF. Let's consider this a temporary step in the right direction and create an IDF after the release?
> On Feb. 7, 2015, 3:30 p.m., Veena Basavaraj wrote:
> > connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java, line 188
> > <https://reviews.apache.org/r/30303/diff/4/?file=853438#file853438line188>
> >
> > this seems such a overkill, whoveever intends to use the object format.
> >
> > We convert it back and forth between csv to OBJECT and then object to CSV and then the same dance hapens on the TO side.
> >
> > Is this going to be a common use case anyone will use? Just wondnering.
> >
> > The whole point of using text format in a connector and then use CSVIDF was to make sure we have very little conversions going on, but this use case seems to just completely do the opposite.
> >
> > Now if I imagine supporting a avro/ parquet in this conenctor in future, I wonder what format it will use. food for thought.
HDFS connector should get its own IDF in the future.
> On Feb. 7, 2015, 3:30 p.m., Veena Basavaraj wrote:
> > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java, line 565
> > <https://reviews.apache.org/r/30303/diff/4/?file=853446#file853446line565>
> >
> > As comments above, lets not bloat the SQoopIDFUtils, please move the CSVIDF methods to its own class, since it is doing more than just the common column types we support
A couple of thoughts on this:
1. I think it belongs in SqoopIDFUtils because most IDFs will need to go back and forth between CSV and Object form. CSV needs to be implemented by all IDFs.
2. If we want to change this, let's move it out in a different Jira. ``parseCSVString`` was originally part of SqoopIDFUtils any ways. https://github.com/apache/sqoop/blob/f7efa38005525e2fb22ce1af0323bf265d719c00/connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java#L518
> On Feb. 7, 2015, 3:30 p.m., Veena Basavaraj wrote:
> > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java, line 688
> > <https://reviews.apache.org/r/30303/diff/4/?file=853446#file853446line688>
> >
> > why do we need to move this to IDFUtils, if this is speciifc to CSV, then its better to create a SQOOPCSVUtils and not bloat the IDF Utils. IDF Utils has methods to handle each column type and hence it was named so.
See above comment.
> On Feb. 7, 2015, 3:30 p.m., Veena Basavaraj wrote:
> > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java, line 445
> > <https://reviews.apache.org/r/30303/diff/4/?file=853446#file853446line445>
> >
> > there is some new lines added >>>> that shows up in RB
I think those are because review board has trouble discerning between method blocks and code blocks?
- Abraham
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review71553
-----------------------------------------------------------
On Feb. 7, 2015, 2:23 a.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 7, 2015, 2:23 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> A question for the reviewer:
> # Does the option "unescape" with default to "true" make sense? It did for me, but I can imagine from the users perspective that it might be confusing.
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Veena Basavaraj <vb...@cloudera.com>.
> On Feb. 7, 2015, 7:30 a.m., Veena Basavaraj wrote:
> > connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java, line 688
> > <https://reviews.apache.org/r/30303/diff/4/?file=853446#file853446line688>
> >
> > why do we need to move this to IDFUtils, if this is speciifc to CSV, then its better to create a SQOOPCSVUtils and not bloat the IDF Utils. IDF Utils has methods to handle each column type and hence it was named so.
>
> Abraham Elmahrek wrote:
> See above comment.
I missed this point on this, it is used in the hdfs connector, so dropping the comment
- Veena
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review71553
-----------------------------------------------------------
On Feb. 8, 2015, 10:13 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 8, 2015, 10:13 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Veena Basavaraj <vb...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review71553
-----------------------------------------------------------
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
<https://reviews.apache.org/r/30303/#comment117309>
I would prefer we keep these methods in CSVIDFUtils.
SqoopIDFUtils was named so, because it handles the IDF column types.
If we are extracting away the functionality of CSVIDF to be used in other connectors. then it is best to move thsoe methods into its own utils so connectors can use it and have correponding tests for it. there.
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
<https://reviews.apache.org/r/30303/#comment117308>
this seems such a overkill, whoveever intends to use the object format.
We convert it back and forth between csv to OBJECT and then object to CSV and then the same dance hapens on the TO side.
Is this going to be a common use case anyone will use? Just wondnering.
The whole point of using text format in a connector and then use CSVIDF was to make sure we have very little conversions going on, but this use case seems to just completely do the opposite.
Now if I imagine supporting a avro/ parquet in this conenctor in future, I wonder what format it will use. food for thought.
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
<https://reviews.apache.org/r/30303/#comment117305>
there is some new lines added >>>> that shows up in RB
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
<https://reviews.apache.org/r/30303/#comment117310>
As comments above, lets not bloat the SQoopIDFUtils, please move the CSVIDF methods to its own class, since it is doing more than just the common column types we support
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
<https://reviews.apache.org/r/30303/#comment117306>
why do we need to move this to IDFUtils, if this is speciifc to CSV, then its better to create a SQOOPCSVUtils and not bloat the IDF Utils. IDF Utils has methods to handle each column type and hence it was named so.
- Veena Basavaraj
On Feb. 6, 2015, 6:23 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 6, 2015, 6:23 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> A question for the reviewer:
> # Does the option "unescape" with default to "true" make sense? It did for me, but I can imagine from the users perspective that it might be confusing.
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Abraham Elmahrek <ab...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/
-----------------------------------------------------------
(Updated Feb. 7, 2015, 2:23 a.m.)
Review request for Sqoop.
Bugs: SQOOP-1579
https://issues.apache.org/jira/browse/SQOOP-1579
Repository: sqoop-sqoop2
Description
-------
A question for the reviewer:
# Does the option "unescape" with default to "true" make sense? It did for me, but I can imagine from the users perspective that it might be confusing.
commit 74274f1d745215035eb766645b9e8dbb4c591139
Author: Abraham Elmahrek <ab...@apache.org>
Date: Mon Jan 26 19:21:15 2015 -0800
SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
:100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
:100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
:100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
:100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
:100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
:100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
:100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
:100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
:100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
:100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
:100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
:100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
:100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
:100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
:000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
:100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
:100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
:100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
:100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
Diffs (updated)
-----
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 398a051
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 397ce6f
test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
Diff: https://reviews.apache.org/r/30303/diff/
Testing
-------
mvn clean integration-test
Thanks,
Abraham Elmahrek
Re: Review Request 30303: SQOOP-1579: Sqoop2: Data transfer to load
into Hive does not work
Posted by Veena Basavaraj <vb...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30303/#review70934
-----------------------------------------------------------
I have some basic question in the JIRA, I do not understand in depth the proposed changes in the RB, so lets have a chat offline or in JIRA about this.
- Veena Basavaraj
On Feb. 3, 2015, 5:17 p.m., Abraham Elmahrek wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30303/
> -----------------------------------------------------------
>
> (Updated Feb. 3, 2015, 5:17 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-1579
> https://issues.apache.org/jira/browse/SQOOP-1579
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> A question for the reviewer:
> # Does the option "unescape" with default to "true" make sense? It did for me, but I can imagine from the users perspective that it might be confusing.
>
> commit 74274f1d745215035eb766645b9e8dbb4c591139
> Author: Abraham Elmahrek <ab...@apache.org>
> Date: Mon Jan 26 19:21:15 2015 -0800
>
> SQOOP-1579: Sqoop2: Data transfer to load into Hive does not work
>
> :100644 100644 9f3367b... 6b8ae52... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java
> :100644 100644 556c112... 9dba157... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java
> :100644 100644 353c1f2... 941c378... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java
> :100644 100644 b7a9c3d... aa5f316... M connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java
> :100644 100644 2dca634... 3efc76a... M connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties
> :100644 100644 9a5ec5e... c3b2975... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java
> :100644 100644 8eefe05... c8d404b... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java
> :100644 100644 b5ec6da... 62d5eb8... M connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java
> :100644 100644 c5eb6a3... c2dd812... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java
> :100644 100644 3350599... c92bf61... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java
> :100644 100644 b716e79... 71d9534... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java
> :100644 100644 4c67c1a... a8d186d... M connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java
> :100644 100644 588ce29... 1166496... M connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java
> :100644 100644 f658d09... a5be42b... M execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java
> :000000 100644 0000000... 17148bc... A test/src/main/java/org/apache/sqoop/test/data/ShortStories.java
> :100644 100644 1124cd3... 12bf99c... M test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java
> :100644 100644 a65d862... f6f7188... M test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java
> :100644 100644 f82abc7... a81b349... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java
> :100644 100644 aa9f212... bb91a83... M test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java
>
>
> Diffs
> -----
>
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java f70d56b
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java bdee878
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsUtils.java 353c1f2
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToJobConfig.java b7a9c3d
> connector/connector-hdfs/src/main/resources/hdfs-connector-config.properties 2dca634
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestExtractor.java 9a5ec5e
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestHdfsUtils.java 8eefe05
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java b5ec6da
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopIDFUtils.java 71641ed
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java fef8202
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/CSVIntermediateDataFormat.java c233fb2
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/JSONIntermediateDataFormat.java 0fac41a
> connector/connector-sdk/src/test/java/org/apache/sqoop/connector/common/TestSqoopIDFUtils.java 588ce29
> execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMatching.java f658d09
> test/src/main/java/org/apache/sqoop/test/data/ShortStories.java PRE-CREATION
> test/src/main/java/org/apache/sqoop/test/testcases/ConnectorTestCase.java 1124cd3
> test/src/main/java/org/apache/sqoop/test/testcases/KafkaConnectorTestCase.java a65d862
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromHDFSToRDBMSTest.java 0b530b9
> test/src/test/java/org/apache/sqoop/integration/connector/jdbc/generic/FromRDBMSToHDFSTest.java ced52cc
>
> Diff: https://reviews.apache.org/r/30303/diff/
>
>
> Testing
> -------
>
> mvn clean integration-test
>
>
> Thanks,
>
> Abraham Elmahrek
>
>