You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Abraham Fine <ab...@abrahamfine.com> on 2016/02/03 07:33:01 UTC
Re: Review Request 42327: SQOOP-2788: Parquet support for
HdfsConnector
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42327/
-----------------------------------------------------------
(Updated Feb. 3, 2016, 6:32 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2788
https://issues.apache.org/jira/browse/SQOOP-2788
Repository: sqoop-sqoop2
Description
-------
read and write parquet in hdfsconnector
Diffs (updated)
-----
common/src/main/resources/org.apache.sqoop.connector-classloader.properties c0082cc05a164a13fe612658df4e4b07f3f49bd2
connector/connector-hdfs/pom.xml 599631418ca63cc43d645c1ee1e7a73dc824b313
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 9ef2a051a9a137d17af5c584c975f58bf8b8639d
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 5de20c620c3a5d244ec3394e772c0a87c2e7688d
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToFormat.java 27d121f529ecb4d5bd79e2b8c74ab8f7cc15fb10
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/GenericHdfsWriter.java 2ccccc4a94a582c8b47ccdefa523d1fd1632e627
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsParquetWriter.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsSequenceWriter.java 75c2e7ef192d7d9628e622cc3c5ef176e33a73d0
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsTextWriter.java 78cf9732fdb89689b04d43e4af70ca5a43732dbf
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java adede3a7e4ed16c059842c510066968c0ed739d4
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopAvroUtils.java 985149cbb0d28b55a19d17076d996364d7f2ae90
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java ace1bdf5dd20de1fb2abbe6723499373f66fd5a0
pom.xml cb8a973abc96af1de905cebd80d30177cbaf1cb4
test/pom.xml 644a9c7dbc746d4a3268532bdcf0babd4faaafba
test/src/test/java/org/apache/sqoop/integration/connector/hdfs/NullValueTest.java 4363814368e56150519f19fd9d682ce85cad914f
test/src/test/java/org/apache/sqoop/integration/connector/hdfs/ParquetTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/42327/diff/
Testing
-------
integration tests pass
Thanks,
Abraham Fine
Re: Review Request 42327: SQOOP-2788: Parquet support for
HdfsConnector
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42327/#review118054
-----------------------------------------------------------
Few nits/questions:
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java (line 112)
<https://reviews.apache.org/r/42327/#comment179324>
Are we removing the locations argument only because it's not used right now?
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToFormat.java (line 37)
<https://reviews.apache.org/r/42327/#comment179323>
Super nit: Please put comma at the end - when adding another format, we won't have to change this line which will make git blame a bit easier to follow.
Jarcec
- Jarek Cecho
On Feb. 3, 2016, 6:32 a.m., Abraham Fine wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42327/
> -----------------------------------------------------------
>
> (Updated Feb. 3, 2016, 6:32 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2788
> https://issues.apache.org/jira/browse/SQOOP-2788
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> read and write parquet in hdfsconnector
>
>
> Diffs
> -----
>
> common/src/main/resources/org.apache.sqoop.connector-classloader.properties c0082cc05a164a13fe612658df4e4b07f3f49bd2
> connector/connector-hdfs/pom.xml 599631418ca63cc43d645c1ee1e7a73dc824b313
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 9ef2a051a9a137d17af5c584c975f58bf8b8639d
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 5de20c620c3a5d244ec3394e772c0a87c2e7688d
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToFormat.java 27d121f529ecb4d5bd79e2b8c74ab8f7cc15fb10
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/GenericHdfsWriter.java 2ccccc4a94a582c8b47ccdefa523d1fd1632e627
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsParquetWriter.java PRE-CREATION
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsSequenceWriter.java 75c2e7ef192d7d9628e622cc3c5ef176e33a73d0
> connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsTextWriter.java 78cf9732fdb89689b04d43e4af70ca5a43732dbf
> connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java adede3a7e4ed16c059842c510066968c0ed739d4
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopAvroUtils.java 985149cbb0d28b55a19d17076d996364d7f2ae90
> connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java ace1bdf5dd20de1fb2abbe6723499373f66fd5a0
> pom.xml cb8a973abc96af1de905cebd80d30177cbaf1cb4
> test/pom.xml 644a9c7dbc746d4a3268532bdcf0babd4faaafba
> test/src/test/java/org/apache/sqoop/integration/connector/hdfs/NullValueTest.java 4363814368e56150519f19fd9d682ce85cad914f
> test/src/test/java/org/apache/sqoop/integration/connector/hdfs/ParquetTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42327/diff/
>
>
> Testing
> -------
>
> integration tests pass
>
>
> Thanks,
>
> Abraham Fine
>
>
Re: Review Request 42327: SQOOP-2788: Parquet support for
HdfsConnector
Posted by Abraham Fine <ab...@abrahamfine.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42327/
-----------------------------------------------------------
(Updated Feb. 5, 2016, 7:09 p.m.)
Review request for Sqoop.
Bugs: SQOOP-2788
https://issues.apache.org/jira/browse/SQOOP-2788
Repository: sqoop-sqoop2
Description
-------
read and write parquet in hdfsconnector
Diffs (updated)
-----
common/src/main/resources/org.apache.sqoop.connector-classloader.properties c0082cc05a164a13fe612658df4e4b07f3f49bd2
connector/connector-hdfs/pom.xml 599631418ca63cc43d645c1ee1e7a73dc824b313
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 9ef2a051a9a137d17af5c584c975f58bf8b8639d
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 5de20c620c3a5d244ec3394e772c0a87c2e7688d
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToFormat.java 27d121f529ecb4d5bd79e2b8c74ab8f7cc15fb10
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/GenericHdfsWriter.java 2ccccc4a94a582c8b47ccdefa523d1fd1632e627
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsParquetWriter.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsSequenceWriter.java 75c2e7ef192d7d9628e622cc3c5ef176e33a73d0
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsTextWriter.java 78cf9732fdb89689b04d43e4af70ca5a43732dbf
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java adede3a7e4ed16c059842c510066968c0ed739d4
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopAvroUtils.java 985149cbb0d28b55a19d17076d996364d7f2ae90
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java ace1bdf5dd20de1fb2abbe6723499373f66fd5a0
pom.xml cb8a973abc96af1de905cebd80d30177cbaf1cb4
test/pom.xml 644a9c7dbc746d4a3268532bdcf0babd4faaafba
test/src/test/java/org/apache/sqoop/integration/connector/hdfs/NullValueTest.java 3ec4f661577d3a945d082491b2add32abb9d7120
test/src/test/java/org/apache/sqoop/integration/connector/hdfs/ParquetTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/42327/diff/
Testing
-------
integration tests pass
Thanks,
Abraham Fine
Re: Review Request 42327: SQOOP-2788: Parquet support for
HdfsConnector
Posted by Abraham Fine <ab...@abrahamfine.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42327/
-----------------------------------------------------------
(Updated Feb. 5, 2016, 6:59 p.m.)
Review request for Sqoop.
Bugs: SQOOP-2788
https://issues.apache.org/jira/browse/SQOOP-2788
Repository: sqoop-sqoop2
Description
-------
read and write parquet in hdfsconnector
Diffs (updated)
-----
common/src/main/resources/org.apache.sqoop.connector-classloader.properties c0082cc05a164a13fe612658df4e4b07f3f49bd2
connector/connector-hdfs/pom.xml 599631418ca63cc43d645c1ee1e7a73dc824b313
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsExtractor.java 9ef2a051a9a137d17af5c584c975f58bf8b8639d
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/HdfsLoader.java 5de20c620c3a5d244ec3394e772c0a87c2e7688d
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/configuration/ToFormat.java 27d121f529ecb4d5bd79e2b8c74ab8f7cc15fb10
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/GenericHdfsWriter.java 2ccccc4a94a582c8b47ccdefa523d1fd1632e627
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsParquetWriter.java PRE-CREATION
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsSequenceWriter.java 75c2e7ef192d7d9628e622cc3c5ef176e33a73d0
connector/connector-hdfs/src/main/java/org/apache/sqoop/connector/hdfs/hdfsWriter/HdfsTextWriter.java 78cf9732fdb89689b04d43e4af70ca5a43732dbf
connector/connector-hdfs/src/test/java/org/apache/sqoop/connector/hdfs/TestLoader.java adede3a7e4ed16c059842c510066968c0ed739d4
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/common/SqoopAvroUtils.java 985149cbb0d28b55a19d17076d996364d7f2ae90
connector/connector-sdk/src/main/java/org/apache/sqoop/connector/idf/AVROIntermediateDataFormat.java ace1bdf5dd20de1fb2abbe6723499373f66fd5a0
pom.xml cb8a973abc96af1de905cebd80d30177cbaf1cb4
test/pom.xml 644a9c7dbc746d4a3268532bdcf0babd4faaafba
test/src/test/java/org/apache/sqoop/integration/connector/hdfs/NullValueTest.java 4363814368e56150519f19fd9d682ce85cad914f
test/src/test/java/org/apache/sqoop/integration/connector/hdfs/ParquetTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/42327/diff/
Testing
-------
integration tests pass
Thanks,
Abraham Fine