You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Sandish Kumar HN <sa...@gmail.com> on 2017/08/09 10:46:35 UTC
Review Request 61522: SQOOP-2907 : Export parquet files to RDBMS:
don't require .metadata for parquet files
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61522/
-----------------------------------------------------------
Review request for Sqoop, AnnaTW AnnaTW and Anna Szonyi.
Bugs: SQOOP-2907
https://issues.apache.org/jira/browse/SQOOP-2907
Repository: sqoop-trunk
Description
-------
Kite currently requires .metadata.
Parquet files have their own metadata stored along data files.
It would be great for Export operation on parquet files to RDBMS not to require .metadata.
We have most of the files created by Spark and Hive, and they don't create .metadata, it only Kite that does.
It makes sqoop export of parquet files usability very limited.
Diffs
-----
src/java/org/apache/sqoop/avro/AvroUtil.java ee29f140
src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 6f9afaf9
src/test/com/cloudera/sqoop/TestParquetExport.java 680fd73b
Diff: https://reviews.apache.org/r/61522/diff/1/
Testing
-------
Thanks,
Sandish Kumar HN
Re: Review Request 61522: SQOOP-2907 : Export parquet files to RDBMS:
don't require .metadata for parquet files
Posted by Sandish Kumar HN <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61522/#review182476
-----------------------------------------------------------
Ship it!
tested
- Sandish Kumar HN
On Aug. 9, 2017, 10:46 a.m., Sandish Kumar HN wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61522/
> -----------------------------------------------------------
>
> (Updated Aug. 9, 2017, 10:46 a.m.)
>
>
> Review request for Sqoop, AnnaTW AnnaTW and Anna Szonyi.
>
>
> Bugs: SQOOP-2907
> https://issues.apache.org/jira/browse/SQOOP-2907
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> Kite currently requires .metadata.
> Parquet files have their own metadata stored along data files.
> It would be great for Export operation on parquet files to RDBMS not to require .metadata.
> We have most of the files created by Spark and Hive, and they don't create .metadata, it only Kite that does.
> It makes sqoop export of parquet files usability very limited.
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/avro/AvroUtil.java ee29f140
> src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 6f9afaf9
> src/test/com/cloudera/sqoop/TestParquetExport.java 680fd73b
>
>
> Diff: https://reviews.apache.org/r/61522/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Sandish Kumar HN
>
>
Re: Review Request 61522: SQOOP-2907 : Export parquet files to RDBMS:
don't require .metadata for parquet files
Posted by Sandish Kumar HN <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61522/
-----------------------------------------------------------
(Updated Aug. 9, 2017, 10:50 a.m.)
Review request for Sqoop, AnnaTW AnnaTW and Anna Szonyi.
Bugs: SQOOP-2907
https://issues.apache.org/jira/browse/SQOOP-2907
Repository: sqoop-trunk
Description
-------
Kite currently requires .metadata.
Parquet files have their own metadata stored along data files.
It would be great for Export operation on parquet files to RDBMS not to require .metadata.
We have most of the files created by Spark and Hive, and they don't create .metadata, it only Kite that does.
It makes sqoop export of parquet files usability very limited.
Diffs
-----
src/java/org/apache/sqoop/avro/AvroUtil.java ee29f140
src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 6f9afaf9
src/test/com/cloudera/sqoop/TestParquetExport.java 680fd73b
Diff: https://reviews.apache.org/r/61522/diff/1/
Testing (updated)
-------
testSupportedParquetTypesForWithoutParquetMeta - done
testNullableFieldWithoutParquetMeta - done
testParquetRecordsNotSupportedWithoutParquetMeta -done
testMissingDatabaseFieldsWithoutParquetMeta - done
testMissingParquetFieldsWithoutParquetMeta - done
Thanks,
Sandish Kumar HN
Re: Review Request 61522: SQOOP-2907 : Export parquet files to RDBMS:
don't require .metadata for parquet files
Posted by Sandish Kumar HN <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61522/
-----------------------------------------------------------
(Updated Aug. 9, 2017, 10:48 a.m.)
Review request for Sqoop, AnnaTW AnnaTW and Anna Szonyi.
Bugs: SQOOP-2907
https://issues.apache.org/jira/browse/SQOOP-2907
Repository: sqoop-trunk
Description
-------
Kite currently requires .metadata.
Parquet files have their own metadata stored along data files.
It would be great for Export operation on parquet files to RDBMS not to require .metadata.
We have most of the files created by Spark and Hive, and they don't create .metadata, it only Kite that does.
It makes sqoop export of parquet files usability very limited.
Diffs
-----
src/java/org/apache/sqoop/avro/AvroUtil.java ee29f140
src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 6f9afaf9
src/test/com/cloudera/sqoop/TestParquetExport.java 680fd73b
Diff: https://reviews.apache.org/r/61522/diff/1/
Testing
-------
Thanks,
Sandish Kumar HN