You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Abraham Elmahrek <ab...@cloudera.com> on 2015/07/07 02:21:04 UTC
Re: Review Request 34104: Fix for SQOOP-1369 Avro export ignores
--columns option
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34104/#review90611
-----------------------------------------------------------
Ship it!
LGTM, just a couple of other things.
src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java (line 26)
<https://reviews.apache.org/r/34104/#comment143724>
Unnecessary
src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java (line 104)
<https://reviews.apache.org/r/34104/#comment143725>
Why not set this to "null" in some cases instead? Then the below condition could be:
if (specifiedColumns == null || specifiedColumns.contains())...
- Abraham Elmahrek
On May 21, 2015, 7:52 p.m., Paul Mazak wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34104/
> -----------------------------------------------------------
>
> (Updated May 21, 2015, 7:52 p.m.)
>
>
> Review request for Sqoop and Yibing Shi.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> Supplied a patch that worked for us.
> There are 2 global places to consider - parent class SqlManager and child class CatalogQueryManager - when returning getColumnNames to actually rely on those column names you specified with --columns option.
> There was 1 place to consider for Avro/Parquet in JdbcExportJob.configureGenericRecordExportInputFormat() to only add the columnType if it's one that was specified with --columns option.
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/manager/CatalogQueryManager.java 4e063ed
> src/java/org/apache/sqoop/manager/OracleManager.java 69b613f
> src/java/org/apache/sqoop/manager/SqlManager.java ead581d
> src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 93d438a
> src/test/org/apache/sqoop/manager/SqlManagerTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34104/diff/
>
>
> Testing
> -------
>
> Integration testing passed exporting AVRO to MS SQL Server
>
>
> Thanks,
>
> Paul Mazak
>
>
Re: Review Request 34104: Fix for SQOOP-1369 Avro export ignores
--columns option
Posted by Paul Mazak <pm...@yahoo.com>.
> On July 7, 2015, 12:21 a.m., Abraham Elmahrek wrote:
> > src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java, line 104
> > <https://reviews.apache.org/r/34104/diff/2/?file=966944#file966944line104>
> >
> > Why not set this to "null" in some cases instead? Then the below condition could be:
> > if (specifiedColumns == null || specifiedColumns.contains())...
Fixed. Also made SqlManager.filterSpecifiedColumnNames not use .isEmpty() for consistency. Specified --columns option will have stuff in it or be null. Args parser does not allow empty.
Also, in order to test faster, added JdbcExportJobTest.
- Paul
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34104/#review90611
-----------------------------------------------------------
On July 7, 2015, 8:03 p.m., Paul Mazak wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34104/
> -----------------------------------------------------------
>
> (Updated July 7, 2015, 8:03 p.m.)
>
>
> Review request for Sqoop and Yibing Shi.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> Supplied a patch that worked for us.
> There are 2 global places to consider - parent class SqlManager and child class CatalogQueryManager - when returning getColumnNames to actually rely on those column names you specified with --columns option.
> There was 1 place to consider for Avro/Parquet in JdbcExportJob.configureGenericRecordExportInputFormat() to only add the columnType if it's one that was specified with --columns option.
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/manager/CatalogQueryManager.java 4e063ed
> src/java/org/apache/sqoop/manager/OracleManager.java 69b613f
> src/java/org/apache/sqoop/manager/SqlManager.java ead581d
> src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 93d438a
> src/test/org/apache/sqoop/manager/SqlManagerTest.java PRE-CREATION
> src/test/org/apache/sqoop/mapreduce/JdbcExportJobTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/34104/diff/
>
>
> Testing
> -------
>
> Integration testing passed exporting AVRO to MS SQL Server
>
>
> Thanks,
>
> Paul Mazak
>
>