You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Ying Cao <an...@gmail.com> on 2017/05/08 07:35:17 UTC

Re: Review Request 58477: support DB2 xml data type when sqoop import with parquet

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58477/
-----------------------------------------------------------

(Updated 五月 8, 2017, 7:35 a.m.)


Review request for Sqoop, Boglarka Egyed, Attila Szabo, and Szabolcs Vasas.


Bugs: SQOOP-3173
    https://issues.apache.org/jira/browse/SQOOP-3173


Repository: sqoop-trunk


Description
-------

SQOOP-3173 : support DB2 xml data type when sqoop import with parquet


Diffs
-----

  src/java/org/apache/sqoop/manager/ConnManager.java 1811ce0 
  src/test/org/apache/sqoop/manager/db2/DB2XmlTypeParquetImportManualTest.java PRE-CREATION 


Diff: https://reviews.apache.org/r/58477/diff/3/


Testing (updated)
-------

UT passed by manually

This is third-part test, I ran it by :

ant clean test -Dthirdparty=true -Dsqoop.thirdparty.lib.dir=3rdparty -Dsqoop.test.db2.connectstring.host_url=connectstring -Dsqoop.test.db2.connectstring.database=databasename -Dsqoop.test.db2.connectstring.username=username -Dsqoop.test.db2.connectstring.password=password -Dtestcase=DB2XmlTypeParquetImportManualTest


Thanks,

Ying Cao

Re: Review Request 58477: support DB2 xml data type when sqoop import with parquet

Posted by Attila Szabo <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58477/#review175237
-----------------------------------------------------------




src/java/org/apache/sqoop/manager/ConnManager.java
Line 211 (original), 211-212 (patched)
<https://reviews.apache.org/r/58477/#comment248691>

    Hey Angela,
    
    I'm a bit concerned here.
    According to the specs of java.sql.Types.OTHER
    
    """The constant in the Java programming language that indicates that the SQL type is database-specific and gets mapped to a Java object that can be accessed via the methods getObject and setObject."""
    
    thus I'm a bit unsure, if it is the best thing to do representing it directly as a String object/entity.
    
    Especially your change would like to fix something DB2 related, although your changes affecting ConnManager, which is quite a central part of the whole Sqoop system (and this affecting all the RDBMS implementations more or less).
    
    I have two requests for you:
    A. could you please double check this with someone from the Avro/Parquet team that every XML/Unicode/UTF-8 character could be stored in the Type.STRING "column"?  One connection point could be Gabor Szadovszky (https://github.com/gszadovszky). AFAIK he's been quite active in the Avro/Parquet community recently, and working in the same hours as I do.
    B. could you please move this change to the DB2Manager.java (IMHO this change would be better suited there, as right now the task is only about to handle XML types for Parquet).
    
    Thanks in advance!


- Attila Szabo


On May 8, 2017, 7:35 a.m., Ying Cao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58477/
> -----------------------------------------------------------
> 
> (Updated May 8, 2017, 7:35 a.m.)
> 
> 
> Review request for Sqoop, Boglarka Egyed, Attila Szabo, and Szabolcs Vasas.
> 
> 
> Bugs: SQOOP-3173
>     https://issues.apache.org/jira/browse/SQOOP-3173
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> SQOOP-3173 : support DB2 xml data type when sqoop import with parquet
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/manager/ConnManager.java 1811ce0 
>   src/test/org/apache/sqoop/manager/db2/DB2XmlTypeParquetImportManualTest.java PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/58477/diff/3/
> 
> 
> Testing
> -------
> 
> UT passed by manually
> 
> This is third-part test, I ran it by :
> 
> ant clean test -Dthirdparty=true -Dsqoop.thirdparty.lib.dir=3rdparty -Dsqoop.test.db2.connectstring.host_url=connectstring -Dsqoop.test.db2.connectstring.database=databasename -Dsqoop.test.db2.connectstring.username=username -Dsqoop.test.db2.connectstring.password=password -Dtestcase=DB2XmlTypeParquetImportManualTest
> 
> 
> Thanks,
> 
> Ying Cao
> 
>