Re: Review Request 12032: Incremental import using lastmodified mode always assumes column type to be timestamp, this patch fixes that

[Jarek Cecho <ja...@apache.org>]

> On June 26, 2013, 7 p.m., Jarek Cecho wrote:
> > Hi Raghav,
> > thank you very much for taking up this JIRA, appreciated! I'm wondering what is the reasoning for introducing a new query in the Oracle manager to retrive the check column type when the type has been already retrieved and seems to be stored in checkColumnType variable [1]? It seems to me that the real problem here is that the code in the ImportTool is simply overriding this type to Timestamp on [2] before passing it to the datetimeToQueryString [3]. Did you by any chance experiment with removing this line?
> > 
> > It would be also great if you could include a new test for this, so that we can be sure that we won't break this in the future again.
> > 
> > Jarcec
> > 
> > Links:
> > 1: https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/tool/ImportTool.java#L70
> > 2: https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/tool/ImportTool.java#L286
> > 3: https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/manager/OracleManager.java#L578
> 
> Raghav Gautam wrote:
>     Hi Jarek,
>     checkColumnType is a private variable which gets assigned in method getMaxColumnId.
>     https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/tool/ImportTool.java#L165
>     Since, this method does not get called in case incremental mode is lastmodified. My changes preserve existing behavior for the most part - changing it only for Oracle. Hence, I have added a new method to the OracleManager.
>     
>     I have added a test as per your comment.
>     
>     Thanks,
>     Raghav.

Thank you Raghav for the explanation, it make complete sense to me! I feel that introducing new method into the ConnManager class for retrieving one column type is a bit of overhaul when we can do simply manager.getColumnTypes(table, query).get(options.getIncrementalTestColumn()) [1] to achieve the same with existing code. Is there a particular reason why you've choose implementing a new method instead?

Links:
1: https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/manager/ConnManager.java#L342


- Jarek


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12032/#review22413
-----------------------------------------------------------


On June 30, 2013, 2:18 p.m., Raghav Gautam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12032/
> -----------------------------------------------------------
> 
> (Updated June 30, 2013, 2:18 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-906
>     https://issues.apache.org/jira/browse/SQOOP-906
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> Incremental import using lastmodified mode always assumes column type to be TIMESTAMP which is causing issues with Oracle Connector. This patch fixes that.
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/manager/ConnManager.java c9e05da 
>   src/java/org/apache/sqoop/manager/OracleManager.java edc888e 
>   src/java/org/apache/sqoop/tool/ImportTool.java cb800b6 
>   src/test/com/cloudera/sqoop/manager/OracleIncrementalImportTest.java PRE-CREATION 
>   src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java 877d7f8 
> 
> Diff: https://reviews.apache.org/r/12032/diff/
> 
> 
> Testing
> -------
> 
> Unit tests & oracle third party tests are passing. Also tested manually.
> 
> 
> Thanks,
> 
> Raghav Gautam
> 
>