You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Bilung Lee (JIRA)" <ji...@apache.org> on 2011/08/19 20:22:29 UTC
[jira] [Created] (SQOOP-321) Support date/time columns for
"--incremental append" option
Support date/time columns for "--incremental append" option
-----------------------------------------------------------
Key: SQOOP-321
URL: https://issues.apache.org/jira/browse/SQOOP-321
Project: Sqoop
Issue Type: Improvement
Reporter: Bilung Lee
Assignee: Bilung Lee
Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
One workaround to "mimic" append mode with date/time values is
1) select max(dt) from table into max_dt
2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "Arvind Prabhakar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arvind Prabhakar resolved SQOOP-321.
------------------------------------
Resolution: Fixed
Fix Version/s: 1.4.0
Patch committed. Thanks Bilung!
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
> Fix For: 1.4.0
>
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090806#comment-13090806 ]
jiraposter@reviews.apache.org commented on SQOOP-321:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1623/#review1634
-----------------------------------------------------------
Ship it!
+1
- Arvind
On 2011-08-25 01:57:36, Bilung Lee wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1623/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-25 01:57:36)
bq.
bq.
bq. Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
bq.
bq.
bq. This addresses bug SQOOP-321.
bq. https://issues.apache.org/jira/browse/SQOOP-321
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/com/cloudera/sqoop/manager/ConnManager.java cea0a73
bq. src/java/com/cloudera/sqoop/manager/OracleManager.java bad45dc
bq. src/java/com/cloudera/sqoop/tool/ImportTool.java 66e60bd
bq. src/test/com/cloudera/sqoop/TestIncrementalImport.java 96d4309
bq.
bq. Diff: https://reviews.apache.org/r/1623/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Bilung
bq.
bq.
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090687#comment-13090687 ]
jiraposter@reviews.apache.org commented on SQOOP-321:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1623/
-----------------------------------------------------------
(Updated 2011-08-25 01:57:36.357578)
Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.
Summary
-------
Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
This addresses bug SQOOP-321.
https://issues.apache.org/jira/browse/SQOOP-321
Diffs (updated)
-----
src/java/com/cloudera/sqoop/manager/ConnManager.java cea0a73
src/java/com/cloudera/sqoop/manager/OracleManager.java bad45dc
src/java/com/cloudera/sqoop/tool/ImportTool.java 66e60bd
src/test/com/cloudera/sqoop/TestIncrementalImport.java 96d4309
Diff: https://reviews.apache.org/r/1623/diff
Testing
-------
Thanks,
Bilung
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089979#comment-13089979 ]
jiraposter@reviews.apache.org commented on SQOOP-321:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1623/#review1610
-----------------------------------------------------------
Changes look good Bilung. I do have a backward compatibility concern noted below. Other than that, it will be good to give some examples of use of this feature in the user guide. The more specific examples you can give, the better it would be for the users.
src/java/com/cloudera/sqoop/manager/ConnManager.java
<https://reviews.apache.org/r/1623/#comment3639>
This may be a compatibility issue for third party connectors. It is best to leave this method as is and introduce a new method that gets called when the check column type is not numeric.
- Arvind
On 2011-08-22 23:20:55, Bilung Lee wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1623/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-22 23:20:55)
bq.
bq.
bq. Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
bq.
bq.
bq. This addresses bug SQOOP-321.
bq. https://issues.apache.org/jira/browse/SQOOP-321
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/com/cloudera/sqoop/manager/ConnManager.java cea0a73
bq. src/java/com/cloudera/sqoop/manager/OracleManager.java bad45dc
bq. src/java/com/cloudera/sqoop/tool/ImportTool.java 66e60bd
bq. src/test/com/cloudera/sqoop/TestIncrementalImport.java 96d4309
bq.
bq. Diff: https://reviews.apache.org/r/1623/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Bilung
bq.
bq.
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089142#comment-13089142 ]
jiraposter@reviews.apache.org commented on SQOOP-321:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1623/
-----------------------------------------------------------
Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.
Summary
-------
Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
This addresses bug SQOOP-321.
https://issues.apache.org/jira/browse/SQOOP-321
Diffs
-----
src/java/com/cloudera/sqoop/manager/ConnManager.java cea0a73
src/java/com/cloudera/sqoop/manager/OracleManager.java bad45dc
src/java/com/cloudera/sqoop/tool/ImportTool.java 66e60bd
src/test/com/cloudera/sqoop/TestIncrementalImport.java 96d4309
Diff: https://reviews.apache.org/r/1623/diff
Testing
-------
Thanks,
Bilung
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "Bilung Lee (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bilung Lee updated SQOOP-321:
-----------------------------
Attachment: SQOOP-321.diff
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
> Fix For: 1.4.0
>
> Attachments: SQOOP-321.diff
>
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090807#comment-13090807 ]
jiraposter@reviews.apache.org commented on SQOOP-321:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1623/#review1633
-----------------------------------------------------------
Ship it!
+1
- Arvind
On 2011-08-25 01:57:36, Bilung Lee wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1623/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-25 01:57:36)
bq.
bq.
bq. Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
bq.
bq.
bq. This addresses bug SQOOP-321.
bq. https://issues.apache.org/jira/browse/SQOOP-321
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/com/cloudera/sqoop/manager/ConnManager.java cea0a73
bq. src/java/com/cloudera/sqoop/manager/OracleManager.java bad45dc
bq. src/java/com/cloudera/sqoop/tool/ImportTool.java 66e60bd
bq. src/test/com/cloudera/sqoop/TestIncrementalImport.java 96d4309
bq.
bq. Diff: https://reviews.apache.org/r/1623/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Bilung
bq.
bq.
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090686#comment-13090686 ]
jiraposter@reviews.apache.org commented on SQOOP-321:
-----------------------------------------------------
bq. On 2011-08-24 03:29:53, Arvind Prabhakar wrote:
bq. > src/java/com/cloudera/sqoop/manager/ConnManager.java, lines 300-301
bq. > <https://reviews.apache.org/r/1623/diff/1/?file=34386#file34386line300>
bq. >
bq. > This may be a compatibility issue for third party connectors. It is best to leave this method as is and introduce a new method that gets called when the check column type is not numeric.
Existing method is kept in new patch. Thanks!
- Bilung
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1623/#review1610
-----------------------------------------------------------
On 2011-08-25 01:57:36, Bilung Lee wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1623/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-25 01:57:36)
bq.
bq.
bq. Review request for Sqoop, Ahmed Radwan and Arvind Prabhakar.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
bq.
bq.
bq. This addresses bug SQOOP-321.
bq. https://issues.apache.org/jira/browse/SQOOP-321
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/com/cloudera/sqoop/manager/ConnManager.java cea0a73
bq. src/java/com/cloudera/sqoop/manager/OracleManager.java bad45dc
bq. src/java/com/cloudera/sqoop/tool/ImportTool.java 66e60bd
bq. src/test/com/cloudera/sqoop/TestIncrementalImport.java 96d4309
bq.
bq. Diff: https://reviews.apache.org/r/1623/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Bilung
bq.
bq.
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-321) Support date/time columns for
"--incremental append" option
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090859#comment-13090859 ]
Hudson commented on SQOOP-321:
------------------------------
Integrated in Sqoop-jdk-1.6 #18 (See [https://builds.apache.org/job/Sqoop-jdk-1.6/18/])
SQOOP-321. Support date/time for incremental append imports.
(Bilung Lee via Arvind Prabhakar)
arvind : http://svn.apache.org/viewvc/?view=rev&rev=1161404
Files :
* /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/ConnManager.java
* /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/manager/OracleManager.java
* /incubator/sqoop/trunk/src/java/com/cloudera/sqoop/tool/ImportTool.java
* /incubator/sqoop/trunk/src/test/com/cloudera/sqoop/TestIncrementalImport.java
> Support date/time columns for "--incremental append" option
> -----------------------------------------------------------
>
> Key: SQOOP-321
> URL: https://issues.apache.org/jira/browse/SQOOP-321
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Bilung Lee
> Assignee: Bilung Lee
> Fix For: 1.4.0
>
>
> Currently, when "--incremental append" option is used, only numeric columns are supported for check column.
> One workaround to "mimic" append mode with date/time values is
> 1) select max(dt) from table into max_dt
> 2) sqoop import --incremental lastmodified --check-column dt --last-value max_dt
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira