You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org> on 2011/08/18 19:48:29 UTC
[jira] [Commented] (SQOOP-314) Basic export hangs when target
database does not support INSERT syntax with multiple rows of values
[ https://issues.apache.org/jira/browse/SQOOP-314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087143#comment-13087143 ]
jiraposter@reviews.apache.org commented on SQOOP-314:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1585/
-----------------------------------------------------------
(Updated 2011-08-18 17:46:40.735309)
Review request for Sqoop, Arvind Prabhakar and jmhsieh.
Summary
-------
Basic export job will hang when the target database does not support insert syntax with multiple rows of values, such as INSERT INTO tbl (col1, col2) VALUES(11, 12),(21, 22),(23, 24)
This is because, in close(), AsyncSqlRecordWriter will still wait for AsyncSqlExecThread to finish even when an SQLException is thrown underneath.
The fix is to introduce a new "--batch" option to execute underlying statements in batch mode instead of a single multirow insert statement.
This addresses bug SQOOP-314.
https://issues.apache.org/jira/browse/SQOOP-314
Diffs
-----
src/docs/man/sqoop-export.txt 6090ea1
src/docs/user/export.txt 4f87886
src/java/com/cloudera/sqoop/SqoopOptions.java d760d39
src/java/com/cloudera/sqoop/mapreduce/AsyncSqlRecordWriter.java 193cf41
src/java/com/cloudera/sqoop/mapreduce/ExportBatchOutputFormat.java PRE-CREATION
src/java/com/cloudera/sqoop/mapreduce/ExportJobBase.java 9799e37
src/java/com/cloudera/sqoop/mapreduce/ExportOutputFormat.java d2a6cf6
src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 8f629f1
src/java/com/cloudera/sqoop/tool/ExportTool.java b4b1091
src/test/com/cloudera/sqoop/manager/JdbcMySQLExportTest.java 8687b0c
Diff: https://reviews.apache.org/r/1585/diff
Testing
-------
Thanks,
Bilung
> Basic export hangs when target database does not support INSERT syntax with multiple rows of values
> ---------------------------------------------------------------------------------------------------
>
> Key: SQOOP-314
> URL: https://issues.apache.org/jira/browse/SQOOP-314
> Project: Sqoop
> Issue Type: Bug
> Reporter: Bilung Lee
> Assignee: Bilung Lee
>
> Basic export job will hang when the target database does not support INSERT syntax with multiple rows of values, such as INSERT INTO tbl (col1, col2) VALUES(11, 12),(21, 22),(23, 24)
> This is because, in close(), AsyncSqlRecordWriter will still wait for AsyncSqlExecThread to finish even when an SQLException is thrown underneath.
> The configuration variable "sqoop.export.records.per.statement" can be set to 1 as a workaround for this problem.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira