[jira] [Updated] (SQOOP-1617) MySQL fetch-size behavior changed with SQOOP-1400

["Jarek Jarcec Cecho (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SQOOP-1617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jarek Jarcec Cecho updated SQOOP-1617:
--------------------------------------
    Attachment: SQOOP-1617.patch

It seems that our change in SQOOP-1400 has changed the default behavior from "streaming row by row" to "cache everything in memory" which is causing various out of memory problems to our users as Sqoop has been design to transfer huge amount of data. Hence I'm recommending to put the code back.

> MySQL fetch-size behavior changed with SQOOP-1400
> -------------------------------------------------
>
>                 Key: SQOOP-1617
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1617
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/mysql
>    Affects Versions: 1.4.6
>         Environment: CDH 5.2
> sqoop 1.4.5 (seems to include SQOOP-1400)
> mysql connector version 5.1.33
>            Reporter: Jürgen Thomann
>            Priority: Minor
>             Fix For: 1.4.6
>
>         Attachments: SQOOP-1617.patch
>
>
> SQOOP-1400 changed the default behavior for the connector to load everything in memory. The only working way to get the old streaming back is to use --fetch-size -2147483648 (Integer.MIN_VALUE)
> It would be nice if that could be changed and/or documented that mysql does not support a fetch size and does only support row-by-row or loading everything in memory



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)