[jira] [Commented] (SQOOP-2983) OraOop export has degraded performance with wide tables

["Attila Szabo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414395#comment-15414395 ] 

Attila Szabo commented on SQOOP-2983:
-------------------------------------

With the help of [~david.robson] I was able to identify one issue around "update-key" option, and also was able to spot another issue (left behind after the changes around Oracle escaped column name support). Both of them are fixed. New test case attached as well. New diff reflects all of the changes. Please do another round of review!

> OraOop export has degraded performance with wide tables
> -------------------------------------------------------
>
>                 Key: SQOOP-2983
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2983
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Attila Szabo
>            Assignee: Attila Szabo
>            Priority: Critical
>         Attachments: SQOOP-2983-1.patch
>
>
> The current version of OraOOP seems to perform very low from performance POV when --direct mode turned on (regardless if the partitioned feature is turned of).
> Just as a baseline from the current trunk version:
> Inserting 100.000 rows into a 800 column wide Oracle table has 400-600 kb/sec with direct mode on my cluster, while the standard oracle driver can produce up to 1.2-1.8 mb/sec. (depending on the number of mappers, batch size).
> Inserting 1.000.000 rows into the same table goes up to 800k-1mb/sec with OraOOP, however with the standard Oracle connector it's around 3.5mb/sec.
> It seems OraOOP export needs a thorough review and some fixing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)