[jira] [Resolved] (OOZIE-1521) Optimize I/O for Sqoop workflow launched by Oozie

["Shuaishuai Nie (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-1521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shuaishuai Nie resolved OOZIE-1521.
-----------------------------------

    Resolution: Not A Problem
    
> Optimize I/O for Sqoop workflow launched by Oozie
> -------------------------------------------------
>
>                 Key: OOZIE-1521
>                 URL: https://issues.apache.org/jira/browse/OOZIE-1521
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Shuaishuai Nie
>         Attachments: OOZIE-1521.1.patch
>
>
> Now Sqoop work flow launched by Oozie will copy the jar files in  %SQOOP_HOME%\lib folder to the job cache each time a Sqoop job is launched. We should add these jars to Sqoop sharelib so that for each worker node, the jars in the Sqoop sharelib only copied to distribute cache and localize to each worker node once and reuse by all the subsequent Sqoop jobs. In Sqoop-1192, an option 'skip-dist-cache' is introduced which disable Sqoop add lib jars to job cache action and reuse the jars in the Sqoop sharelib. In the patch attached, Oozie will add "--skip-dist-cache" option to Sqoop argument if share lib is used. Also add the Sqoop dependencies in the Sqoop share lib.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira