[jira] [Issue Comment Deleted] (SQOOP-1532) Sqoop2: Support Sqoop on Spark Execution Engine

["Enrique Ruiz Garcia (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SQOOP-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Enrique Ruiz Garcia updated SQOOP-1532:
---------------------------------------
    Comment: was deleted

(was: We have implemented the Issue SQOOP-1532 to support sqoop jobs to run on spark environment. You can run on the standalone spark cluster, using sqoop client.)

> Sqoop2: Support Sqoop on Spark Execution Engine
> -----------------------------------------------
>
>                 Key: SQOOP-1532
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1532
>             Project: Sqoop
>          Issue Type: Improvement
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 2.0.0
>
>
> The current execution engine supported in sqoop is MR.
> The goal if this ticket is to support sqoop jobs ( map only and map+reduce ) to run on spark environment.
> It should at the minimum support running on the standalone spark cluster and then subsequently work with YARN/mesos.
> High level goals
> 1. Hook up with the connector apis to provide the basic load/ extract to the spark RDD.
> 2. Implementation of the Sqoop RDD to support extraction from different data sources . The design proposal will discuss the alternatives on how this can be achieved.
> 3. Optimizing the loading/writing ( re-use/ refactor the consumer thread code to be agnostic of the hadoop output format)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)