You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Veena Basavaraj (JIRA)" <ji...@apache.org> on 2015/03/06 18:09:38 UTC
[jira] [Updated] (SQOOP-1532) Sqoop2: Support Sqoop on Spark
Execution Engine
[ https://issues.apache.org/jira/browse/SQOOP-1532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Veena Basavaraj updated SQOOP-1532:
-----------------------------------
Description:
The current execution engine supported in sqoop is MR.
The goal if this ticket is to support sqoop jobs ( map only and map+reduce ) to run on spark environment.
It should at the minimum support running on the standalone spark cluster and then subsequently work with YARN/mesos.
High level goals
1. Hook up with the connector apis to provide the basic load/ extract to the spark RDD.
2. Implementation of the Sqoop RDD to support extraction from different data sources . The design proposal will discuss the alternatives on how this can be achieved.
3. Optimizing the loading/writing with
was:
The goal of this ticket is to support a pluggable way to select the execution engine on which we can run the sqoop jobs.
> Sqoop2: Support Sqoop on Spark Execution Engine
> -----------------------------------------------
>
> Key: SQOOP-1532
> URL: https://issues.apache.org/jira/browse/SQOOP-1532
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 1.99.6
>
>
> The current execution engine supported in sqoop is MR.
> The goal if this ticket is to support sqoop jobs ( map only and map+reduce ) to run on spark environment.
> It should at the minimum support running on the standalone spark cluster and then subsequently work with YARN/mesos.
> High level goals
> 1. Hook up with the connector apis to provide the basic load/ extract to the spark RDD.
> 2. Implementation of the Sqoop RDD to support extraction from different data sources . The design proposal will discuss the alternatives on how this can be achieved.
> 3. Optimizing the loading/writing with
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)