You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Veena Basavaraj (JIRA)" <ji...@apache.org> on 2014/12/22 21:27:13 UTC
[jira] [Updated] (SQOOP-1803) JobManager and Execution Engine
changes: Support for a injecting and pulling out configs and job output in
connectors
[ https://issues.apache.org/jira/browse/SQOOP-1803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Veena Basavaraj updated SQOOP-1803:
-----------------------------------
Description:
The details are in the design wiki, as the implementation happens more discussions can happen here.
https://cwiki.apache.org/confluence/display/SQOOP/Delta+Fetch+And+Merge+Design#DeltaFetchAndMergeDesign-Howtogettheinputfromsqoopintoconnector?
https://cwiki.apache.org/confluence/display/SQOOP/Delta+Fetch+And+Merge+Design#DeltaFetchAndMergeDesign-Howtogetoutputfromconnectortosqoop?
The goal is to dynamically inject a IncrementalConfig instance into the FromJobConfiguration. The current MFromConfig and MToConfig can already hold a list of configs, and a strong sentiment was expressed to keep it as a list, why not for the first time actually make use of it and group the incremental related configs in one config object
This task will prepare the FromJobConfiguration from the job config data, ExtractorContext with the relevant values from the prev job run
This task will prepare the ToJobConfiguration from the job config data, LoaderContext with the relevant values from the prev job run if any
was:
The goal is to dynamically inject a IncrementalConfig instance into the FromJobConfiguration. The current MFromConfig and MToConfig can already hold a list of configs, and a strong sentiment was expressed to keep it as a list, why not for the first time actually make use of it and group the incremental related configs in one config object
This task will prepare the FromJobConfiguration from the job config data, ExtractorContext with the relevant values from the prev job run
This task will prepare the ToJobConfiguration from the job config data, LoaderContext with the relevant values from the prev job run if any
Summary: JobManager and Execution Engine changes: Support for a injecting and pulling out configs and job output in connectors (was: JobManager and Execution Engine changes: Support for a IncrementalConfig in FromJobConfiguration)
> JobManager and Execution Engine changes: Support for a injecting and pulling out configs and job output in connectors
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: SQOOP-1803
> URL: https://issues.apache.org/jira/browse/SQOOP-1803
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 1.99.5
>
>
> The details are in the design wiki, as the implementation happens more discussions can happen here.
> https://cwiki.apache.org/confluence/display/SQOOP/Delta+Fetch+And+Merge+Design#DeltaFetchAndMergeDesign-Howtogettheinputfromsqoopintoconnector?
> https://cwiki.apache.org/confluence/display/SQOOP/Delta+Fetch+And+Merge+Design#DeltaFetchAndMergeDesign-Howtogetoutputfromconnectortosqoop?
> The goal is to dynamically inject a IncrementalConfig instance into the FromJobConfiguration. The current MFromConfig and MToConfig can already hold a list of configs, and a strong sentiment was expressed to keep it as a list, why not for the first time actually make use of it and group the incremental related configs in one config object
> This task will prepare the FromJobConfiguration from the job config data, ExtractorContext with the relevant values from the prev job run
> This task will prepare the ToJobConfiguration from the job config data, LoaderContext with the relevant values from the prev job run if any
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)