You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Veena Basavaraj (JIRA)" <ji...@apache.org> on 2015/03/23 21:51:53 UTC
[jira] [Comment Edited] (SQOOP-1810) GenericJdbcConnector :
ToJobConfiguration and Loader enhancements for Incremental write ( INSERT
or UPDATE)
[ https://issues.apache.org/jira/browse/SQOOP-1810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376343#comment-14376343 ]
Veena Basavaraj edited comment on SQOOP-1810 at 3/23/15 8:51 PM:
-----------------------------------------------------------------
The patch contains the following
1. A new IncrementalWriteConfig similar to IncrementalRead, it has a enum that expects a value for type of writes to happen
a. INSERT ( default)
b. UPSERT ( insert or update ), for new rows will be inserted
c. UPDAT ( only updates, new rows are left)
Not in the patch
d. DELETE ( in future even support deleting records if need be )
2. Changes to the Loader to use the relevant JDBC statements, such as INSERT or UPDATE based on the primary key
3. The integration test to assert the insert / upsert happened
The current patch (version 1) uploaded does not have an integration test, works manually, if this approach is acceptable, I will follow up with the full patch with integration tests and also support UPSERT, which should be along the same lines, but does a SELECT before updating or inserting. The code in the JDBCToInitializer might need some refactor to cleanly support this, since there is a lot of overlap between the insert and update case.
was (Author: vybs):
The patch contains the following
1. A new IncrementalWriteConfig similar to IncrementalRead, it has a enum that expects a value for type of writes to happen
a. INSERT ( default)
b. UPSERT ( insert or update ), for new rows will be inserted
c. UPDAT ( only updates, new rows are left)
Not in the patch
d. DELETE ( in future even support deleting records if need be )
2. Changes to the Loader to use the relevant JDBC statements, such as INSERT or UPDATE based on the primary key
3. The integration test to assert the insert / upsert happened
> GenericJdbcConnector : ToJobConfiguration and Loader enhancements for Incremental write ( INSERT or UPDATE)
> -----------------------------------------------------------------------------------------------------------
>
> Key: SQOOP-1810
> URL: https://issues.apache.org/jira/browse/SQOOP-1810
> Project: Sqoop
> Issue Type: Bug
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Fix For: 2.0.0
>
>
> This feature is a POC to make JDBC Connector support Incremental/Delta read and writes
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)