[jira] [Created] (SQOOP-1656) SQOOP2: Support for "merge" with different changing dimension types

["Gwen Shapira (JIRA)" <ji...@apache.org>]

Gwen Shapira created SQOOP-1656:
-----------------------------------

             Summary: SQOOP2: Support for "merge" with different changing dimension types
                 Key: SQOOP-1656
                 URL: https://issues.apache.org/jira/browse/SQOOP-1656
             Project: Sqoop
          Issue Type: Wish
            Reporter: Gwen Shapira


Our current "incremental" design is for "append" only.

However, we do plan on adding "merge" capabilities somewhere in the future. Maybe.

Sqoop1 merges by overwriting existing rows with their newer versions.

But for DWH dimensions, there are other ways to merge:
http://en.wikipedia.org/wiki/Slowly_changing_dimension

For example, preserving both versions and adding "start date" and "end date" for each.

ETL tools can handle these situations. Will be cool if Sqoop2 will be able to do it too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)