You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hive.apache.org by "Sankar Hariappan (JIRA)" <ji...@apache.org> on 2019/02/14 06:56:00 UTC

[jira] [Updated] (HIVE-21269) Mandate -update and -delete as DistCp options to avoid data inconsistency with external tables replication.

     [ https://issues.apache.org/jira/browse/HIVE-21269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sankar Hariappan updated HIVE-21269:
------------------------------------
    Summary:  Mandate -update and -delete as DistCp options to avoid data inconsistency with external tables replication.  (was: Hive replication should mandate -update and -delete as DistCp options to avoid data inconsistency.)

>  Mandate -update and -delete as DistCp options to avoid data inconsistency with external tables replication.
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-21269
>                 URL: https://issues.apache.org/jira/browse/HIVE-21269
>             Project: Hive
>          Issue Type: Bug
>          Components: repl
>    Affects Versions: 4.0.0
>            Reporter: Sankar Hariappan
>            Assignee: Sankar Hariappan
>            Priority: Major
>              Labels: DR, replication
>
> Currently, external tables replication, copies the data in directory level. So, if target directory exist, then DistCp should compare and update or skip data files in the directory instead of creating new directory inside pre-existing target directory.
> This can be achieved using -update.
> Also, -delete option is needed to delete the files missing in source directory but present in target.
> Hive should mandate these DistCp options even if user passes other options.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)