You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by "Venkat, Ankam" <An...@centurylink.com> on 2014/08/01 00:35:28 UTC

Change Data Capture in Hive

Experts,

Can you please suggest the strategies with some code samples for doing Change Data Capture in Hive?

Right now, we are staging lot of tables from Oracle to Oracle with change data capture (mainly Type 2).   We are planning to migrate all the staging tables to Hadoop.
We do bulk import on day 1 and then followed by incremental imports which contains inserts, updates and deletes.

How can we apply these inserts, updates and deletes on Hive tables to achieve Type2, Type1 and Type 0 based CDC?

CDC Type 0: New Records only; Discard Changes & Deletes
CDC Type 1: Overwrite
CDC Type 2: Expire and Create New Record

Regards,
Venkat Ankam