[jira] [Created] (HIVE-24481) Skipped compaction can cause data corruption with streaming

["Peter Varga (Jira)" <ji...@apache.org>]

Peter Varga created HIVE-24481:
----------------------------------

             Summary: Skipped compaction can cause data corruption with streaming
                 Key: HIVE-24481
                 URL: https://issues.apache.org/jira/browse/HIVE-24481
             Project: Hive
          Issue Type: Bug
            Reporter: Peter Varga
            Assignee: Peter Varga


Timeline:
1. create a partitioned table, add one static partition
2. transaction 1 writes delta_1, and aborts
3. create streaming connection, with batch 3, withStaticPartitionValues with the existing partition
4. beginTransaction, write, commitTransaction
5. beginTransaction, write, abortTransaction
6. beingTransaction, write, commitTransaction
7. close connection, count of the table is 2
8. run manual minor compaction on the partition. it will skip compaction, because deltacount =1 but clean, because there is aborted txn1
9. cleaner will remove both aborted record from txn_components
10. wait for acidhousekeeper to remove empty aborted txns
11. select * from table return *3* records, reading the aborted record



--
This message was sent by Atlassian Jira
(v8.3.4#803005)