You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Eugene Koifman (JIRA)" <ji...@apache.org> on 2017/10/17 00:53:00 UTC
[jira] [Issue Comment Deleted] (HIVE-16669) Fine tune Compaction to
take advantage of Acid 2.0
[ https://issues.apache.org/jira/browse/HIVE-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eugene Koifman updated HIVE-16669:
----------------------------------
Comment: was deleted
(was: OK, this is more than just fine tuning. Suppose we have
base_8
delta_9
delete_delta_10 - this affects rows in base_8
Minor compaction (as currently implemented (inherited from Acid 1)) will produce delta_9_10 which means all deletes by txn 10 affecting rows in base_8 are lost.
so HIVE-17089 is effectively incomplete w/o this)
> Fine tune Compaction to take advantage of Acid 2.0
> --------------------------------------------------
>
> Key: HIVE-16669
> URL: https://issues.apache.org/jira/browse/HIVE-16669
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
>
> * There is little point using 2.0 vectorized reader since there is no operator pipeline in compaction
> * If minor compaction just concats delete_delta files together, then the 2 stage compaction should always ensure that we have a limited number of Orc readers to do the merging and current OrcRawRecordMerger should be fine
> * ...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)