You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Jacques (Jira)" <ji...@apache.org> on 2021/04/08 12:19:00 UTC
[jira] [Commented] (HIVE-11388) Allow ACID Compactor components to
run in multiple metastores
[ https://issues.apache.org/jira/browse/HIVE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317139#comment-17317139 ]
Jacques commented on HIVE-11388:
--------------------------------
The documentation for hive.compactor.initiator.on
at [https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties] still mentions "It's critical that this is enabled on exactly one metastore service instance (not enforced yet)."
i.e. missing the "As of Hive 1.3.0 this property may be enabled on any number of standalone metastore instances." comment added to the link above.
Should both links be in sync?
> Allow ACID Compactor components to run in multiple metastores
> -------------------------------------------------------------
>
> Key: HIVE-11388
> URL: https://issues.apache.org/jira/browse/HIVE-11388
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 1.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
> Fix For: 1.3.0, 2.1.0
>
> Attachments: HIVE-11388.2.patch, HIVE-11388.4.patch, HIVE-11388.5.patch, HIVE-11388.6.patch, HIVE-11388.7.patch, HIVE-11388.branch-1.patch, HIVE-11388.patch
>
>
> (this description is no loner accurate; see further comments)
> org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs inside the metastore service to manage compactions of ACID tables. There should be exactly 1 instance of this thread (even with multiple Thrift services).
> This is documented in https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration but not enforced.
> Should add enforcement, since more than 1 Initiator could cause concurrent attempts to compact the same table/partition - which will not work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)