You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2019/07/21 21:21:00 UTC
[jira] [Comment Edited] (HIVE-21637) Synchronized metastore cache
[ https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889685#comment-16889685 ]
Daniel Dai edited comment on HIVE-21637 at 7/21/19 9:20 PM:
------------------------------------------------------------
Green run with CachedStore off, and #18119 is the result with CachedStore on with 28 test failures. All these failures are investigated and reasons are:
Due to switch to DbTxnManager:
lock2,lock1,insert_into1,insert_into3,insert_into4,lockneg1,lockneg3,lockneg4,lockneg_try_db_lock_conflict,lockneg2,lockneg_try_drop_locked_db,lockneg_try_lock_db_in_use,insert_into2,lockneg_query_tbl_in_locked_db: DbTxnManager does not allow "lock table"
lock4, lock3: show locks w/wo header with different TxnManager
lockneg5: show locks tbl behave differently with different TxnManager
Due to enabling DbNotificationListener
repl_3_exim_metadata, repl_2_exim_basic: repl.last.id are not 0 due to the usage of DbNotificationListener
alter_table_update_status,alter_table_update_status_disable_bitvector,alter_partition_update_status: update statistics for column does not contain required column, DbNotificationListener refuse to serialize
results_cache_invalidation2: exception when both transactional/nontransactional listener both hooked
orc_wide_table: message too long for DbNotification listener
Due to CachedStore change
materialized_view_describe, materialized_view_create: "Outdated for Rewriting" No (with cache) vs Unknown, I believe cached behavior is the right one
sample10_mm: different sample set retrieved, benign
explain_locks: show lock partition order change, benign
was (Author: daijy):
Green run with CachedStore off, and #18119 is the result with CachedStore on with 28 test failures. All these failures are investigated and reasons are:
Due to switch to DbTxnManager:
lock2,lock1,insert_into1,insert_into3,insert_into4,lockneg1,lockneg3,lockneg4,lockneg_try_db_lock_conflict,lockneg2,lockneg_try_drop_locked_db,lockneg_try_lock_db_in_use,insert_into2,lockneg_query_tbl_in_locked_db: DbTxnManager does not allow "lock table"
lock4, lock3: show locks w/wo header with different TxnManager
lockneg5: show locks tbl behave differently with different TxnManager
Due to enabling DbNotificationListener
repl_3_exim_metadata, repl_2_exim_basic: repl.last.id are not 0 due to the usage of DbNotificationListener
alter_table_update_status,alter_table_update_status_disable_bitvector,alter_partition_update_status: update statistics for column does not contain required column, DbNotificationListener refuse to serialize
results_cache_invalidation2: exception when both transactional/nontransactional listener both hooked
orc_wide_table: message too long for DbNotification listener
Due to CachedStore change
materialized_view_describe, materialized_view_create: "Outdated for Rewriting" No (with cache) vs Unknown, I believe cached behavior is the right one
sample10_mm: order change, benign
explain_locks: show lock partition order change, benign
> Synchronized metastore cache
> ----------------------------
>
> Key: HIVE-21637
> URL: https://issues.apache.org/jira/browse/HIVE-21637
> Project: Hive
> Issue Type: New Feature
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Priority: Major
> Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch, HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch, HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch, HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch, HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch, HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch, HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch, HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch, HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch, HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch, HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch, HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch, HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch, HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch, HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch, HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch, HIVE-21637.6.patch, HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch
>
>
> Currently, HMS has a cache implemented by CachedStore. The cache is asynchronized and in HMS HA setting, we can only get eventual consistency. In this Jira, we try to make it synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)