[jira] [Work logged] (HIVE-25900) Materialized view registry does not clean non existing views at refresh

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25900?focusedWorklogId=716546&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-716546 ]

ASF GitHub Bot logged work on HIVE-25900:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 27/Jan/22 16:59
            Start Date: 27/Jan/22 16:59
    Worklog Time Spent: 10m 
      Work Description: kasakrisz opened a new pull request #2984:
URL: https://github.com/apache/hive/pull/2984


   ### What changes were proposed in this pull request?
   1. Split implementation of `HiveMaterializedViewsRegistry` to a default `MaterializedViewsRegistry` and `InMemoryMaterializedViewsRegistry`
   2. Add functionality to remove dropped MVs from cache at refresh
   3. Add unit tests for Refresh algorithm
   
   ### Why are the changes needed?
   Dropped MVs stayed forver in the cache if the Drop operation was done by a different HS2 instance.
   
   ### Does this PR introduce _any_ user-facing change?
   No.
   
   ### How was this patch tested?
   ```
   mvn test -Dtest.output.overwrite -DskipSparkTests -Dtest=TestMiniLlapLocalCliDriver -Dqfile=materialized_view_create_rewrite_rebuild_dummy.q,materialized_view_create_rewrite_dummy.q -pl itests/qtest -Pitests
   mvn test -Dtest=TestHiveMaterializedViewsRegistryLoader -pl ql
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 716546)
    Remaining Estimate: 0h
            Time Spent: 10m

> Materialized view registry does not clean non existing views at refresh
> -----------------------------------------------------------------------
>
>                 Key: HIVE-25900
>                 URL: https://issues.apache.org/jira/browse/HIVE-25900
>             Project: Hive
>          Issue Type: Bug
>          Components: Materialized views
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> CBO plans of materialized views which are enabled for query rewrite are cached in HS2 (MaterializedViewsCache, HiveMaterializedViewsRegistry)
> The registry is refreshed periodically from HMS:
> {code:java}
> set hive.server2.materializedviews.registry.refresh.period=1500s;
> {code}
> This functionality is required when multiple HS2 instances are used in a cluster: MV drop operation is served by one of the HS2 instances and the registry is updated at that time in that instance. However other HS2 instances still cache the non-existent view and need to be refreshed by the updater thread.
> Currently the updater thread adds new entries, refresh existing ones but does not remove the outdated entries.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)