You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gitbox@hive.apache.org by GitBox <gi...@apache.org> on 2022/01/26 12:09:30 UTC
[GitHub] [hive] kasakrisz opened a new pull request #2975: HIVE-25899: Materialized view registry does not clean dropped views
kasakrisz opened a new pull request #2975:
URL: https://github.com/apache/hive/pull/2975
### What changes were proposed in this pull request?
`MaterializedViewsCache` nested maps.
```
somedb -> someview -> Materialization
```
1. When removing entries from the inner map check whether that map is empty and remove it from the outer map.
2. Add `isEmpty()` method to `HiveMaterializedViewsRegistry`
### Why are the changes needed?
See description of jira.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
```
mvn test -Dtest=TestMaterializedViewsCache -pl ql
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kasakrisz merged pull request #2975: HIVE-25899: Materialized view registry does not clean dropped views
Posted by GitBox <gi...@apache.org>.
kasakrisz merged pull request #2975:
URL: https://github.com/apache/hive/pull/2975
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kgyrtkirk commented on a change in pull request #2975: HIVE-25899: Materialized view registry does not clean dropped views
Posted by GitBox <gi...@apache.org>.
kgyrtkirk commented on a change in pull request #2975:
URL: https://github.com/apache/hive/pull/2975#discussion_r793427503
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/MaterializedViewsCache.java
##########
@@ -124,30 +124,44 @@ public void remove(Table materializedViewTable) {
// Delete only if the create time for the input materialized view table and the table
// in the map match. Otherwise, keep the one in the map.
dbMap.computeIfPresent(materializedViewTable.getTableName(), (mvTableName, oldMaterialization) -> {
- if (HiveMaterializedViewUtils.extractTable(oldMaterialization).equals(materializedViewTable)) {
- List<HiveRelOptMaterialization> materializationList =
- sqlToMaterializedView.get(materializedViewTable.getViewExpandedText());
- materializationList.remove(oldMaterialization);
+ Table oldTable = HiveMaterializedViewUtils.extractTable(oldMaterialization);
+ if (oldTable.equals(materializedViewTable)) {
+ remove(oldMaterialization, oldTable);
return null;
}
return oldMaterialization;
});
+
+ if (dbMap.isEmpty()) {
+ materializedViews.remove(materializedViewTable.getDbName());
+ }
}
LOG.debug("Materialized view {}.{} removed from registry",
materializedViewTable.getDbName(), materializedViewTable.getTableName());
}
+ private void remove(HiveRelOptMaterialization materialization, Table mvTable) {
+ List<HiveRelOptMaterialization> materializationList =
+ sqlToMaterializedView.get(mvTable.getViewExpandedText());
+ materializationList.remove(materialization);
Review comment:
can this be null?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org
[GitHub] [hive] kasakrisz commented on a change in pull request #2975: HIVE-25899: Materialized view registry does not clean dropped views
Posted by GitBox <gi...@apache.org>.
kasakrisz commented on a change in pull request #2975:
URL: https://github.com/apache/hive/pull/2975#discussion_r794316518
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/metadata/MaterializedViewsCache.java
##########
@@ -124,30 +124,44 @@ public void remove(Table materializedViewTable) {
// Delete only if the create time for the input materialized view table and the table
// in the map match. Otherwise, keep the one in the map.
dbMap.computeIfPresent(materializedViewTable.getTableName(), (mvTableName, oldMaterialization) -> {
- if (HiveMaterializedViewUtils.extractTable(oldMaterialization).equals(materializedViewTable)) {
- List<HiveRelOptMaterialization> materializationList =
- sqlToMaterializedView.get(materializedViewTable.getViewExpandedText());
- materializationList.remove(oldMaterialization);
+ Table oldTable = HiveMaterializedViewUtils.extractTable(oldMaterialization);
+ if (oldTable.equals(materializedViewTable)) {
+ remove(oldMaterialization, oldTable);
return null;
}
return oldMaterialization;
});
+
+ if (dbMap.isEmpty()) {
+ materializedViews.remove(materializedViewTable.getDbName());
+ }
}
LOG.debug("Materialized view {}.{} removed from registry",
materializedViewTable.getDbName(), materializedViewTable.getTableName());
}
+ private void remove(HiveRelOptMaterialization materialization, Table mvTable) {
+ List<HiveRelOptMaterialization> materializationList =
+ sqlToMaterializedView.get(mvTable.getViewExpandedText());
+ materializationList.remove(materialization);
Review comment:
Added check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org