You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "avamingli (via GitHub)" <gi...@apache.org> on 2026/03/02 12:52:08 UTC
[PR] Support matview status maintenance for multi-table JOIN queries [cloudberry]
avamingli opened a new pull request, #1594:
URL: https://github.com/apache/cloudberry/pull/1594
Previously, GetViewBaseRelids() rejected any query with more than one base table, so materialized views defined with JOINs were never registered in gp_matview_aux/gp_matview_tables. This meant no status tracking and no staleness propagation for join matviews.
Add a recursive helper extract_base_relids_from_jointree() that walks RangeTblRef, JoinExpr, and FromExpr nodes to collect all base relation OIDs. This is the only C function changed -- the existing downstream infrastructure (InsertMatviewTablesEntries, SetRelativeMatviewAuxStatus, MaintainMaterializedViewStatus, reference counting) already supports N base tables per matview.
This is a first step toward AQUMV support for join queries. Users can also inspect a join matview's freshness status manually via gp_matview_aux.
Key behaviors:
- Self-joins (t1 JOIN t1) are deduplicated to one catalog entry
- All join types supported: INNER, LEFT, RIGHT, FULL, implicit cross
- Subquery/function RTEs in FROM are still rejected
- Partitioned tables in joins propagate DML status correctly
- Status escalation across multiple base tables works (i→e on delete)
- Transaction rollback correctly reverts status changes
Includes regression tests for: two/three-table joins, implicit joins, self-joins, all outer join types, mixed join types, join with GROUP BY, shared base tables across multiple MVs, multi-DML transactions, transaction rollback, cross joins, partitioned tables in joins, VACUUM FULL, TRUNCATE, WITH NO DATA, and DROP CASCADE.
<!-- Thank you for your contribution to Apache Cloudberry (Incubating)! -->
Fixes #ISSUE_Number
### What does this PR do?
<!-- Brief overview of the changes, including any major features or fixes -->
### Type of Change
- [ ] Bug fix (non-breaking change)
- [ ] New feature (non-breaking change)
- [ ] Breaking change (fix or feature with breaking changes)
- [ ] Documentation update
### Breaking Changes
<!-- Remove if not applicable. If yes, explain impact and migration path -->
### Test Plan
<!-- How did you test these changes? -->
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Passed `make installcheck`
- [ ] Passed `make -C src/test installcheck-cbdb-parallel`
### Impact
<!-- Remove sections that don't apply -->
**Performance:**
<!-- Any performance implications? -->
**User-facing changes:**
<!-- Any changes visible to users? -->
**Dependencies:**
<!-- New dependencies or version changes? -->
### Checklist
- [ ] Followed [contribution guide](https://cloudberry.apache.org/contribute/code)
- [ ] Added/updated documentation
- [ ] Reviewed code for security implications
- [ ] Requested review from [cloudberry committers](https://github.com/orgs/apache/teams/cloudberry-committers)
### Additional Context
<!-- Any other information that would help reviewers? Remove if none -->
### CI Skip Instructions
<!--
To skip CI builds, add the appropriate CI skip identifier to your PR title.
The identifier must:
- Be in square brackets []
- Include the word "ci" and either "skip" or "no"
- Only use for documentation-only changes or when absolutely necessary
-->
---
<!-- Join our community:
- Mailing list: [dev@cloudberry.apache.org](https://lists.apache.org/list.html?dev@cloudberry.apache.org) (subscribe: dev-subscribe@cloudberry.apache.org)
- Discussions: https://github.com/apache/cloudberry/discussions -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Support matview status maintenance for multi-table JOIN queries [cloudberry]
Posted by "avamingli (via GitHub)" <gi...@apache.org>.
avamingli commented on code in PR #1594:
URL: https://github.com/apache/cloudberry/pull/1594#discussion_r2887352270
##########
src/backend/catalog/gp_matview_aux.c:
##########
@@ -109,43 +198,16 @@ GetViewBaseRelids(const Query *viewQuery, bool *has_foreign)
if (contain_mutable_functions((Node*)viewQuery))
return NIL;
- if (list_length(viewQuery->jointree->fromlist) != 1)
- return NIL;
-
- mvjtnode = (Node *) linitial(viewQuery->jointree->fromlist);
- if (!IsA(mvjtnode, RangeTblRef))
- return NIL;
-
- RangeTblEntry *rte = rt_fetch(1, viewQuery->rtable);
- if (rte->rtekind != RTE_RELATION)
- return NIL;
-
- char relkind = get_rel_relkind(rte->relid);
-
- /*
- * Allow foreign table here, however we don't know if the data is
- * up to date or not of the view.
- * But if users want to query matview instead of query foreign tables
- * outside CBDB, let them decide with aqumv_allow_foreign_table.
- */
- if (relkind != RELKIND_RELATION &&
- relkind != RELKIND_PARTITIONED_TABLE &&
- relkind != RELKIND_FOREIGN_TABLE)
- return NIL;
-
if (has_foreign)
- *has_foreign = relkind == RELKIND_FOREIGN_TABLE;
+ *has_foreign = false;
- /*
- * inherit tables are not supported.
- */
- bool can_be_partition = (relkind == RELKIND_PARTITIONED_TABLE) || get_rel_relispartition(rte->relid);
-
- if (!can_be_partition &&
- (has_superclass(rte->relid) || has_subclass(rte->relid)))
+ if (!extract_base_relids_from_jointree((Node *) viewQuery->jointree,
+ viewQuery->rtable,
+ &relids, has_foreign))
return NIL;
- relids = list_make1_oid(rte->relid);
+ if (relids == NIL)
+ return NIL;
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Support matview status maintenance for multi-table JOIN queries [cloudberry]
Posted by "gfphoenix78 (via GitHub)" <gi...@apache.org>.
gfphoenix78 commented on code in PR #1594:
URL: https://github.com/apache/cloudberry/pull/1594#discussion_r2887012886
##########
src/backend/catalog/gp_matview_aux.c:
##########
@@ -109,43 +198,16 @@ GetViewBaseRelids(const Query *viewQuery, bool *has_foreign)
if (contain_mutable_functions((Node*)viewQuery))
return NIL;
- if (list_length(viewQuery->jointree->fromlist) != 1)
- return NIL;
-
- mvjtnode = (Node *) linitial(viewQuery->jointree->fromlist);
- if (!IsA(mvjtnode, RangeTblRef))
- return NIL;
-
- RangeTblEntry *rte = rt_fetch(1, viewQuery->rtable);
- if (rte->rtekind != RTE_RELATION)
- return NIL;
-
- char relkind = get_rel_relkind(rte->relid);
-
- /*
- * Allow foreign table here, however we don't know if the data is
- * up to date or not of the view.
- * But if users want to query matview instead of query foreign tables
- * outside CBDB, let them decide with aqumv_allow_foreign_table.
- */
- if (relkind != RELKIND_RELATION &&
- relkind != RELKIND_PARTITIONED_TABLE &&
- relkind != RELKIND_FOREIGN_TABLE)
- return NIL;
-
if (has_foreign)
- *has_foreign = relkind == RELKIND_FOREIGN_TABLE;
+ *has_foreign = false;
- /*
- * inherit tables are not supported.
- */
- bool can_be_partition = (relkind == RELKIND_PARTITIONED_TABLE) || get_rel_relispartition(rte->relid);
-
- if (!can_be_partition &&
- (has_superclass(rte->relid) || has_subclass(rte->relid)))
+ if (!extract_base_relids_from_jointree((Node *) viewQuery->jointree,
+ viewQuery->rtable,
+ &relids, has_foreign))
return NIL;
- relids = list_make1_oid(rte->relid);
+ if (relids == NIL)
+ return NIL;
Review Comment:
These 2 lines are unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Support matview status maintenance for multi-table JOIN queries [cloudberry]
Posted by "avamingli (via GitHub)" <gi...@apache.org>.
avamingli commented on PR #1594:
URL: https://github.com/apache/cloudberry/pull/1594#issuecomment-3984236393
This is the first step of https://github.com/apache/cloudberry/discussions/1348
> Our approach to this feature is structured in two primary phases:
> 1. Materialized View Maintenance:
> The foundational catalog design already incorporates the necessary structures for multi-relations. If we decide to do that, the upcoming work will focus on developing the maintenance mechanisms for these views.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Support matview status maintenance for multi-table JOIN queries [cloudberry]
Posted by "avamingli (via GitHub)" <gi...@apache.org>.
avamingli commented on code in PR #1594:
URL: https://github.com/apache/cloudberry/pull/1594#discussion_r2887328527
##########
src/backend/catalog/gp_matview_aux.c:
##########
@@ -109,43 +198,16 @@ GetViewBaseRelids(const Query *viewQuery, bool *has_foreign)
if (contain_mutable_functions((Node*)viewQuery))
return NIL;
- if (list_length(viewQuery->jointree->fromlist) != 1)
- return NIL;
-
- mvjtnode = (Node *) linitial(viewQuery->jointree->fromlist);
- if (!IsA(mvjtnode, RangeTblRef))
- return NIL;
-
- RangeTblEntry *rte = rt_fetch(1, viewQuery->rtable);
- if (rte->rtekind != RTE_RELATION)
- return NIL;
-
- char relkind = get_rel_relkind(rte->relid);
-
- /*
- * Allow foreign table here, however we don't know if the data is
- * up to date or not of the view.
- * But if users want to query matview instead of query foreign tables
- * outside CBDB, let them decide with aqumv_allow_foreign_table.
- */
- if (relkind != RELKIND_RELATION &&
- relkind != RELKIND_PARTITIONED_TABLE &&
- relkind != RELKIND_FOREIGN_TABLE)
- return NIL;
-
if (has_foreign)
- *has_foreign = relkind == RELKIND_FOREIGN_TABLE;
+ *has_foreign = false;
- /*
- * inherit tables are not supported.
- */
- bool can_be_partition = (relkind == RELKIND_PARTITIONED_TABLE) || get_rel_relispartition(rte->relid);
-
- if (!can_be_partition &&
- (has_superclass(rte->relid) || has_subclass(rte->relid)))
+ if (!extract_base_relids_from_jointree((Node *) viewQuery->jointree,
+ viewQuery->rtable,
+ &relids, has_foreign))
return NIL;
- relids = list_make1_oid(rte->relid);
+ if (relids == NIL)
+ return NIL;
Review Comment:
Good Catch!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Support matview status maintenance for multi-table JOIN queries [cloudberry]
Posted by "avamingli (via GitHub)" <gi...@apache.org>.
avamingli merged PR #1594:
URL: https://github.com/apache/cloudberry/pull/1594
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Support matview status maintenance for multi-table JOIN queries [cloudberry]
Posted by "zhangwenchao-123 (via GitHub)" <gi...@apache.org>.
zhangwenchao-123 commented on PR #1594:
URL: https://github.com/apache/cloudberry/pull/1594#issuecomment-3995944948
LGTM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org