You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "jiaqizho (via GitHub)" <gi...@apache.org> on 2025/06/25 03:25:26 UTC
[PR] ORCA: Support pushdown partial aggregate below join [cloudberry]
jiaqizho opened a new pull request, #1187:
URL: https://github.com/apache/cloudberry/pull/1187
<!-- Thank you for your contribution to Apache Cloudberry (Incubating)! -->
Fixes #ISSUE_Number
### What does this PR do?
<!-- Brief overview of the changes, including any major features or fixes -->
### Type of Change
- [ ] Bug fix (non-breaking change)
- [ ] New feature (non-breaking change)
- [ ] Breaking change (fix or feature with breaking changes)
- [ ] Documentation update
### Breaking Changes
<!-- Remove if not applicable. If yes, explain impact and migration path -->
### Test Plan
<!-- How did you test these changes? -->
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Passed `make installcheck`
- [ ] Passed `make -C src/test installcheck-cbdb-parallel`
### Impact
<!-- Remove sections that don't apply -->
**Performance:**
<!-- Any performance implications? -->
**User-facing changes:**
<!-- Any changes visible to users? -->
**Dependencies:**
<!-- New dependencies or version changes? -->
### Checklist
- [ ] Followed [contribution guide](https://cloudberry.apache.org/contribute/code)
- [ ] Added/updated documentation
- [ ] Reviewed code for security implications
- [ ] Requested review from [cloudberry committers](https://github.com/orgs/apache/teams/cloudberry-committers)
### Additional Context
<!-- Any other information that would help reviewers? Remove if none -->
### CI Skip Instructions
<!--
To skip CI builds, add the appropriate CI skip identifier to your PR title.
The identifier must:
- Be in square brackets []
- Include the word "ci" and either "skip" or "no"
- Only use for documentation-only changes or when absolutely necessary
-->
---
<!-- Join our community:
- Mailing list: [dev@cloudberry.apache.org](https://lists.apache.org/list.html?dev@cloudberry.apache.org) (subscribe: dev-subscribe@cloudberry.apache.org)
- Discussions: https://github.com/apache/cloudberry/discussions -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] ORCA: Support pushdown partial aggregate below join [cloudberry]
Posted by "jiaqizho (via GitHub)" <gi...@apache.org>.
jiaqizho merged PR #1187:
URL: https://github.com/apache/cloudberry/pull/1187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] ORCA: Support pushdown partial aggregate below join [cloudberry]
Posted by "jiaqizho (via GitHub)" <gi...@apache.org>.
jiaqizho commented on PR #1187:
URL: https://github.com/apache/cloudberry/pull/1187#issuecomment-3004213098
> Just tried:
>
> ```
> -- 状态转换函数：处理单个输入值并更新状态
> CREATE OR REPLACE FUNCTION avg_trans_fn(
> state numeric[], -- 当前状态：[sum, count]
> next_val numeric -- 新输入值
> ) RETURNS numeric[] AS $$
> BEGIN
> IF next_val IS NOT NULL THEN
> state[1] := COALESCE(state[1], 0) + next_val;
> state[2] := COALESCE(state[2], 0) + 1;
> END IF;
> RETURN state;
> END;
> $$ LANGUAGE plpgsql;
>
> -- 合并函数：合并两个部分聚合状态
> CREATE OR REPLACE FUNCTION avg_combine_fn(
> state1 numeric[],
> state2 numeric[]
> ) RETURNS numeric[] AS $$
> BEGIN
> IF state1 IS NULL THEN
> RETURN state2;
> ELSIF state2 IS NULL THEN
> RETURN state1;
> ELSE
> RETURN ARRAY[state1[1] + state2[1], state1[2] + state2[2]];
> END IF;
> END;
> $$ LANGUAGE plpgsql;
>
> -- 最终函数：从状态计算最终结果
> CREATE OR REPLACE FUNCTION avg_final_fn(
> state numeric[]
> ) RETURNS numeric AS $$
> BEGIN
> IF state[2] = 0 THEN
> RETURN NULL;
> ELSE
> RETURN state[1] / state[2];
> END IF;
> END;
> $$ LANGUAGE plpgsql;
>
> CREATE AGGREGATE my_avg(numeric) (
> sfunc = avg_trans_fn, -- 状态转换函数
> stype = numeric[], -- 状态类型
> combinefunc = avg_combine_fn, -- 合并函数
> finalfunc = avg_final_fn, -- 最终计算函数
> initcond = '{0,0}' -- 初始状态
> );
>
> CREATE TABLE tbl1(v1 int, v2 numeric);
> CREATE TABLE tbl2(v1 int, v2 numeric);
> ```
>
> The agg pushdown doesn't work in ORCA but works in postgres Postgres query optimizer. 
Without any data in the table `tbl1` and `tbl2`. ORCA choose the one-stage agg, but we do have the pushdown path.
we can use the `optimizer_force_multistage_agg` to force ORCA choose the multi-stage path(But won't create the no-pushdown path, because the enforce of distribution).
also use the `my_avg`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] ORCA: Support pushdown partial aggregate below join [cloudberry]
Posted by "my-ship-it (via GitHub)" <gi...@apache.org>.
my-ship-it commented on PR #1187:
URL: https://github.com/apache/cloudberry/pull/1187#issuecomment-3003796481
Just tried:
```
-- 状态转换函数：处理单个输入值并更新状态
CREATE OR REPLACE FUNCTION avg_trans_fn(
state numeric[], -- 当前状态：[sum, count]
next_val numeric -- 新输入值
) RETURNS numeric[] AS $$
BEGIN
IF next_val IS NOT NULL THEN
state[1] := COALESCE(state[1], 0) + next_val;
state[2] := COALESCE(state[2], 0) + 1;
END IF;
RETURN state;
END;
$$ LANGUAGE plpgsql;
-- 合并函数：合并两个部分聚合状态
CREATE OR REPLACE FUNCTION avg_combine_fn(
state1 numeric[],
state2 numeric[]
) RETURNS numeric[] AS $$
BEGIN
IF state1 IS NULL THEN
RETURN state2;
ELSIF state2 IS NULL THEN
RETURN state1;
ELSE
RETURN ARRAY[state1[1] + state2[1], state1[2] + state2[2]];
END IF;
END;
$$ LANGUAGE plpgsql;
-- 最终函数：从状态计算最终结果
CREATE OR REPLACE FUNCTION avg_final_fn(
state numeric[]
) RETURNS numeric AS $$
BEGIN
IF state[2] = 0 THEN
RETURN NULL;
ELSE
RETURN state[1] / state[2];
END IF;
END;
$$ LANGUAGE plpgsql;
CREATE AGGREGATE my_avg(numeric) (
sfunc = avg_trans_fn, -- 状态转换函数
stype = numeric[], -- 状态类型
combinefunc = avg_combine_fn, -- 合并函数
finalfunc = avg_final_fn, -- 最终计算函数
initcond = '{0,0}' -- 初始状态
);
CREATE TABLE tbl1(v1 int, v2 numeric);
CREATE TABLE tbl2(v1 int, v2 numeric);
```
The agg pushdown doesn't work in ORCA but works in postgres Postgres query optimizer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org