You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "lss602726449 (via GitHub)" <gi...@apache.org> on 2026/10/08 09:34:07 UTC
[PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
lss602726449 opened a new pull request, #2094:
URL: https://github.com/apache/cloudberry/pull/2094
Cherry-picked two commits from the hashdata-lightning repo.
### Commits
- **Let ORCA plan an in-place UPDATE of append-optimized tables** (orig `befb03fea31`)
- **Fix AO row scan expanding unread columns (#965)** (orig `47a358712540`)
### Conflict resolution notes
- `src/test/regress/greenplum_schedule`: added only `orca_split_update` to the ORCA group. Dropped `orca_material_rescan` from the lightning hunk because its test files do not exist in this repo.
- Kept `contrib/pax_storage/src/test/vec/expected/partition_prune_optimizer.out` and `contrib/vectorization/src/test/regress/expected/partition_prune_aocs_optimizer.out` deleted (modify/delete conflicts — these files do not exist in cloudberry).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "lss602726449 (via GitHub)" <gi...@apache.org>.
lss602726449 commented on PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#issuecomment-6183981670
> Please add initial authors of the patches and the projects where these commits came from in commit messages.
initial authors is me
>I've noticed some UPDATEs now dispatch to all segments and use two-phase commit instead of single-segment execution. Could you benchmark this regression, especially on larger clusters?
Also please add targeted regression tests for partial attribute reads and materialization, including reading a materialized slot after the scan advances to another buffer or ends.
I have't test this. But I just enable update in orca, and you can use PG optimizer if you like. It's just optional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "Alena0704 (via GitHub)" <gi...@apache.org>.
Alena0704 commented on PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#issuecomment-6185707537
> > Please add initial authors of the patches and the projects where these commits came from in commit messages.
>
> initial authors is me
>
> > I've noticed some UPDATEs now dispatch to all segments and use two-phase commit instead of single-segment execution. Could you benchmark this regression, especially on larger clusters?
> > Also please add targeted regression tests for partial attribute reads and materialization, including reading a materialized slot after the scan advances to another buffer or ends.
>
> I have't test this. But I just enable update in orca, and you can use PG optimizer if you like. It's just optional.
> > Please add initial authors of the patches and the projects where these commits came from in commit messages.
>
> initial authors is me
>
> > I've noticed some UPDATEs now dispatch to all segments and use two-phase commit instead of single-segment execution. Could you benchmark this regression, especially on larger clusters?
> > Also please add targeted regression tests for partial attribute reads and materialization, including reading a materialized slot after the scan advances to another buffer or ends.
>
> I have't test this. But I just enable update in orca, and you can use PG optimizer if you like. It's just optional.
Sorry. I couldn't find the initial commit and when you linked to the commit of another author in the PR I thought it was another author. As for test, I have noticed that the query plan in one of the tests became worse and because of this suggested to run benchmark.
Anyway, I think changes look fine in the commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "lss602726449 (via GitHub)" <gi...@apache.org>.
lss602726449 commented on code in PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#discussion_r4217436580
##########
src/backend/access/appendonly/memtuple_slot.c:
##########
@@ -0,0 +1,298 @@
+/*-------------------------------------------------------------------------
+ *
+ * memtuple_slot.c
+ * TupleTableSlot implementation for MemTuples.
+ *
+ * Append-only row tables store tuples in the MemTuple format. Unlike heap
+ * and minimal tuples a MemTuple is not self describing: a MemTupleBinding
+ * is required to locate its attributes. That binding lives here in the
+ * slot, which is what lets the slot hand out attributes on demand instead
+ * of forcing the scan to expand every column of every tuple up front.
+ *
+ * Attribute offsets are precomputed in the binding, so each attribute is
+ * fetched independently in constant time. tts_nvalid therefore doubles as
+ * the resume point for getsomeattrs, and no separate deforming cursor (the
+ * "off" field heap and minimal tuple slots carry) is needed.
+ *
+ * Portions Copyright (c) 2023-2025, HashData Technology Limited.
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/appendonly/memtuple_slot.c
+ *-------------------------------------------------------------------------
+ */
Review Comment:
OK， I'll use the standard ASF license header.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on code in PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#discussion_r4217353510
##########
src/backend/access/appendonly/memtuple_slot.c:
##########
@@ -0,0 +1,298 @@
+/*-------------------------------------------------------------------------
+ *
+ * memtuple_slot.c
+ * TupleTableSlot implementation for MemTuples.
+ *
+ * Append-only row tables store tuples in the MemTuple format. Unlike heap
+ * and minimal tuples a MemTuple is not self describing: a MemTupleBinding
+ * is required to locate its attributes. That binding lives here in the
+ * slot, which is what lets the slot hand out attributes on demand instead
+ * of forcing the scan to expand every column of every tuple up front.
+ *
+ * Attribute offsets are precomputed in the binding, so each attribute is
+ * fetched independently in constant time. tts_nvalid therefore doubles as
+ * the resume point for getsomeattrs, and no separate deforming cursor (the
+ * "off" field heap and minimal tuple slots carry) is needed.
+ *
+ * Portions Copyright (c) 2023-2025, HashData Technology Limited.
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/appendonly/memtuple_slot.c
+ *-------------------------------------------------------------------------
+ */
Review Comment:
We can adopt the standard ASF license header here, like:
https://github.com/apache/cloudberry/blob/dce6f6c0b83edbbffeab768aaeb0d308790e25c1/contrib/pax_storage/src/cpp/pax_gbench.cc#L1-L26
Ps, is it an originally created file, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "lss602726449 (via GitHub)" <gi...@apache.org>.
lss602726449 commented on code in PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#discussion_r4217419884
##########
src/backend/access/appendonly/memtuple_slot.c:
##########
@@ -0,0 +1,298 @@
+/*-------------------------------------------------------------------------
+ *
+ * memtuple_slot.c
+ * TupleTableSlot implementation for MemTuples.
+ *
+ * Append-only row tables store tuples in the MemTuple format. Unlike heap
+ * and minimal tuples a MemTuple is not self describing: a MemTupleBinding
+ * is required to locate its attributes. That binding lives here in the
+ * slot, which is what lets the slot hand out attributes on demand instead
+ * of forcing the scan to expand every column of every tuple up front.
+ *
+ * Attribute offsets are precomputed in the binding, so each attribute is
+ * fetched independently in constant time. tts_nvalid therefore doubles as
+ * the resume point for getsomeattrs, and no separate deforming cursor (the
+ * "off" field heap and minimal tuple slots carry) is needed.
+ *
+ * Portions Copyright (c) 2023-2025, HashData Technology Limited.
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/appendonly/memtuple_slot.c
+ *-------------------------------------------------------------------------
+ */
Review Comment:
Good catch — yes, this is an originally created file. Replaced the header with the standard ASF license block (dropped the HashData/PostgreSQL copyright lines since the file is original). Done in the latest push.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on code in PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#discussion_r4217353510
##########
src/backend/access/appendonly/memtuple_slot.c:
##########
@@ -0,0 +1,298 @@
+/*-------------------------------------------------------------------------
+ *
+ * memtuple_slot.c
+ * TupleTableSlot implementation for MemTuples.
+ *
+ * Append-only row tables store tuples in the MemTuple format. Unlike heap
+ * and minimal tuples a MemTuple is not self describing: a MemTupleBinding
+ * is required to locate its attributes. That binding lives here in the
+ * slot, which is what lets the slot hand out attributes on demand instead
+ * of forcing the scan to expand every column of every tuple up front.
+ *
+ * Attribute offsets are precomputed in the binding, so each attribute is
+ * fetched independently in constant time. tts_nvalid therefore doubles as
+ * the resume point for getsomeattrs, and no separate deforming cursor (the
+ * "off" field heap and minimal tuple slots carry) is needed.
+ *
+ * Portions Copyright (c) 2023-2025, HashData Technology Limited.
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/appendonly/memtuple_slot.c
+ *-------------------------------------------------------------------------
+ */
Review Comment:
We can adopt the standard ASF license header here, like:
https://github.com/apache/cloudberry/blob/dce6f6c0b83edbbffeab768aaeb0d308790e25c1/contrib/pax_storage/src/cpp/pax_gbench.cc#L1-L26
Ps, is is an original created file, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "Alena0704 (via GitHub)" <gi...@apache.org>.
Alena0704 commented on PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#issuecomment-6085667231
Please add initial authors of the patches and the projects where these commits came from in commit messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "Alena0704 (via GitHub)" <gi...@apache.org>.
Alena0704 commented on PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#issuecomment-6097151663
I've noticed some UPDATEs now dispatch to all segments and use two-phase commit instead of single-segment execution. Could you benchmark this regression, especially on larger clusters?
Also please add targeted regression tests for partial attribute reads and materialization, including reading a materialized slot after the scan advances to another buffer or ends.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on code in PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094#discussion_r4217353510
##########
src/backend/access/appendonly/memtuple_slot.c:
##########
@@ -0,0 +1,298 @@
+/*-------------------------------------------------------------------------
+ *
+ * memtuple_slot.c
+ * TupleTableSlot implementation for MemTuples.
+ *
+ * Append-only row tables store tuples in the MemTuple format. Unlike heap
+ * and minimal tuples a MemTuple is not self describing: a MemTupleBinding
+ * is required to locate its attributes. That binding lives here in the
+ * slot, which is what lets the slot hand out attributes on demand instead
+ * of forcing the scan to expand every column of every tuple up front.
+ *
+ * Attribute offsets are precomputed in the binding, so each attribute is
+ * fetched independently in constant time. tts_nvalid therefore doubles as
+ * the resume point for getsomeattrs, and no separate deforming cursor (the
+ * "off" field heap and minimal tuple slots carry) is needed.
+ *
+ * Portions Copyright (c) 2023-2025, HashData Technology Limited.
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/appendonly/memtuple_slot.c
+ *-------------------------------------------------------------------------
+ */
Review Comment:
We can adopt the standard ASF license header here, like:
https://github.com/apache/cloudberry/blob/dce6f6c0b83edbbffeab768aaeb0d308790e25c1/contrib/pax_storage/src/cpp/pax_gbench.cc#L1-L26
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Let ORCA plan in-place UPDATE of AO tables; fix AO row scan expanding unread columns [cloudberry]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe merged PR #2094:
URL: https://github.com/apache/cloudberry/pull/2094
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org