You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "MisterRaindrop (via GitHub)" <gi...@apache.org> on 2026/02/10 10:56:42 UTC
[PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
MisterRaindrop opened a new pull request, #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61
<!--Thank you for contributing! -->
<!--In case of an existing issue or discussions, please reference it-->
https://github.com/apache/cloudberry-pxf/issues/58
<!--Remove this section if no corresponding issue.-->
---
## Change logs
Currently, parallel FDW is supported. This implementation depends on the kernel's commit.
https://github.com/apache/cloudberry/pull/1571
## Contributor's checklist
Here are some reminders before you submit your pull request:
* Make sure that your Pull Request has a clear title and commit message. You can take the [Git commit template](https://github.com/apache/cloudberry/blob/main/.gitmessage) as a reference.
* Learn the [code contribution](https://cloudberry.apache.org/contribute/code) and [doc contribution](https://cloudberry.apache.org/contribute/doc) guides for better collaboration.
* Make sure that CICD workflow is successful.
* List your communications in the [GitHub Issues](https://github.com/apache/cloudberry-pxf/issues) or [Discussions](https://github.com/apache/cloudberry/discussions) (if has or needed).
* Feel free to ask for the [cloudberry committers](https://github.com/orgs/apache/teams/cloudberry-committers) or other people to help review and approve.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "ostinru (via GitHub)" <gi...@apache.org>.
ostinru commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r2795094954
##########
server/pxf-service/src/main/java/org/apache/cloudberry/pxf/service/controller/ReadServiceImpl.java:
##########
@@ -78,11 +82,26 @@ private OperationResult writeStream(RequestContext context, OutputStream outputS
OperationStats queryStats = new OperationStats(OperationStats.Operation.READ, metricsReporter, context);
OperationResult queryResult = new OperationResult();
- // dataStream (and outputStream as the result) will close automatically at the end of the try block
+ // dataStream (and outputStream as the result) will close automatically at the
+ // end of the try block
CountingOutputStream countingOutputStream = new CountingOutputStream(outputStream);
String sourceName = null;
try {
- List<Fragment> fragments = fragmenterService.getFragmentsForSegment(context);
+ List<Fragment> fragments;
+
+ // Check if a specific fragment index was requested (parallel execution mode)
+ if (context.hasSpecificFragment()) {
+ // Parallel mode: only process the specified fragment
+ Fragment specificFragment = fragmenterService.getFragmentByIndex(
+ context, context.getSpecificFragmentIndex());
+ fragments = java.util.Collections.singletonList(specificFragment);
Review Comment:
NIT: `import java.util.Collections`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r4438587855
##########
automation/src/test/java/org/apache/cloudberry/pxf/automation/features/parallel/ParallelScanTest.java:
##########
Review Comment:
Since this is a new created file, we can add the standard ASF license header.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub) via commits" <co...@cloudberry.apache.org>.
MisterRaindrop commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r4434455837
##########
automation/sqlrepo/features/parallel/explain/expected/query01.ans:
##########
@@ -0,0 +1,32 @@
+-- @description query01 for PXF parallel scan EXPLAIN - Gather node present when parallel enabled
+-- start_matchsubs
+--
+-- m/Workers Planned: \d+/
+-- s/Workers Planned: \d+/Workers Planned: N/
+--
+-- m/cost=\d+\.\d+\.\.\d+\.\d+/
+-- s/cost=\d+\.\d+\.\.\d+\.\d+/cost=XXX/
+--
+-- m/rows=\d+/
+-- s/rows=\d+/rows=NNN/
+--
+-- m/width=\d+/
+-- s/width=\d+/width=NN/
+--
+-- end_matchsubs
+SET optimizer = off;
+SET
+SET enable_parallel = true;
+SET
+SET max_parallel_workers_per_gather = 4;
+SET
+EXPLAIN SELECT count(*) FROM pxf_parallel_enabled;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------
+ Gather Motion 1:1 (slice1; segments: 1) (cost=XXX rows=NNN width=NN)
Review Comment:
A correction to my reply above: testParallelExplain passing in CI didn't show anything. `atmsort.pl` sends statements that look like `EXPLAIN ... SELECT` to `explain.pl`, which isn't installed in the CI image, so the actual and expected output were both empty. 8757e6eb returns the plan from a function as ordinary query rows, so it is really compared now. The expected plan is the same as before. I checked that a wrong expected plan fails, with and without `explain.pl`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "my-ship-it (via GitHub)" <gi...@apache.org>.
my-ship-it commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r2876242781
##########
automation/sqlrepo/features/parallel/explain/expected/query01.ans:
##########
@@ -0,0 +1,32 @@
+-- @description query01 for PXF parallel scan EXPLAIN - Gather node present when parallel enabled
+-- start_matchsubs
+--
+-- m/Workers Planned: \d+/
+-- s/Workers Planned: \d+/Workers Planned: N/
+--
+-- m/cost=\d+\.\d+\.\.\d+\.\d+/
+-- s/cost=\d+\.\d+\.\.\d+\.\d+/cost=XXX/
+--
+-- m/rows=\d+/
+-- s/rows=\d+/rows=NNN/
+--
+-- m/width=\d+/
+-- s/width=\d+/width=NN/
+--
+-- end_matchsubs
+SET optimizer = off;
+SET
+SET enable_parallel = true;
+SET
+SET max_parallel_workers_per_gather = 4;
+SET
+EXPLAIN SELECT count(*) FROM pxf_parallel_enabled;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------
+ Gather Motion 1:1 (slice1; segments: 1) (cost=XXX rows=NNN width=NN)
Review Comment:
The plan seems not correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r4438602222
##########
pom.xml:
##########
@@ -364,6 +364,20 @@
<exclude>automation/README.Linux.md</exclude>
<exclude>automation/sqlrepo/features/reader/multi_block/expected/**</exclude>
<exclude>automation/sqlrepo/features/reader/multi_block/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/append/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/append/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/correctness/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/correctness/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/explain/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/explain/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/option_precedence/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/option_precedence/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/reject_limit/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/reject_limit/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/rescan/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/rescan/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/single_fragment/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/single_fragment/sql/**</exclude>
Review Comment:
For these new files created by the Cloudberry community, we can add a new section to explain how they were introduced by Cloudberry, instead of legacy Greenplum files.
```
<!--New files introduced by Cloudberry>
<exclude>automation/sqlrepo/features/parallel/append/expected/**</exclude>
<exclude>automation/sqlrepo/features/parallel/append/sql/**</exclude>
...
<!-- End -->
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "my-ship-it (via GitHub) via commits" <co...@cloudberry.apache.org>.
my-ship-it commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r4535365897
##########
fdw/pxf_bridge.c:
##########
@@ -71,6 +72,22 @@ PxfBridgeImportStart(PxfFdwScanState *pxfsstate)
pxfsstate->retrieved_attrs,
pxfsstate->projectionInfo);
+ /*
+ * A gang-parallel scan reports a virtual segment, so that the server's
+ * round-robin fragment split gives each gang worker a disjoint subset of
+ * the fragments of its physical segment.
+ */
+ if (pxfsstate->gang_parallel)
+ {
+ char seg_id_str[12];
+ char seg_count_str[12];
+
+ pg_ltoa(pxfsstate->virtual_seg_id, seg_id_str);
+ pg_ltoa(pxfsstate->virtual_seg_count, seg_count_str);
+ churl_headers_override(pxfsstate->churl_headers, "X-GP-SEGMENT-ID", seg_id_str);
Review Comment:
Thanks for the work on this! One thing I noticed: since X-GP-SEGMENT-ID is overridden here, `pxf_stat_activity` / `pxf_cancel_backend` / `pxf_interrupt_backend` may miss most of the parallel requests. Those functions run on each segment and filter the registry by `GpIdentity.segindex`, so requests with a virtual id >= N wouldn't match. E.g. with 3 segs x 4 workers, only 3 of the 12 requests would be visible/cancellable. Logs/metrics would also show segment ids that don't exist.
Would it make sense to keep the real segment id and pass the worker id/count in separate headers, letting the server compute the split? If that's better left for a follow-up, maybe a note in the docs for now. What do you think?
##########
fdw/pxf_option.c:
##########
@@ -454,6 +460,19 @@ PxfGetOptions(Oid foreigntableid)
opt->log_errors = defGetBoolean(def);
else if (strcmp(def->defname, FDW_OPTION_DISABLE_PPD) == 0)
opt->disable_ppd = defGetBoolean(def);
+ else if (strcmp(def->defname, FDW_OPTION_ENABLE_PARALLEL) == 0)
+ {
+ /*
+ * Valid at both table and server level. Options are listed with
+ * the table first, so keep the first value to let the table
+ * override the server.
+ */
+ if (!enable_parallel_set)
+ {
+ opt->enable_parallel = defGetBoolean(def);
Review Comment:
Small one: it looks like `enable_parallel` isn't validated in `pxf_fdw_validator`, so something like `OPTIONS (enable_parallel 'on2')` is accepted at CREATE time, and then every query on the table fails here with "requires a Boolean value" (even with parallel off in the session). If it's set on a server, all tables under it would be affected. Could we add a `defGetBoolean()` check in the validator, like `log_errors` / `disable_ppd`? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-6490174514
> > The current code is not yet ready for the review stage.
>
> @MisterRaindrop , is it ready for review?
it is ready for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-6573107976
Thanks for the update! Below are a re-review and a cluster benchmark of the new head (7f708c1f), both prepared with Claude Code.
The results look good: COUNT/SUM still get ~1.8x (CSV) / ~1.7x (Parquet) at 4 workers, the reject_limit issue is fixed, and CSV GROUP BY is no longer slower than main. The one trade-off worth noting is that Parquet GROUP BY no longer uses a parallel scan (1.70x → ~1.0x); details and a few minor suggestions are in the review below.
Thanks for the great work!
<details>
<summary><b>Re-review (review-comment-round2.md)</b></summary>
### Re-review of the new head (7f708c1f)
Thanks for the quick turnaround. I re-reviewed fa855b32..7f708c1f and re-ran the benchmark from the last round on the same EC2 setup, against both `main` and the previous head. Details are in the attached test report.
**Summary:** every item from the last review is addressed, and I found nothing blocking. COUNT/SUM speedups are unchanged, the reject-limit bug is fixed, and CSV GROUP BY is no longer slower than `main`. The one real trade-off is that **Parquet GROUP BY loses its speedup** (1.70x → 0.98x at 4 workers). I'd like the maintainers to agree to that explicitly before merge.
### Last round's items
| Item | Status |
|---|---|
| reject_limit enforced per worker | **Fixed.** No partial path when `reject_limit` is set. Checked at p4/p8: the rows limit and percent limit both error like `main`, and `LOG ERRORS` logs exactly 48 rows. |
| Partial-path row estimate too low (CSV GROUP BY slower) | **Fixed.** I checked `adjust_costs_for_mpp_foreign_scan()` in the kernel and agree with your analysis. CSV GROUP BY at p2 went from 0.92x to 0.98x (now the same plan as `main`). |
| Small tables always go parallel | **Unchanged, documented.** A 100-row table costs about +3 ms with parallel on. Fine given the new docs section. |
| Duplicated `*Virtual` functions | **Fixed.** The header override in `PxfBridgeImportStart` is much cleaner. |
| Rescan | **Fixed.** The four rescan queries match non-parallel results. |
| Error callback rewrite | **Fixed.** Restored to `main`. |
| Nits / docs | **Fixed.** |
Good catch on the EXPLAIN tests comparing nothing; the plpgsql wrapper is a nice fix.
### Remaining points
1. **Parquet GROUP BY regression vs the previous head (needs a decision).** With the corrected estimate, the planner no longer chooses a parallel scan under any GROUP BY. That is the right result for CSV, where the old plan spilled to a Sort, but on Parquet the old plan was 1.70x faster (39.7 s → 23.4 s on 10 GB). I think shipping the conservative version is reasonable, since COUNT/SUM/filters still get about 1.7–1.8x. Could you add one line to the Parallel Scan docs saying aggregates with GROUP BY usually won't use a parallel scan? A follow-up issue to revisit GROUP BY costing would also help.
2. **`EndForeignScan` now calls `churl_cleanup` for every scan,** not only parallel ones. I measured this with interleaved base/new runs. `LIMIT 10` is about +2.5 ms (59 → 61.5 ms). `LIMIT 100000`, full scans and cancels show no change. The PXF "Remote connection closed by the client" WARN count is identical. No action needed; I'd mention it in the commit message since it changes behaviour on the non-parallel path.
3. **`SegmentRows()` mirrors a kernel heuristic.** If the kernel ever applies the `numsegments * 10` factor to partial paths too, the rows would be scaled twice. The comment covers it; maybe also point to `adjust_costs_for_mpp_foreign_scan` by name so it's easy to find.
4. **Nit:** the expected output for `parallel/reject_limit` still contains a CI machine path (`/pr61ci/parallel_bad_rows`) in a CONTEXT line. Matchsubs hides it, but regenerating it would be cleaner.
5. **Commit history:** please squash or reword 77ff52c9 and 0bbe2608 before merge, as you suggested.
From my side this is ready to merge once point 1 is agreed and points 4–5 are cleaned up.
</details>
<details>
<summary><b>Benchmark report (test-report-round2.md)</b></summary>
# PR #61 benchmark, round 2: `main` vs previous head vs new head
## Setup
- **Machine:** EC2 m6i.4xlarge (16 vCPU Xeon 8375C, 61 GiB), Rocky 9, 300 GB gp3. Same box type as round 1.
- **Cloudberry:** apache/cloudberry cd260a80 (same as round 1), release build (-O3, no cassert). Demo cluster with 3 primaries and no mirrors.
- **PXF:** server from `main` (40961b78), `-Xmx4g`, local `file` server. Only `pxf_fdw.so` was swapped between builds, each followed by `gpstop -ar`:
- **base:** 40961b78
- **old:** fa855b32
- **new:** 7f708c1f
- **Method:** `optimizer=off`, 1 warm-up and 5 timed runs, median reported. `pN` = `enable_parallel=on, max_parallel_workers_per_gather=N`.
- **Order:** base → old → new → base again. The final base rerun was within 0.7% (CSV) and 1.3% (Parquet) of the first one.
- **Correctness:** query results were identical across every build and setting (md5 of the result rows).
## Datasets
- `csvX_N`: CSV data, total size X, split into N files.
- `pq10g_24`: 10 GB Parquet in 24 files.
The queries are `count(*)`, `sum(...)`, and `GROUP BY grp` with `count` and `sum`.
## Speedup vs `main` at 4 workers, previous head vs new head
| Dataset | count (old → new) | sum (old → new) | GROUP BY (old → new) |
|---|---|---|---|
| csv100m_24 | 1.33 → 1.33 | 1.34 → 1.35 | 1.03 → 1.00 |
| csv1g_24 | 1.71 → 1.68 | 1.56 → 1.60 | 1.00 → 0.96 |
| csv10g_24 | 1.81 → 1.83 | 1.85 → 1.85 | 1.01 → 0.98 |
| csv10g_3 | 1.81 → 1.82 | 1.84 → 1.86 | 1.01 → 0.98 |
| csv10g_12 | 1.81 → 1.83 | 1.83 → 1.86 | 1.01 → 0.98 |
| csv10g_48 | 1.81 → 1.81 | 1.82 → 1.85 | 1.00 → 0.98 |
| pq10g_24 | 1.69 → 1.71 | 1.67 → 1.67 | **1.70 → 0.98** |
At 2 and 8 workers:
| Query | p2 (old → new) | p8 (old → new) |
|---|---|---|
| csv10g_24 count | 1.56 → 1.62 | 1.79 → 1.79 |
| csv10g_24 GROUP BY | **0.92 → 0.98** | 1.07 → 0.98 |
| pq10g_24 GROUP BY | 1.48 → 0.99 | 1.79 → 0.99 |
## Absolute times, median ms (new head)
| Dataset | Query | base | new, no parallel | new p2 | new p4 | new p8 | old p4 |
|---|---|---|---|---|---|---|---|
| csv10g_24 | count | 17,950 | 18,935 | 11,091 | 9,805 | 10,012 | 9,905 |
| csv10g_24 | sum | 19,013 | 19,365 | 11,930 | 10,260 | 10,369 | 10,270 |
| csv10g_24 | groupby | 20,965 | 21,475 | 21,288 | 21,291 | 21,308 | 20,692 |
| pq10g_24 | count | 39,946 | 40,470 | 28,194 | 23,408 | 23,226 | 23,586 |
| pq10g_24 | sum | 39,918 | 40,209 | 29,560 | 23,911 | 23,656 | 23,910 |
| pq10g_24 | groupby | 39,725 | 40,211 | 39,931 | 40,401 | 39,935 | 23,407 |
| csv1g_24 | count | 2,057 | 2,006 | 1,978 | 1,223 | 1,093 | 1,206 |
| csv100m_24 | count | 249 | 265 | 218 | 187 | 189 | 188 |
The new head looks about 3–5% slower than base without parallel on CSV, but **that did not reproduce**. An interleaved A/B run (base, new, base, new) on `count(*)` over csv10g_24 gave 17,973 / 17,935 ms on base and 18,011 / 17,917 ms on new. The gap in the table comes from when the runs happened, not from the PR.
## GROUP BY plans (pq10g_24, p4)
- **base:** Gather 3:1 → Sort → Finalize HashAgg → Redistribute 3:3 → Streaming Partial HashAgg → Foreign Scan
- **old head:** Gather 12:1 → GroupAggregate → Sort → Redistribute 12:12 → **Parallel Foreign Scan** (rows=250)
- **new head:** same as base (Foreign Scan cost 50000..50300). GROUP BY no longer uses a parallel scan.
- **new head, COUNT/SUM:** Finalize Agg → Gather 12:1 → Partial Agg → Parallel Foreign Scan (rows=7500)
## CPU (average %, out of 1600)
| Query | base | new p4 | old p4 |
|---|---|---|---|
| csv10g_24 sum | java 111 / postgres 287 | java 382 / postgres 986 | java 397 / postgres 991 |
| pq10g_24 count | java 298 / postgres 146 | java 923 / postgres 460 | java 917 / postgres 446 |
The CPU profile is the same as with the previous head: CSV is bound on the postgres side, Parquet on the Java side. 8 workers is no faster than 4.
## Functional checks
| Check | Result |
|---|---|
| R1 reject_limit | **Fixed.** At p4/p8 the plan is a plain Foreign Scan. A rows limit of 10 and a percent limit of 2% both error like `main` (the previous head returned 23,952 and 2,328 rows instead). With `LOG ERRORS` and limit 100: 23,952 rows and 48 errors logged in every configuration. |
| R2 small tables | **Still parallel, as documented.** 100-row table: 25.0 ms parallel vs 21.8 ms not. 1 MB table: 29.8 vs 27.3 ms. |
| R3 non-parallel plans | **No shape change.** With `optimizer=off`, each PXF Foreign Scan's cost goes up by 300; nothing else changes. With ORCA, plans are byte-identical to `main`. |
| R4 end-of-scan cleanup | `LIMIT 10`: about +2.5 ms (59.3/59.0 → 61.4/61.9 ms). `LIMIT 100000` and full scans: no change. Same PXF WARN count (126 "Remote connection closed by the client") on both builds. Cancels with `statement_timeout` (non-parallel, p4 CSV, p8 Parquet) were clean: no leftover backends, and the next query worked. No PANIC or FATAL. |
| R5 rescan | All four rescan queries match the non-parallel results. |
| R6 UNION ALL / partitions / hash join / skew | Results match non-parallel. Partitioned tables run their children as non-parallel scans under Parallel Append. |
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "ostinru (via GitHub)" <gi...@apache.org>.
ostinru commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r2795154622
##########
fdw/pxf_bridge.h:
##########
@@ -30,10 +30,35 @@
#include "nodes/execnodes.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
+#include "storage/spin.h"
#define PXF_SEGMENT_ID GpIdentity.segindex
#define PXF_SEGMENT_COUNT getgpsegmentCount()
+/*
+ * Fragment metadata for parallel execution.
+ * Stored in local memory, not in shared memory.
+ */
+typedef struct PxfFragmentData
+{
+ char *source_name; /* fragment source name (e.g. file path) */
+ int index; /* fragment index */
+ char *metadata; /* fragment metadata (base64 encoded) */
+ char *profile; /* optional profile override */
+} PxfFragmentData;
+
+/*
+ * Shared state for parallel foreign scan.
+ * This structure is stored in DSM (dynamic shared memory).
+ */
+typedef struct PxfParallelScanState
+{
+ slock_t mutex; /* mutex for accessing shared state */
+ int total_fragments; /* total number of fragments */
+ int next_fragment; /* next fragment index to be assigned */
+ bool finished; /* true if all fragments have been processed */
Review Comment:
> true if all fragments have been processed... or cancelled?
And. What is a purpose of write-only variable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "my-ship-it (via GitHub) via commits" <co...@cloudberry.apache.org>.
my-ship-it commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r4535365905
##########
fdw/pxf_option.c:
##########
@@ -454,6 +460,19 @@ PxfGetOptions(Oid foreigntableid)
opt->log_errors = defGetBoolean(def);
else if (strcmp(def->defname, FDW_OPTION_DISABLE_PPD) == 0)
opt->disable_ppd = defGetBoolean(def);
+ else if (strcmp(def->defname, FDW_OPTION_ENABLE_PARALLEL) == 0)
+ {
+ /*
+ * Valid at both table and server level. Options are listed with
+ * the table first, so keep the first value to let the table
+ * override the server.
+ */
+ if (!enable_parallel_set)
+ {
+ opt->enable_parallel = defGetBoolean(def);
Review Comment:
`enable_parallel` isn't checked in `pxf_fdw_validator`, so `OPTIONS (enable_parallel 'on2')` is accepted at CREATE time, and then every query on the table fails here with "requires a Boolean value" (even with parallel off in the session). Set on a server, it breaks every table under it. Needs a `defGetBoolean()` in the validator like `log_errors` / `disable_ppd`.
##########
fdw/pxf_bridge.c:
##########
@@ -71,6 +72,22 @@ PxfBridgeImportStart(PxfFdwScanState *pxfsstate)
pxfsstate->retrieved_attrs,
pxfsstate->projectionInfo);
+ /*
+ * A gang-parallel scan reports a virtual segment, so that the server's
+ * round-robin fragment split gives each gang worker a disjoint subset of
+ * the fragments of its physical segment.
+ */
+ if (pxfsstate->gang_parallel)
+ {
+ char seg_id_str[12];
+ char seg_count_str[12];
+
+ pg_ltoa(pxfsstate->virtual_seg_id, seg_id_str);
+ pg_ltoa(pxfsstate->virtual_seg_count, seg_count_str);
+ churl_headers_override(pxfsstate->churl_headers, "X-GP-SEGMENT-ID", seg_id_str);
Review Comment:
Overriding X-GP-SEGMENT-ID breaks `pxf_stat_activity` / `pxf_cancel_backend` / `pxf_interrupt_backend`. They run on every segment and filter the registry by `GpIdentity.segindex`, so requests with a virtual id >= N never match. With 3 segs x 4 workers, only 3 of the 12 requests show up or can be cancelled. Logs/metrics also end up with segment ids that don't exist.
Could we keep the real segment id and pass the worker id/count in separate headers, letting the server compute the split? If not in this PR, at least document it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r2876221867
##########
fdw/pxf_bridge.h:
##########
@@ -30,10 +30,35 @@
#include "nodes/execnodes.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
+#include "storage/spin.h"
#define PXF_SEGMENT_ID GpIdentity.segindex
#define PXF_SEGMENT_COUNT getgpsegmentCount()
+/*
+ * Fragment metadata for parallel execution.
+ * Stored in local memory, not in shared memory.
+ */
+typedef struct PxfFragmentData
+{
+ char *source_name; /* fragment source name (e.g. file path) */
+ int index; /* fragment index */
+ char *metadata; /* fragment metadata (base64 encoded) */
+ char *profile; /* optional profile override */
+} PxfFragmentData;
+
+/*
+ * Shared state for parallel foreign scan.
+ * This structure is stored in DSM (dynamic shared memory).
+ */
+typedef struct PxfParallelScanState
+{
+ slock_t mutex; /* mutex for accessing shared state */
+ int total_fragments; /* total number of fragments */
+ int next_fragment; /* next fragment index to be assigned */
+ bool finished; /* true if all fragments have been processed */
Review Comment:
code is refact
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-6789683918
How about this PR's progress? If there are no more comments, I think we can merge it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-3888146388
@ostinru
Thanks for the review. The current code is not yet ready for the review stage.
My approach to kernel parallel processing is still too simplistic. Maybe I will change or refactor later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r2876222982
##########
server/pxf-service/src/main/java/org/apache/cloudberry/pxf/service/controller/ReadServiceImpl.java:
##########
@@ -78,11 +82,26 @@ private OperationResult writeStream(RequestContext context, OutputStream outputS
OperationStats queryStats = new OperationStats(OperationStats.Operation.READ, metricsReporter, context);
OperationResult queryResult = new OperationResult();
- // dataStream (and outputStream as the result) will close automatically at the end of the try block
+ // dataStream (and outputStream as the result) will close automatically at the
+ // end of the try block
CountingOutputStream countingOutputStream = new CountingOutputStream(outputStream);
String sourceName = null;
try {
- List<Fragment> fragments = fragmenterService.getFragmentsForSegment(context);
+ List<Fragment> fragments;
+
+ // Check if a specific fragment index was requested (parallel execution mode)
+ if (context.hasSpecificFragment()) {
+ // Parallel mode: only process the specified fragment
+ Fragment specificFragment = fragmenterService.getFragmentByIndex(
+ context, context.getSpecificFragmentIndex());
+ fragments = java.util.Collections.singletonList(specificFragment);
Review Comment:
code is refact
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-6513460367
Review by `ocr delegate` model (backed by Claude Opus 5.5), FYI
---
[review-comment.md](https://github.com/user-attachments/files/34162344/review-comment.md)
[test-report.md](https://github.com/user-attachments/files/34162345/test-report.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub) via commits" <co...@cloudberry.apache.org>.
MisterRaindrop commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r4438878835
##########
automation/src/test/java/org/apache/cloudberry/pxf/automation/features/parallel/ParallelScanTest.java:
##########
Review Comment:
Added in 7f708c1f.
##########
pom.xml:
##########
@@ -364,6 +364,20 @@
<exclude>automation/README.Linux.md</exclude>
<exclude>automation/sqlrepo/features/reader/multi_block/expected/**</exclude>
<exclude>automation/sqlrepo/features/reader/multi_block/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/append/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/append/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/correctness/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/correctness/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/explain/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/explain/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/option_precedence/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/option_precedence/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/reject_limit/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/reject_limit/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/rescan/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/rescan/sql/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/single_fragment/expected/**</exclude>
+ <exclude>automation/sqlrepo/features/parallel/single_fragment/sql/**</exclude>
Review Comment:
Moved them to the Cloudberry section in 7f708c1f.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-3888340431
All deployments are local.
Sizes: 100MB, 1GB, 10GB
Workers: 4
Size | Rows | Workers | COUNT seq (ms) | COUNT par (ms) | Speedup | SUM seq (ms) | SUM par (ms) | Speedup
---------|--------------|---------|----------------|----------------|---------|----------------|----------------|--------
100MB | 487,700 | 4 | 282 | 311 | 0.91x | 290 | 188 | 1.54x
1GB | 4,994,140 | 4 | 2352 | 1514 | 1.55x | 2448 | 1314 | 1.86x
10GB | 49,941,480 | 4 | 21524 | 11589 | 1.86x | 21954 | 11547 | 1.90x
When exploring parallelization, the good news is that parallelization does indeed improve efficiency. For small data volumes, the improvement is not obvious and may even be less efficient than non-parallel processing. Only when the data volume is large does it show a noticeable improvement.
However, the current improvement still falls short of the expected level. Theoretically, the speedup factor should be almost equal to the number of workers. The reason it hasn’t reached the expected level may be due to bottlenecks in I/O or CPU. Further exploration will be conducted in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Add gang-parallel scan to pxf_fdw [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub) via commits" <co...@cloudberry.apache.org>.
MisterRaindrop commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-6553650077
Thanks @tuhaihe, the review was useful and the repros made each item easy to check. Addressed in 8757e6eb, f4a487cb, ce0afaf1 and d722f96a:
**1. Reject limit per worker.** No parallel path is offered for a table with a `reject_limit`. New test `features/parallel/reject_limit`: 12 files with one bad row each; limit 3 still fails with parallel enabled, limit 20 succeeds, uses a plain scan and returns the 1200 good rows. The old build fails this test.
**2. Costing and row estimate.** The 30x gap comes from `adjust_costs_for_mpp_foreign_scan()` in allpaths.c, which multiplies the rows of a strewn ForeignPath by `numsegments * 10`, but only for the pathlist, not the partial pathlist. The partial path now starts from the same estimate and divides it by the workers (`get_parallel_divisor()` is the worker count on Cloudberry, as there is no leader). Both paths charge `cpu_tuple_cost` per estimated row, and the parallel path divides it by the workers.
What this changes for GROUP BY: the parallel path is no longer chosen. Cloudberry does not plan a parallel partial HashAggregate over randomly distributed input, even with the parallel costs set to 0 (a randomly distributed heap table behaves the same way). So the plan is now the non-parallel Streaming Partial HashAggregate: no slowdown on CSV, but also no speedup on Parquet. COUNT/SUM, filters and joins still use the parallel scan. I did not size the worker count or skip small tables, as PXF does not know the data size at plan time; the docs say a small table can be slower.
**3. Rescan test.** I couldn't put a partial path on a rescanned side either, so I simplified the rescan to close the import and let the next iterate start a new one, and said in a comment that a partial path is never rescanned.
**4. Duplicated import setup.** `InitCopyStateVirtual` and `PxfBridgeImportStartVirtual` are gone. `PxfBridgeImportStart` overrides the segment headers when `gang_parallel` is set.
**5. PxfCopyFromErrorCallback.** Restored to main, together with `pxfEndForeignScan` and the iterate loop.
**Nits.** `worker_index`, the bare scope and `utils/guc.h` are removed. The cost change is in the commit message: 32 plans (16 queries, both optimizers) are the same as with 2.2.0. `enable_parallel` is documented in `fdw/PXF_FDW.md`. I left 77ff52c9 and 0bbe2608 as they are to avoid a force-push during the review; they can be reworded or squashed before merge.
One more fix: the EXPLAIN tests compared nothing. `atmsort.pl` hands any statement matching `explain ... select` to `explain.pl`, which the CI image doesn't have, so both sides were empty. The plan is now returned by a function as query rows, and a wrong expected plan fails the test.
Happy to have the benchmark re-run on d722f96a.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "MisterRaindrop (via GitHub)" <gi...@apache.org>.
MisterRaindrop commented on code in PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#discussion_r4400172168
##########
automation/sqlrepo/features/parallel/explain/expected/query01.ans:
##########
@@ -0,0 +1,32 @@
+-- @description query01 for PXF parallel scan EXPLAIN - Gather node present when parallel enabled
+-- start_matchsubs
+--
+-- m/Workers Planned: \d+/
+-- s/Workers Planned: \d+/Workers Planned: N/
+--
+-- m/cost=\d+\.\d+\.\.\d+\.\d+/
+-- s/cost=\d+\.\d+\.\.\d+\.\d+/cost=XXX/
+--
+-- m/rows=\d+/
+-- s/rows=\d+/rows=NNN/
+--
+-- m/width=\d+/
+-- s/width=\d+/width=NN/
+--
+-- end_matchsubs
+SET optimizer = off;
+SET
+SET enable_parallel = true;
+SET
+SET max_parallel_workers_per_gather = 4;
+SET
+EXPLAIN SELECT count(*) FROM pxf_parallel_enabled;
+ QUERY PLAN
+----------------------------------------------------------------------------------------------
+ Gather Motion 1:1 (slice1; segments: 1) (cost=XXX rows=NNN width=NN)
Review Comment:
You're right, it wasn't. That expected output was written by hand: a PostgreSQL Gather with Workers Planned under a one-segment Gather Motion. Cloudberry plans a parallel pxf_fdw scan as gang-parallel workers under the Gather Motion, so there is no Gather node and no Workers Planned, and CI runs on three segments.
3981ccb2 replaced it with the plan Cloudberry actually produces:
```
Finalize Aggregate
-> Gather Motion 12:1 (slice1; segments: 12)
-> Partial Aggregate
-> Parallel Foreign Scan on pxf_parallel_enabled
```
It uses EXPLAIN (COSTS OFF) and normalizes the Gather Motion size, so the test doesn't depend on the number of segments. ParallelScanTest.testParallelExplain passes in the features_fdw jobs of the latest CI run. Sorry for the slow reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] feat: pxf fdw support parallel scan [cloudberry-pxf]
Posted by "ostinru (via GitHub)" <gi...@apache.org>.
ostinru commented on PR #61:
URL: https://github.com/apache/cloudberry-pxf/pull/61#issuecomment-6490135144
> The current code is not yet ready for the review stage.
@MisterRaindrop , is it ready for review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org