You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "roaldm153 (via GitHub)" <gi...@apache.org> on 2026/08/27 13:29:23 UTC
[PR] Feature: add pg_query_state to gp_stats_collector [cloudberry]
roaldm153 opened a new pull request, #1934:
URL: https://github.com/apache/cloudberry/pull/1934
### What does this PR do?
Adds a signal-based runtime query-state facility to the `gp_stats_collector` extension. It lets a session inspect the live execution state of another running backend on demand - walking its active plan tree across the QD and all QEs - without waiting for the query to finish, pushing batches to the UDS(unix domain socket).
New SQL API (extension v1.2, schema `gpsc`):
- `gpsc.pg_query_state(pid, trace_id)` - fan out a poll to the query running on `pid`; each participating backend walks its plan tree and logs a per-node snapshot.
- `gpsc.pg_query_state_backends(pid)` - list the `(segid, pid)` QE backends taking part in that query.
- `cbdb_mpp_query_state(gpsc.gp_segment_pid[], trace_id)` - QE-side dispatch target.
The extension embeds the `pg_query_state` signal layer, which depends on three PostgreSQL core changes folded directly into the tree (configure enables the extension by default, so the tree must build without a manual patch step):
- custom `ProcSignal` handlers (`procsignal.c/.h`, `postgres.c`);
- end-of-node instrumentation flag readable mid-run (`instrument.c/.h`);
- runtime `EXPLAIN` entry points (`explain.c/.h`).
### Type of Change
- [ ] Bug fix (non-breaking change)
- [x] New feature (non-breaking change)
- [ ] Breaking change (fix or feature with breaking changes)
- [ ] Documentation update
### Breaking Changes
None. Core changes are additive (new signal reasons, a new Instrumentation field, new runtime-EXPLAIN paths); existing behavior is unchanged.
### Test Plan
- [x] Unit tests added/updated
- [x] Integration tests added/updated
- [x] Passed `make installcheck`
- pg_regress (`gpcontrib/gp_stats_collector/test`): catalog contract (function/type registration, exec location) and input-validation errors.
- isolation2 (`gpcontrib/gp_stats_collector/test/isolation2`), multi-session:
- `gpsc_pqs_backends` - idle backend yields an empty list;
- `gpsc_pqs_running` - happy path: poll a query suspended on the QEs via a fault injector;
- `gpsc_pqs_perms` - permission gate (non-owner non-superuser denied);
- `gpsc_pqs_disabled` - `STAT_DISABLED` reports no backends;
- `gpsc_pqs_seg_count` - one backend per primary segment.
- `gpsc-crash-test` - parallel tracing on `make installcheck-parallel`, core dumps checking
- A dedicated CI workflow builds Cloudberry with the extension across ubuntu22.04/rocky8/rocky9 and runs the pg_regress, isolation2, and core regression suites.
### Impact
**Performance:**
No steady-state cost. Work happens only when a poll is issued: one signal fan-out plus a bounded shared-memory round-trip; the shmem lock is held only around the request, not during dispatch.
**User-facing changes:**
New `gpsc.pg_query_state*` functions and the `pg_query_state.enable` GUC. Functions are granted to PUBLIC; access is gated in C so a caller may poll a backend only if it is a superuser or owns the target query.
**Dependencies:**
None.
### Checklist
- [x] Followed contribution guide
- [x] Added/updated documentation
- [x] Reviewed code for security implications
- [x] Requested review from cloudberry committers
### Additional Context
The signal layer under `src/pg_query_state/` is derived from [pg_query_state](https://github.com/postgrespro/pg_query_state) (PostgreSQL License). Derived files carry dual license headers (ASF + `Portions Copyright Postgres Professional`) and the root `LICENSE` records a PostgreSQL-License notice for it. Happy to provide provenance details for IP clearance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Feature: add pg_query_state to gp_stats_collector [cloudberry]
Posted by "leborchuk (via GitHub)" <gi...@apache.org>.
leborchuk commented on PR #1934:
URL: https://github.com/apache/cloudberry/pull/1934#issuecomment-5659930614
Could be merged, failed yezzey tests do not relate changes - Minio is no longer available opensource
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Feature: add pg_query_state to gp_stats_collector [cloudberry]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on PR #1934:
URL: https://github.com/apache/cloudberry/pull/1934#issuecomment-5658731941
Now, after the changes, I think it would be OK to merge. If there are any questions, we can sort them out later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Feature: add pg_query_state to gp_stats_collector [cloudberry]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on code in PR #1934:
URL: https://github.com/apache/cloudberry/pull/1934#discussion_r3902726645
##########
gpcontrib/gp_stats_collector/protos/yagpcc_metrics.proto:
##########
Review Comment:
Once the YAGPCC is donated to the ASF, the new project will be called `cloudberry-command-center`. So, is it better to align this new project name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Feature: add pg_query_state to gp_stats_collector [cloudberry]
Posted by "tuhaihe (via GitHub)" <gi...@apache.org>.
tuhaihe commented on code in PR #1934:
URL: https://github.com/apache/cloudberry/pull/1934#discussion_r3902726645
##########
gpcontrib/gp_stats_collector/protos/yagpcc_metrics.proto:
##########
Review Comment:
Once the YAGPCC is donated to the ASF, the new project will be called `cloudberry-command-center'. So, is it better to align this new project name?
##########
LICENSE:
##########
@@ -200,6 +200,28 @@
See the License for the specific language governing permissions and
limitations under the License.
+================================================================================
+This product includes software derived from pg_query_state
+(https://github.com/postgrespro/pg_query_state), under the PostgreSQL License:
+
+ Copyright (c) 2016-2025, Postgres Professional
+
+ Permission to use, copy, modify, and distribute this software and its
+ documentation for any purpose, without fee, and without a written agreement
+ is hereby granted, provided that the above copyright notice and this
+ paragraph and the following two paragraphs appear in all copies.
+
+ IN NO EVENT SHALL POSTGRES PROFESSIONAL BE LIABLE TO ANY PARTY FOR DIRECT,
+ INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST
+ PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN
+ IF POSTGRES PROFESSIONAL HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ POSTGRES PROFESSIONAL SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT
+ NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+ PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS,
+ AND POSTGRES PROFESSIONAL HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT,
+ UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
+
Review Comment:
You can append it to the `Apache Cloudberry includes codes from this file ` part.
##########
gpcontrib/gp_stats_collector/test/crash/README.md:
##########
Review Comment:
We can also add the standard ASF license header for `.md` file.
##########
.github/workflows/gpsc-ci.yaml:
##########
Review Comment:
Any possibility that we could integrate this into the existing CI workflow?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Feature: add pg_query_state to gp_stats_collector [cloudberry]
Posted by "leborchuk (via GitHub)" <gi...@apache.org>.
leborchuk commented on PR #1934:
URL: https://github.com/apache/cloudberry/pull/1934#issuecomment-5492154683
It's the last part of functionality from https://github.com/apache/cloudberry/pull/1085 - sent data about query execution progress to the outer agent. As in https://github.com/apache/cloudberry/pull/1085 we sent signal to the all backend processes, they report progress, and external agent could aggregate it and show to the user. The whole idea is the same but implementation is quite differ since neither existing postgres extensions nor #1085 proposal is ready for production use in Apache Cloudberry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] Feature: add pg_query_state to gp_stats_collector [cloudberry]
Posted by "leborchuk (via GitHub)" <gi...@apache.org>.
leborchuk merged PR #1934:
URL: https://github.com/apache/cloudberry/pull/1934
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org