You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by ma...@apache.org on 2024/12/23 11:10:55 UTC
(cloudberry) branch main updated (b54afa4602 -> 11074eb3e9)
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
from b54afa4602 Add test case with ORCA query optimizer enabled
new 04d7ba708c Add distributed snapshot support to pg_export_snapshot
new edbe8f473c Check if the source snapshot already has a distributed snapshot.
new daf51fc1c2 Fix export snapshot test
new 56fd24223c Validate sql exec_location in CreateFunction (#13431)
new 0e5498e2db Resolve GPDB_12_MERGE_FIXME in pathnode.c (#13466)
new e71edfc1ed pylib: Fix the error while doing remote file operation (#12433)
new 4f08c7fdac Check recoveryTargetAction before put PM_STATUS_STANDBY
new 2da8ab93a7 Allow max_replication_slots=0
new 7faf374821 Print errno for QE->QD dispatch checks
new b58ca90db5 Resolve GPDB_12_MERGE_FIXME in system_views.sql (#13469)
new f19bd8db51 [README][typo] fix gpdemo path typo (#13670)
new bae05aa8b8 Fix a flakiness in test recovery test overwrite_contrecord
new abd1545a6d Add logicalEof to errdetail in OpenAOSegmentFile
new 11f5f19dc9 remove a superfluous variable (#13681)
new 59715849fd [typo] fix a typo to make the comment more precise (#13702)
new 0ea097e2f1 Fix small typos in config parameters description. (#13697)
new 866ec2b91a fix fixme in planner about calculating dNumGroups for grouping sets
new 88b431ce26 Fix gppkg's coordinator hook not execute.
new 29bb0e77d5 Gut the appendoptimized_basic test
new c85510a13b blkdir: Rename CurrentSegmentFile and CurrentBlock
new 549dac7fc0 aoblkdir: Remove dead function prototypes
new 0494308e32 Harden dsm_impl.c against unexpected EEXIST.
new 11074eb3e9 Fix singlenode workfile_mgr_test test
The 23 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
configure | 1 -
gpAux/gpdemo/README | 2 +-
gpMgmt/bin/gpinitsystem | 7 +-
gpMgmt/bin/gppylib/operations/package.py | 2 +-
gpMgmt/bin/gppylib/operations/unix.py | 18 +-
gpMgmt/bin/lib/gp_bash_functions.sh | 13 +-
.../gp_debug_numsegments--1.0.sql | 12 +-
src/backend/access/aocs/aocsam.c | 4 +-
src/backend/access/appendonly/aomd.c | 3 +-
src/backend/access/appendonly/appendonlyam.c | 4 +-
src/backend/catalog/system_functions.sql | 5 +-
src/backend/cdb/endpoint/cdbendpoint.c | 4 +-
src/backend/commands/functioncmds.c | 2 +
src/backend/optimizer/plan/planner.c | 17 -
src/backend/optimizer/util/pathnode.c | 20 +-
src/backend/postmaster/postmaster.c | 19 +-
src/backend/storage/ipc/dsm_impl.c | 6 +-
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/guc_gp.c | 4 +-
src/backend/utils/time/snapmgr.c | 103 +++++-
src/include/access/appendonlytid.h | 2 +-
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 8 +-
src/include/cdb/cdbappendonlyam.h | 4 +-
src/include/cdb/cdbappendonlyblockdirectory.h | 21 +-
src/include/utils/datumstream.h | 4 +-
.../expected/heap_checksum_corruption.out | 49 ++-
.../heap_checksum/sql/heap_checksum_corruption.sql | 14 +-
.../expected/export_distributed_snapshot.out | 396 +++++++++++++++++++++
src/test/isolation2/expected/sharedsnapshot.out | 2 +-
src/test/isolation2/input/workfile_mgr_test.source | 4 +-
src/test/isolation2/isolation2_schedule | 2 +
.../isolation2/output/workfile_mgr_test.source | 4 +-
.../isolation2/sql/export_distributed_snapshot.sql | 249 +++++++++++++
src/test/isolation2/sql/sharedsnapshot.sql | 2 +-
src/test/perl/PostgresNode.pm | 2 +-
.../recovery/t/103_promotion_wait_status_ready.pl | 59 +++
.../regress/expected/ao_checksum_corruption.out | 71 ++--
src/test/regress/expected/motion_socket.out | 2 +-
src/test/regress/expected/privileges.out | 2 +-
src/test/regress/expected/psql_gp_commands.out | 12 +-
src/test/regress/expected/transactions.out | 8 +-
.../regress/input/gp_dispatch_keepalives.source | 2 +-
src/test/regress/input/oid_wraparound.source | 6 +-
src/test/regress/input/pgstat_qd_tabstat.source | 2 +-
.../regress/output/gp_dispatch_keepalives.source | 2 +-
src/test/regress/output/oid_wraparound.source | 6 +-
src/test/regress/output/pgstat_qd_tabstat.source | 2 +-
src/test/regress/sql/ao_checksum_corruption.sql | 22 +-
src/test/regress/sql/appendoptimized_basic.sql | 71 ----
src/test/regress/sql/motion_socket.sql | 2 +-
src/test/regress/sql/privileges.sql | 2 +-
src/test/regress/sql/psql_gp_commands.sql | 6 +-
src/test/regress/sql/transactions.sql | 8 +-
.../input/workfile_mgr_test.source | 4 +-
.../output/workfile_mgr_test.source | 4 +-
.../singlenode_regress/expected/transactions.out | 16 +-
src/test/singlenode_regress/sql/transactions.sql | 24 +-
58 files changed, 1027 insertions(+), 319 deletions(-)
create mode 100644 src/test/isolation2/expected/export_distributed_snapshot.out
create mode 100644 src/test/isolation2/sql/export_distributed_snapshot.sql
create mode 100644 src/test/recovery/t/103_promotion_wait_status_ready.pl
delete mode 100644 src/test/regress/sql/appendoptimized_basic.sql
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 23/23: Fix singlenode workfile_mgr_test test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 11074eb3e946b5687a02e75623f0e75d90c2e235
Author: yangjianghua <ya...@hashdata.cn>
AuthorDate: Wed Dec 18 16:06:48 2024 +0800
Fix singlenode workfile_mgr_test test
---
src/test/singlenode_isolation2/input/workfile_mgr_test.source | 4 ++--
src/test/singlenode_isolation2/output/workfile_mgr_test.source | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/test/singlenode_isolation2/input/workfile_mgr_test.source b/src/test/singlenode_isolation2/input/workfile_mgr_test.source
index aec099f6b4..30f03315d7 100644
--- a/src/test/singlenode_isolation2/input/workfile_mgr_test.source
+++ b/src/test/singlenode_isolation2/input/workfile_mgr_test.source
@@ -17,11 +17,11 @@ $$
LANGUAGE SQL;
CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_workset(worksetname text, interXact bool, holdPin bool, closeFile bool)
-RETURNS void
+RETURNS setof void
LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_empty_workset(worksetname text)
-RETURNS void
+RETURNS setof void
LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE FUNCTION gp_workfile_mgr_cache_entries()
diff --git a/src/test/singlenode_isolation2/output/workfile_mgr_test.source b/src/test/singlenode_isolation2/output/workfile_mgr_test.source
index af85e1100e..260aac112e 100644
--- a/src/test/singlenode_isolation2/output/workfile_mgr_test.source
+++ b/src/test/singlenode_isolation2/output/workfile_mgr_test.source
@@ -7,10 +7,10 @@ CREATE
CREATE FUNCTION gp_workfile_mgr_test(testname text, numfiles int) RETURNS SETOF BOOL AS $$ SELECT C.* FROM gp_workfile_mgr_test_on_master($1, $2) as C UNION ALL SELECT C.* FROM gp_workfile_mgr_test_on_segments($1, $2) as C $$ LANGUAGE SQL;
CREATE
-CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_workset(worksetname text, interXact bool, holdPin bool, closeFile bool) RETURNS void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
+CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_workset(worksetname text, interXact bool, holdPin bool, closeFile bool) RETURNS setof void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE
-CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_empty_workset(worksetname text) RETURNS void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
+CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_empty_workset(worksetname text) RETURNS setof void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE
CREATE FUNCTION gp_workfile_mgr_cache_entries() RETURNS TABLE(segid int4, prefix text, size int8, operation text, slice int4, sessionid int4, commandid int4, numfiles int4) AS '$libdir/gp_workfile_mgr', 'gp_workfile_mgr_cache_entries' LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 02/23: Check if the source snapshot already has a distributed snapshot.
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit edbe8f473c04c6525cb03cdeffacde59d1bc6248
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Thu Mar 10 16:04:41 2022 -0500
Check if the source snapshot already has a distributed snapshot.
In SetTransactionSnapshot, If the source snapshot already has a distributed snapshot,
pass in DTX_CONTEXT_LOCAL_ONLY to GetSnapshotData(). This prevents a new
distributed snapshot from being created in GetSnapshotData() and ensures
that we can use the distributed snapshot from the source snapshot
---
src/backend/utils/time/snapmgr.c | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 3050735dc2..514fe5d1f7 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -593,13 +593,16 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid,
* the state for GlobalVis*.
*/
- /*
- * Pass in DTX_CONTEXT_LOCAL_ONLY to prevent a distributed snapshot from
- * getting created here, as we will be using the one we parsed in
- * ImportSnapshot.
+ /*
+ * GPDB: If the source snapshot already has a distributed snapshot, pass in
+ * DTX_CONTEXT_LOCAL_ONLY to GetSnapshotData(). This prevents a new
+ * distributed snapshot from being created in GetSnapshotData() and ensures
+ * that we can use the distributed snapshot from the source snapshot below.
*/
- CurrentSnapshot = GetSnapshotData(&CurrentSnapshotData, DTX_CONTEXT_LOCAL_ONLY);
-
+ if (sourcesnap->haveDistribSnapshot)
+ CurrentSnapshot = GetSnapshotData(&CurrentSnapshotData, DTX_CONTEXT_LOCAL_ONLY);
+ else
+ CurrentSnapshot = GetSnapshotData(&CurrentSnapshotData, DistributedTransactionContext);
/*
* Now copy appropriate fields from the source snapshot.
*/
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 03/23: Fix export snapshot test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit daf51fc1c2e7aaa7904c27a1455a8b17781ea727
Author: Brent Doil <bd...@vmware.com>
AuthorDate: Fri May 13 18:35:37 2022 -0400
Fix export snapshot test
Commit 8eaa3b682efb050ccdd7b22c0f0b7087b8ea6efd updated the
behavior of ON [COORDINATOR | ALL SEGMENTS] to only allow
set returning functions.
This commit updates the export snapshot isolation tests
to reflect that change.
Add tests for UPDATE and DELETE table
---
.../expected/export_distributed_snapshot.out | 132 +++++++++++++++++++--
.../isolation2/sql/export_distributed_snapshot.sql | 65 ++++++++--
2 files changed, 177 insertions(+), 20 deletions(-)
diff --git a/src/test/isolation2/expected/export_distributed_snapshot.out b/src/test/isolation2/expected/export_distributed_snapshot.out
index 130b319551..d9270f20e3 100644
--- a/src/test/isolation2/expected/export_distributed_snapshot.out
+++ b/src/test/isolation2/expected/export_distributed_snapshot.out
@@ -23,11 +23,11 @@ CREATE
-- end_ignore
-- Corrupt field entry for given snapshot file
-CREATE OR REPLACE FUNCTION corrupt_snapshot_file(token text, field text) RETURNS integer as $$ import os content = bytearray() query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path" rv = plpy.execute(query) abs_path = rv[0]['path'] snapshot_file = abs_path + token if not os.path.isfile(snapshot_file): plpy.info('skipping non-existent file %s' % (snapshot_file)) else: plpy.info('corrupting file %s for field %s' % (snapsho [...]
+CREATE OR REPLACE FUNCTION corrupt_snapshot_file(token text, field text) RETURNS integer as $$ import os content = bytearray() query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path" rv = plpy.execute(query) abs_path = rv[0]['path'] snapshot_file = abs_path + token if not os.path.isfile(snapshot_file): plpy.info('skipping non-existent file %s' % (snapshot_file)) else: plpy.info('corrupting file %s for field %s' % (snapsho [...]
CREATE
-- Determine if field exists for given snapshot file
-CREATE OR REPLACE FUNCTION snapshot_file_ds_fields_exist(token text) RETURNS boolean as $$ import os content = bytearray() query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path" rv = plpy.execute(query) abs_path = rv[0]['path'] snapshot_file = abs_path + token if not os.path.isfile(snapshot_file): plpy.info('snapshot file %s does not exist' % (snapshot_file)) return -1 else: plpy.info('checking file %s for ds fields' % ( [...]
+CREATE OR REPLACE FUNCTION snapshot_file_ds_fields_exist(token text) RETURNS boolean as $$ import os content = bytearray() query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path" rv = plpy.execute(query) abs_path = rv[0]['path'] snapshot_file = abs_path + token if not os.path.isfile(snapshot_file): plpy.info('snapshot file %s does not exist' % (snapshot_file)) return -1 else: plpy.info('checking file %s for ds fields' % ( [...]
CREATE
-- INSERT test
@@ -79,9 +79,115 @@ COMMIT
1: COMMIT;
COMMIT
--- DROP test
+-- DELETE test
CREATE TABLE export_distributed_snapshot_test2 (a int);
CREATE
+INSERT INTO export_distributed_snapshot_test2 SELECT a FROM generate_series(1,3) a;
+INSERT 3
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000007-0000007C-1
+(1 row)
+
+DELETE FROM export_distributed_snapshot_test2 WHERE a=1;
+DELETE 1
+
+-- Should return 3 rows
+1: SELECT * FROM export_distributed_snapshot_test2 ;
+ a
+---
+ 1
+ 2
+ 3
+(3 rows)
+
+-- Should return 2 rows
+2: SELECT * FROM export_distributed_snapshot_test2;
+ a
+---
+ 2
+ 3
+(2 rows)
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+SET
+
+-- Should return 3 rows
+2: SELECT * FROM export_distributed_snapshot_test2;
+ a
+---
+ 2
+ 3
+ 1
+(3 rows)
+2: COMMIT;
+COMMIT
+
+1: COMMIT;
+COMMIT
+
+-- UPDATE test
+CREATE TABLE export_distributed_snapshot_test3 (a int);
+CREATE
+INSERT INTO export_distributed_snapshot_test3 SELECT a FROM generate_series(1,5) a;
+INSERT 5
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000007-0000007D-1
+(1 row)
+
+UPDATE export_distributed_snapshot_test3 SET a=99 WHERE a=1;
+UPDATE 1
+
+-- Should return 0 rows
+1: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+ a
+---
+(0 rows)
+
+-- Should return 1 row
+2: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+ a
+----
+ 99
+(1 row)
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+SET
+
+-- Should return 0 rows
+2: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+ a
+---
+(0 rows)
+2: COMMIT;
+COMMIT
+
+-- Should return 1 row
+2: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+ a
+----
+ 99
+(1 row)
+
+1: COMMIT;
+COMMIT
+
+-- DROP test
+CREATE TABLE export_distributed_snapshot_test4 (a int);
+CREATE
1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
BEGIN
@@ -93,16 +199,16 @@ BEGIN
---------------------
00000007-00000012-1
(1 row)
-1: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+1: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test4';
gp_segment_id | relname
---------------+-----------------------------------
- 1 | export_distributed_snapshot_test2
- 2 | export_distributed_snapshot_test2
- 0 | export_distributed_snapshot_test2
+ 1 | export_distributed_snapshot_test4
+ 2 | export_distributed_snapshot_test4
+ 0 | export_distributed_snapshot_test4
(3 rows)
-- Drop table in transaction
-2: DROP TABLE export_distributed_snapshot_test2;
+2: DROP TABLE export_distributed_snapshot_test4;
DROP
2: COMMIT;
COMMIT
@@ -112,17 +218,17 @@ BEGIN
3: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
SET
-- The table should still be visible to transaction 3 using transaction 1's snapshot.
-3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test4';
gp_segment_id | relname
---------------+-----------------------------------
- 1 | export_distributed_snapshot_test2
- 2 | export_distributed_snapshot_test2
- 0 | export_distributed_snapshot_test2
+ 1 | export_distributed_snapshot_test4
+ 2 | export_distributed_snapshot_test4
+ 0 | export_distributed_snapshot_test4
(3 rows)
3: COMMIT;
COMMIT
-- The table should no longer be visible to transaction 3.
-3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test4';
gp_segment_id | relname
---------------+---------
(0 rows)
diff --git a/src/test/isolation2/sql/export_distributed_snapshot.sql b/src/test/isolation2/sql/export_distributed_snapshot.sql
index eedd3f779a..c1e2a3c916 100644
--- a/src/test/isolation2/sql/export_distributed_snapshot.sql
+++ b/src/test/isolation2/sql/export_distributed_snapshot.sql
@@ -44,7 +44,7 @@ $$
f.write(content)
f.close()
return 0
-$$ LANGUAGE plpython3u EXECUTE ON MASTER;
+$$ LANGUAGE plpython3u;
-- Determine if field exists for given snapshot file
CREATE OR REPLACE FUNCTION snapshot_file_ds_fields_exist(token text) RETURNS boolean as
@@ -66,7 +66,7 @@ $$
if "ds" in l:
return True
return False
-$$ LANGUAGE plpython3u EXECUTE ON MASTER;
+$$ LANGUAGE plpython3u;
-- INSERT test
CREATE TABLE export_distributed_snapshot_test1 (a int);
@@ -90,26 +90,77 @@ SELECT * FROM export_distributed_snapshot_test1;
1: COMMIT;
--- DROP test
+-- DELETE test
CREATE TABLE export_distributed_snapshot_test2 (a int);
+INSERT INTO export_distributed_snapshot_test2 SELECT a FROM generate_series(1,3) a;
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+DELETE FROM export_distributed_snapshot_test2 WHERE a=1;
+
+-- Should return 3 rows
+1: SELECT * FROM export_distributed_snapshot_test2 ;
+
+-- Should return 2 rows
+2: SELECT * FROM export_distributed_snapshot_test2;
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+
+-- Should return 3 rows
+2: SELECT * FROM export_distributed_snapshot_test2;
+2: COMMIT;
+
+1: COMMIT;
+
+-- UPDATE test
+CREATE TABLE export_distributed_snapshot_test3 (a int);
+INSERT INTO export_distributed_snapshot_test3 SELECT a FROM generate_series(1,5) a;
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+UPDATE export_distributed_snapshot_test3 SET a=99 WHERE a=1;
+
+-- Should return 0 rows
+1: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+
+-- Should return 1 row
+2: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+
+-- Should return 0 rows
+2: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+2: COMMIT;
+
+-- Should return 1 row
+2: SELECT * FROM export_distributed_snapshot_test3 WHERE a=99;
+
+1: COMMIT;
+
+-- DROP test
+CREATE TABLE export_distributed_snapshot_test4 (a int);
1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
2: BEGIN;
1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
-1: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+1: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test4';
-- Drop table in transaction
-2: DROP TABLE export_distributed_snapshot_test2;
+2: DROP TABLE export_distributed_snapshot_test4;
2: COMMIT;
3: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
3: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
-- The table should still be visible to transaction 3 using transaction 1's snapshot.
-3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test4';
3: COMMIT;
-- The table should no longer be visible to transaction 3.
-3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test4';
1: COMMIT;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 22/23: Harden dsm_impl.c against unexpected EEXIST.
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 0494308e320dc058163678cc14f83065ec55714c
Author: Thomas Munro <tm...@postgresql.org>
AuthorDate: Fri Jul 1 12:05:52 2022 +1200
Harden dsm_impl.c against unexpected EEXIST.
Previously, we trusted the OS not to report EEXIST unless we'd passed in
IPC_CREAT | IPC_EXCL or O_CREAT | O_EXCL, as appropriate. Solaris's
shm_open() can in fact do that, causing us to crash because we didn't
ereport and then we blithely assumed the mapping was successful.
Let's treat EEXIST just like any other error, unless we're actually
trying to create a new segment. This applies to shm_open(), where this
behavior has been seen, and also to the equivalent operations for our
sysv and mmap modes just on principle.
Based on the underlying reason for the error, namely contention on a
lock file managed by Solaris librt for each distinct name, this problem
is only likely to happen on 15 and later, because the new shared memory
stats system produces shm_open() calls for the same path from
potentially large numbers of backends concurrently during
authentication. Earlier releases only shared memory segments between a
small number of parallel workers under one Gather node. You could
probably hit it if you tried hard enough though, and we should have been
more defensive in the first place. Therefore, back-patch to all
supported releases.
Per build farm animal margay. This isn't the end of the story, though,
it just changes random crashes into random "File exists" errors; more
work needed for a green build farm.
Reviewed-by: Robert Haas <ro...@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGKqKrCV5xKWfh9rnm%3Do%3DDwZLTLtnsj_XpUi9g5%3DV%2B9oyg%40mail.gmail.com
---
src/backend/storage/ipc/dsm_impl.c | 6 +++---
src/test/isolation2/expected/sharedsnapshot.out | 2 +-
src/test/isolation2/sql/sharedsnapshot.sql | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/backend/storage/ipc/dsm_impl.c b/src/backend/storage/ipc/dsm_impl.c
index 1487cb7588..4fa0e319e6 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -261,7 +261,7 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size request_size,
if ((fd = shm_open(name, flags, PG_FILE_MODE_OWNER)) == -1)
{
ReleaseExternalFD();
- if (errno != EEXIST)
+ if (op == DSM_OP_ATTACH || errno != EEXIST)
ereport(elevel,
(errcode_for_dynamic_shared_memory(),
errmsg("could not open shared memory segment \"%s\": %m",
@@ -500,7 +500,7 @@ dsm_impl_sysv(dsm_op op, dsm_handle handle, Size request_size,
if ((ident = shmget(key, segsize, flags)) == -1)
{
- if (errno != EEXIST)
+ if (op == DSM_OP_ATTACH || errno != EEXIST)
{
int save_errno = errno;
@@ -822,7 +822,7 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size request_size,
flags = O_RDWR | (op == DSM_OP_CREATE ? O_CREAT | O_EXCL : 0);
if ((fd = OpenTransientFile(name, flags)) == -1)
{
- if (errno != EEXIST)
+ if (op == DSM_OP_ATTACH || errno != EEXIST)
ereport(elevel,
(errcode_for_dynamic_shared_memory(),
errmsg("could not open shared memory segment \"%s\": %m",
diff --git a/src/test/isolation2/expected/sharedsnapshot.out b/src/test/isolation2/expected/sharedsnapshot.out
index 807740fa77..4342368ff7 100644
--- a/src/test/isolation2/expected/sharedsnapshot.out
+++ b/src/test/isolation2/expected/sharedsnapshot.out
@@ -1,4 +1,4 @@
-2: CREATE OR REPLACE PROCEDURE test(bdate text, edate text) LANGUAGE PLPGSQL AS $$/*in func*/ BEGIN /*in func*/ EXECUTE format('ALTER TABLE dummy2 ADD PARTITION START (date ''%s'') INCLUSIVE END (date ''%s'') EXCLUSIVE', bdate, edate);/*in func*/ EXCEPTION/*in func*/ WHEN NO_DATA_FOUND THEN/*in func*/ RAISE EXCEPTION 'exception';/*in func*/ END;/*in func*/ $$ EXECUTE ON COORDINATOR;
+2: CREATE OR REPLACE PROCEDURE test(bdate text, edate text) LANGUAGE PLPGSQL AS $$/*in func*/ BEGIN /*in func*/ EXECUTE format('ALTER TABLE dummy2 ADD PARTITION START (date ''%s'') INCLUSIVE END (date ''%s'') EXCLUSIVE', bdate, edate);/*in func*/ EXCEPTION/*in func*/ WHEN NO_DATA_FOUND THEN/*in func*/ RAISE EXCEPTION 'exception';/*in func*/ END;/*in func*/ $$;
CREATE
vacuum full pg_class;
diff --git a/src/test/isolation2/sql/sharedsnapshot.sql b/src/test/isolation2/sql/sharedsnapshot.sql
index c64ce6b5e5..74198767f1 100644
--- a/src/test/isolation2/sql/sharedsnapshot.sql
+++ b/src/test/isolation2/sql/sharedsnapshot.sql
@@ -5,7 +5,7 @@
WHEN NO_DATA_FOUND THEN/*in func*/
RAISE EXCEPTION 'exception';/*in func*/
END;/*in func*/
- $$ EXECUTE ON COORDINATOR;
+ $$;
vacuum full pg_class;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 05/23: Resolve GPDB_12_MERGE_FIXME in pathnode.c (#13466)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 0e5498e2db9f45feed4219e92152a519b0f16181
Author: QingMa <ma...@vmware.com>
AuthorDate: Mon May 16 17:53:34 2022 +0800
Resolve GPDB_12_MERGE_FIXME in pathnode.c (#13466)
These are limitations for functions defined with the EXECUTE ON MASTER or
EXECUTE ON ALL SEGMENTS attribute in Greenplum doc, one of them is: The
function must be a set-returning function. We can't pull up a subquery
has set-returning functions in tlist as RTE_RESULT relation. Therefore,
we don't need check execute location for functions in RTE_RESULT
relation's tlist.
---
src/backend/optimizer/util/pathnode.c | 20 +++++++-------------
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/src/backend/optimizer/util/pathnode.c b/src/backend/optimizer/util/pathnode.c
index c7b1025bce..8023110740 100644
--- a/src/backend/optimizer/util/pathnode.c
+++ b/src/backend/optimizer/util/pathnode.c
@@ -3494,19 +3494,13 @@ create_resultscan_path(PlannerInfo *root, RelOptInfo *rel,
char exec_location;
exec_location = check_execute_on_functions((Node *) rel->reltarget->exprs);
- if (exec_location == PROEXECLOCATION_COORDINATOR)
- CdbPathLocus_MakeEntry(&pathnode->locus);
- else if (exec_location == PROEXECLOCATION_ALL_SEGMENTS)
- {
- /* CBDB_PARALLEL_FIXME: I'm not sure if this makes sense. This
- * would return multiple rows, one for each segment, but usually
- * a "SELECT func()" is expected to return just one row.
- */
- CdbPathLocus_MakeStrewn(&pathnode->locus,
- getgpsegmentCount(), 0);
- }
- else
- CdbPathLocus_MakeGeneral(&pathnode->locus);
+ /*
+ * A function with EXECUTE ON { COORDINATOR | ALL SEGMENTS } attribute
+ * must be a set-returning function, a subquery has set-returning
+ * functions in tlist can't be pulled up as RTE_RESULT relation.
+ */
+ Assert(exec_location == PROEXECLOCATION_ANY);
+ CdbPathLocus_MakeGeneral(&pathnode->locus);
}
cost_resultscan(pathnode, root, rel, pathnode->param_info);
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 21/23: aoblkdir: Remove dead function prototypes
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 549dac7fc0066a6ff66555c8c15e05f9a892adb2
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Mon Jun 27 21:01:03 2022 -0700
aoblkdir: Remove dead function prototypes
---
src/backend/access/aocs/aocsam.c | 2 +-
src/backend/access/appendonly/appendonlyam.c | 2 +-
src/include/cdb/cdbappendonlyblockdirectory.h | 13 -------------
3 files changed, 2 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/aocs/aocsam.c b/src/backend/access/aocs/aocsam.c
index fc21ccf274..dfc21d117e 100644
--- a/src/backend/access/aocs/aocsam.c
+++ b/src/backend/access/aocs/aocsam.c
@@ -1317,7 +1317,7 @@ scanToFetchValue(AOCSFetchDesc aocsFetchDesc,
{
DatumStreamFetchDesc datumStreamFetchDesc = aocsFetchDesc->datumStreamFetchDesc[colno];
DatumStreamRead *datumStream = datumStreamFetchDesc->datumStream;
- CurrentBlock *currentBlock = &datumStreamFetchDesc->currentBlock;
+ AOFetchBlockMetadata *currentBlock = &datumStreamFetchDesc->currentBlock;
AppendOnlyBlockDirectoryEntry *entry = ¤tBlock->blockDirectoryEntry;
bool found;
diff --git a/src/backend/access/appendonly/appendonlyam.c b/src/backend/access/appendonly/appendonlyam.c
index 05757827fe..ac35185cae 100755
--- a/src/backend/access/appendonly/appendonlyam.c
+++ b/src/backend/access/appendonly/appendonlyam.c
@@ -1938,7 +1938,7 @@ fetchFromCurrentBlock(AppendOnlyFetchDesc aoFetchDesc,
TupleTableSlot *slot)
{
bool fetched;
- CurrentBlock *currentBlock = &aoFetchDesc->currentBlock;
+ AOFetchBlockMetadata *currentBlock = &aoFetchDesc->currentBlock;
AppendOnlyExecutorReadBlock *executorReadBlock = &aoFetchDesc->executorReadBlock;
AppendOnlyBlockDirectoryEntry *entry = ¤tBlock->blockDirectoryEntry;
diff --git a/src/include/cdb/cdbappendonlyblockdirectory.h b/src/include/cdb/cdbappendonlyblockdirectory.h
index 54819bef8c..0eeb2aea2a 100644
--- a/src/include/cdb/cdbappendonlyblockdirectory.h
+++ b/src/include/cdb/cdbappendonlyblockdirectory.h
@@ -233,19 +233,6 @@ extern bool AppendOnlyBlockDirectory_InsertEntry(
int64 fileOffset,
int64 rowCount,
bool addColAction);
-extern bool AppendOnlyBlockDirectory_addCol_InsertEntry(
- AppendOnlyBlockDirectory *blockDirectory,
- int columnGroupNo,
- int64 firstRowNum,
- int64 fileOffset,
- int64 rowCount);
-extern bool AppendOnlyBlockDirectory_DeleteEntry(
- AppendOnlyBlockDirectory *blockDirectory,
- AOTupleId *aoTupleId);
-extern bool AppendOnlyBlockDirectory_DeleteEntryForUpdate(
- AppendOnlyBlockDirectory *visibilityBlockDirectory,
- AppendOnlyBlockDirectory *insertBlockDirectory,
- AOTupleId* aoTupleId);
extern void AppendOnlyBlockDirectory_End_forInsert(
AppendOnlyBlockDirectory *blockDirectory);
extern void AppendOnlyBlockDirectory_End_forSearch(
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 04/23: Validate sql exec_location in CreateFunction (#13431)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 56fd24223cb22332d196d0849cb092b949778a94
Author: QingMa <ma...@vmware.com>
AuthorDate: Wed May 11 14:22:45 2022 +0800
Validate sql exec_location in CreateFunction (#13431)
GPDB add support for CREATE FUNCTION EXECUTE ON [COORDINATOR | ALL SEGMENTS] in #3312. According to
the commit message and Greenplum Doc, this CREATE FUNCTION options are only supported for
set-returning functions, but we didn't validate this limitation in `CreateFunction()` which
`AlterFunction()` does.
This leads us to have several built-in functions and functions for test cases violate this limitation.
Some of them are not set-returning functions but have EXECUTE ON SEGMENTS attribute. This would return
multiple rows, one for each segments.
For example:
```
postgres=# select proname, proretset, provolatile, proexeclocation from pg_proc where proretset != 't' and proexeclocation != 'a';
proname | proretset | provolatile | proexeclocation
----------------------------------+-----------+-------------+-----------------
brin_summarize_range | f | v | s
gin_clean_pending_list | f | v | s
gp_wait_parallel_retrieve_cursor | f | v | c
gp_request_fts_probe_scan | f | v | c
brin_summarize_new_values | f | v | c
(5 rows)
SELECT brin_summarize_range('brin_summarize_idx', 4294967295);
brin_summarize_range
----------------------
0
0
1
(3 rows)
```
In most cases, functions with the EXECUTE ON attributes are used to perform queries on
distributed data or to execute functions on all segments. It is reasonable for these
functions to return set. There are also cases where the functions need to be executed
on the coordinator but does not need to return a set, creating these functions as
set-returning function and using it like `SELECT func_exec_on_coordinator();` will not
affect the execution result.
This commit added validation for this limitation in `CreateFunction()` and modified
several returning non-set built-in functions which execute on master or all segments.
---
.../gp_debug_numsegments--1.0.sql | 12 ++--
src/backend/catalog/system_functions.sql | 2 +-
src/backend/commands/functioncmds.c | 2 +
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 8 ++-
.../expected/heap_checksum_corruption.out | 49 +++++++--------
.../heap_checksum/sql/heap_checksum_corruption.sql | 14 ++---
src/test/isolation2/input/workfile_mgr_test.source | 4 +-
.../isolation2/output/workfile_mgr_test.source | 4 +-
.../regress/expected/ao_checksum_corruption.out | 71 ++++++++++------------
src/test/regress/expected/motion_socket.out | 2 +-
src/test/regress/expected/privileges.out | 2 +-
src/test/regress/expected/psql_gp_commands.out | 12 ++--
src/test/regress/expected/transactions.out | 8 +--
.../regress/input/gp_dispatch_keepalives.source | 2 +-
src/test/regress/input/oid_wraparound.source | 6 +-
src/test/regress/input/pgstat_qd_tabstat.source | 2 +-
.../regress/output/gp_dispatch_keepalives.source | 2 +-
src/test/regress/output/oid_wraparound.source | 6 +-
src/test/regress/output/pgstat_qd_tabstat.source | 2 +-
src/test/regress/sql/ao_checksum_corruption.sql | 22 +++----
src/test/regress/sql/motion_socket.sql | 2 +-
src/test/regress/sql/privileges.sql | 2 +-
src/test/regress/sql/psql_gp_commands.sql | 6 +-
src/test/regress/sql/transactions.sql | 8 +--
.../singlenode_regress/expected/transactions.out | 16 +++--
src/test/singlenode_regress/sql/transactions.sql | 24 ++++++--
27 files changed, 150 insertions(+), 142 deletions(-)
diff --git a/gpcontrib/gp_debug_numsegments/gp_debug_numsegments--1.0.sql b/gpcontrib/gp_debug_numsegments/gp_debug_numsegments--1.0.sql
index 4a121070e7..e248ac7a00 100644
--- a/gpcontrib/gp_debug_numsegments/gp_debug_numsegments--1.0.sql
+++ b/gpcontrib/gp_debug_numsegments/gp_debug_numsegments--1.0.sql
@@ -7,30 +7,30 @@
-- This form accepts a text argument: 'full', 'minimal', 'random'.
CREATE OR REPLACE FUNCTION gp_debug_set_create_table_default_numsegments(text) RETURNS text
AS '$libdir/gp_debug_numsegments','gp_debug_set_create_table_default_numsegments'
-LANGUAGE C IMMUTABLE STRICT EXECUTE ON COORDINATOR;
+LANGUAGE C STRICT;
-- This form accepts an integer argument: [1, gp_num_contents_in_cluster].
CREATE OR REPLACE FUNCTION gp_debug_set_create_table_default_numsegments(integer) RETURNS text
AS '$libdir/gp_debug_numsegments','gp_debug_set_create_table_default_numsegments'
-LANGUAGE C IMMUTABLE STRICT EXECUTE ON COORDINATOR;
+LANGUAGE C STRICT;
-- This function reset the default numsegments.
-- This form accepts a text argument: 'full', 'minimal' or 'random',
-- numsegments will be reset to this value since this time.
CREATE OR REPLACE FUNCTION gp_debug_reset_create_table_default_numsegments(text) RETURNS void
AS '$libdir/gp_debug_numsegments','gp_debug_reset_create_table_default_numsegments'
-LANGUAGE C IMMUTABLE STRICT EXECUTE ON COORDINATOR;
+LANGUAGE C STRICT;
-- This form accepts an integer argument: [1, gp_num_contents_in_cluster],
-- numsegments will be reset to this value since this time.
CREATE OR REPLACE FUNCTION gp_debug_reset_create_table_default_numsegments(integer) RETURNS void
AS '$libdir/gp_debug_numsegments','gp_debug_reset_create_table_default_numsegments'
-LANGUAGE C IMMUTABLE STRICT EXECUTE ON COORDINATOR;
+LANGUAGE C STRICT;
-- This form resets numsegments directly, the value is decided by last reset,
-- or FULL if reset is never executed.
CREATE OR REPLACE FUNCTION gp_debug_reset_create_table_default_numsegments() RETURNS void
AS '$libdir/gp_debug_numsegments','gp_debug_reset_create_table_default_numsegments'
-LANGUAGE C IMMUTABLE STRICT EXECUTE ON COORDINATOR;
+LANGUAGE C STRICT;
-- This function get the default numsegments when creating tables.
CREATE OR REPLACE FUNCTION gp_debug_get_create_table_default_numsegments() RETURNS text
AS '$libdir/gp_debug_numsegments','gp_debug_get_create_table_default_numsegments'
-LANGUAGE C IMMUTABLE STRICT EXECUTE ON COORDINATOR;
+LANGUAGE C STRICT;
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 16c6122822..2aace6d122 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -607,7 +607,7 @@ AS 'unicode_is_normalized';
-- GPDB_12_MERGE_FIXME: This seems out of place..
-- GPDB_12_MERGE_FIXME: Shouldn't we have a wrapper like this for
-- brin_summarize_range(), too?
-create or replace function brin_summarize_new_values(t regclass) returns bigint as
+create or replace function brin_summarize_new_values(t regclass) returns setof bigint as
$$
-- brin_summarize_new_values_internal is marked as EXECUTE ON ALL SEGMENTS.
select sum(n) from brin_summarize_new_values_internal(t) as n;
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index f7c854d98b..473a7a9ea3 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -1598,6 +1598,8 @@ CreateFunction(ParseState *pstate, CreateFunctionStmt *stmt)
returnsSet = false;
}
+ validate_sql_exec_location(execLocation, returnsSet);
+
if (list_length(trftypes_list) > 0)
{
ListCell *lc;
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index 1370f78a11..a66b25fbef 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -56,6 +56,6 @@
*/
/* 3yyymmddN */
-#define CATALOG_VERSION_NO 302412061
+#define CATALOG_VERSION_NO 302412062
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9941c64988..ac15273e86 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9256,7 +9256,7 @@
prosrc => 'gin_tsquery_consistent_oldsig' },
{ oid => '3789', descr => 'clean up GIN pending list',
- proexeclocation => 's',
+ proexeclocation => 's', prorows => '1000', proretset => 't',
proname => 'gin_clean_pending_list', provolatile => 'v', proparallel => 'u',
prorettype => 'int8', proargtypes => 'regclass',
prosrc => 'gin_clean_pending_list' },
@@ -11989,7 +11989,8 @@
proname => 'gp_get_segment_endpoints', prorows => '1000', proretset => 't', provolatile => 'v', proparallel => 'u', prorettype => 'record', proargtypes => '', proallargtypes => '{text,oid,int4,int4,text,int4,int4,text,text,text}', proargmodes => '{o,o,o,o,o,o,o,o,o,o}', proargnames => '{auth_token,databaseid,senderpid,receiverpid,state,gp_segment_id,sessionid,username,endpointname,cursorname}', prosrc => 'gp_get_segment_endpoints' },
{ oid => 7182, descr => 'wait until all endpoint of this parallel retrieve cursor has been retrieved finished',
- proname => 'gp_wait_parallel_retrieve_cursor', provolatile => 'v', proparallel => 'u', prorettype => 'bool', proargtypes => 'text int4', proallargtypes => '{text,int4,bool}', proargmodes => '{i,i,o}', proargnames => '{cursorname,timeout_sec,finished}', prosrc => 'gp_wait_parallel_retrieve_cursor', proexeclocation => 'c' },
+ proname => 'gp_wait_parallel_retrieve_cursor', prorows => '1000', proretset => 't',
+ provolatile => 'v', proparallel => 'u', prorettype => 'bool', proargtypes => 'text int4', proallargtypes => '{text,int4,bool}', proargmodes => '{i,i,o}', proargnames => '{cursorname,timeout_sec,finished}', prosrc => 'gp_wait_parallel_retrieve_cursor', proexeclocation => 'c' },
{ oid => 7146, descr => 'debugging information for segment backends',
proname => 'gp_backend_info', prorettype => 'record', prorows => '1', proretset => 't', proargtypes => '', proallargtypes => '{int4,char,int4,text,int4,int4}', prosrc => 'gp_backend_info', pronargs => 6,
@@ -12128,7 +12129,8 @@
proname => 'gp_remove_segment', proisstrict => 'f', provolatile => 'v', proparallel => 'r', prorettype => 'bool', proargtypes => 'int2', prosrc => 'gp_remove_segment' },
{ oid => 7039, descr => 'Request a FTS probe scan and wait for response',
- proname => 'gp_request_fts_probe_scan', proisstrict => 'f', provolatile => 'v', prorettype => 'bool', proargtypes => '', prosrc => 'gp_request_fts_probe_scan', proexeclocation => 'c' },
+ proname => 'gp_request_fts_probe_scan', prorows => '1000', proretset => 't',
+ proisstrict => 'f', provolatile => 'v', prorettype => 'bool', proargtypes => '', prosrc => 'gp_request_fts_probe_scan', proexeclocation => 'c' },
{ oid => 7054, descr => 'anytable type serialization input function',
proname => 'anytable_in', prorettype => 'anytable', proargtypes => 'cstring', prosrc => 'anytable_in' },
diff --git a/src/test/heap_checksum/expected/heap_checksum_corruption.out b/src/test/heap_checksum/expected/heap_checksum_corruption.out
index a3dc7bae6a..12745c07c8 100644
--- a/src/test/heap_checksum/expected/heap_checksum_corruption.out
+++ b/src/test/heap_checksum/expected/heap_checksum_corruption.out
@@ -60,7 +60,7 @@ $$ language sql VOLATILE;
-- do a SUM() over the return values, and it's always 0, regardless of the
-- number of segments in the cluster.)
CREATE FUNCTION corrupt_file(data_file text, corruption_offset int4)
-RETURNS integer as $$
+RETURNS setof integer as $$
import os;
if not os.path.isfile(data_file):
@@ -80,7 +80,7 @@ RETURNS integer as $$
f.write(write_char)
f.close()
- return 0
+ return []
$$ LANGUAGE plpython3u EXECUTE ON ALL SEGMENTS;
CREATE OR REPLACE FUNCTION invalidate_buffers_for_rel(tablename text) RETURNS BOOL AS
$$
@@ -130,14 +130,13 @@ select invalidate_buffers_for_rel('corrupt_table') from gp_dist_random('gp_id');
(3 rows)
-- Verify corruption on heap table
-select SUM(corrupt_file(get_relation_path('corrupt_table'), -50));
+select corrupt_file(get_relation_path('corrupt_table'), -50);
INFO: corrupting file base/81967/81931 at -50 (seg0 slice1 127.0.0.1:25432 pid=27287)
INFO: corrupting file base/81967/81931 at -50 (seg1 slice1 127.0.0.1:25433 pid=27288)
INFO: corrupting file base/81967/81931 at -50 (seg2 slice1 127.0.0.1:25434 pid=27289)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT COUNT(*) FROM corrupt_table;
WARNING: page verification failed, calculated checksum 20457 but expected 60777 (seg0 slice1 127.0.0.1:25432 pid=27287)
@@ -158,14 +157,13 @@ select invalidate_buffers_for_rel(get_toast_name('corrupt_toast_table')) from gp
(3 rows)
-- Verify corruption on toast table
-select SUM(corrupt_file(get_toast_path('corrupt_toast_table'), -50));
+select corrupt_file(get_toast_path('corrupt_toast_table'), -50);
INFO: corrupting file base/81967/81933 at -50 (seg0 slice1 127.0.0.1:25432 pid=27287)
INFO: corrupting file base/81967/81933 at -50 (seg1 slice1 127.0.0.1:25433 pid=27288)
INFO: corrupting file base/81967/81933 at -50 (seg2 slice1 127.0.0.1:25434 pid=27289)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT md5(comment::text) FROM corrupt_toast_table;
WARNING: page verification failed, calculated checksum 55109 but expected 33676 (seg0 slice1 127.0.0.1:25432 pid=27287)
@@ -187,14 +185,13 @@ select invalidate_buffers_for_rel('btree_index') from gp_dist_random('gp_id');
(3 rows)
-- Verify corruption on Btree index
-select SUM(corrupt_file(get_index_path('corrupt_btree_index'), -50));
+select corrupt_file(get_index_path('corrupt_btree_index'), -50);
INFO: corrupting file base/81967/81936 at -50 (seg0 slice1 127.0.0.1:25432 pid=27287)
INFO: corrupting file base/81967/81936 at -50 (seg1 slice1 127.0.0.1:25433 pid=27288)
INFO: corrupting file base/81967/81936 at -50 (seg2 slice1 127.0.0.1:25434 pid=27289)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
insert into corrupt_btree_index select i, 'a' from generate_series(1, 10) i; -- insert will trigger scan of the index
WARNING: page verification failed, calculated checksum 32758 but expected 56694 (seg1 127.0.0.1:25433 pid=27288)
@@ -216,14 +213,13 @@ select invalidate_buffers_for_rel('bitmap_index') from gp_dist_random('gp_id');
(3 rows)
-- Verify corruption on Bitmap index
-select SUM(corrupt_file(get_index_path('corrupt_bitmap_index'), -50));
+select corrupt_file(get_index_path('corrupt_bitmap_index'), -50);
INFO: corrupting file base/81967/81938 at -50 (seg0 slice1 127.0.0.1:25432 pid=27287)
INFO: corrupting file base/81967/81938 at -50 (seg1 slice1 127.0.0.1:25433 pid=27288)
INFO: corrupting file base/81967/81938 at -50 (seg2 slice1 127.0.0.1:25434 pid=27289)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
insert into corrupt_bitmap_index select i, 'a' from generate_series(1, 10) i; -- insert will trigger scan of the index
WARNING: page verification failed, calculated checksum 43246 but expected 2670 (seg0 127.0.0.1:25432 pid=27287)
@@ -258,14 +254,13 @@ select count(*) from mark_buffer_dirty_hint;
create table flush_xlog_page_to_disk (c int);
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'c' as the Cloudberry Database data distribution key for this table.
-- corrupt the page on disk
-select SUM(corrupt_file(get_relation_path('mark_buffer_dirty_hint'), -50));
+select corrupt_file(get_relation_path('mark_buffer_dirty_hint'), -50);
INFO: corrupting file base/81967/81939 at -50 (seg0 slice1 127.0.0.1:25432 pid=27287)
INFO: corrupting file base/81967/81939 at -50 (seg1 slice1 127.0.0.1:25433 pid=27288)
INFO: corrupting file base/81967/81939 at -50 (seg2 slice1 127.0.0.1:25434 pid=27289)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
-- invalidate buffer and confirm data is corrupted
select invalidate_buffers_for_rel('mark_buffer_dirty_hint') from gp_dist_random('gp_id');
diff --git a/src/test/heap_checksum/sql/heap_checksum_corruption.sql b/src/test/heap_checksum/sql/heap_checksum_corruption.sql
index 7defc9a615..2c95fff331 100644
--- a/src/test/heap_checksum/sql/heap_checksum_corruption.sql
+++ b/src/test/heap_checksum/sql/heap_checksum_corruption.sql
@@ -70,7 +70,7 @@ $$ language sql VOLATILE;
-- do a SUM() over the return values, and it's always 0, regardless of the
-- number of segments in the cluster.)
CREATE FUNCTION corrupt_file(data_file text, corruption_offset int4)
-RETURNS integer as $$
+RETURNS setof integer as $$
import os;
if not os.path.isfile(data_file):
@@ -90,7 +90,7 @@ RETURNS integer as $$
f.write(write_char)
f.close()
- return 0
+ return []
$$ LANGUAGE plpython3u EXECUTE ON ALL SEGMENTS;
CREATE OR REPLACE FUNCTION invalidate_buffers_for_rel(tablename text) RETURNS BOOL AS
@@ -121,7 +121,7 @@ insert into corrupt_table select i from generate_series(1, 10) i;
checkpoint;
select invalidate_buffers_for_rel('corrupt_table') from gp_dist_random('gp_id');
-- Verify corruption on heap table
-select SUM(corrupt_file(get_relation_path('corrupt_table'), -50));
+select corrupt_file(get_relation_path('corrupt_table'), -50);
SELECT COUNT(*) FROM corrupt_table;
-- Corrupt a heap table, with toast table
@@ -130,7 +130,7 @@ insert into corrupt_toast_table select i, ("decode"(repeat('a',3000000),'escape'
checkpoint;
select invalidate_buffers_for_rel(get_toast_name('corrupt_toast_table')) from gp_dist_random('gp_id');
-- Verify corruption on toast table
-select SUM(corrupt_file(get_toast_path('corrupt_toast_table'), -50));
+select corrupt_file(get_toast_path('corrupt_toast_table'), -50);
SELECT md5(comment::text) FROM corrupt_toast_table;
-- Corrupt a Btree Index
@@ -140,7 +140,7 @@ insert into corrupt_btree_index select i, 'a' from generate_series(1, 10) i;
checkpoint;
select invalidate_buffers_for_rel('btree_index') from gp_dist_random('gp_id');
-- Verify corruption on Btree index
-select SUM(corrupt_file(get_index_path('corrupt_btree_index'), -50));
+select corrupt_file(get_index_path('corrupt_btree_index'), -50);
insert into corrupt_btree_index select i, 'a' from generate_series(1, 10) i; -- insert will trigger scan of the index
-- Corrupt a Bitmap Index
@@ -150,7 +150,7 @@ insert into corrupt_bitmap_index select i, 'a' from generate_series(1, 10) i;
checkpoint;
select invalidate_buffers_for_rel('bitmap_index') from gp_dist_random('gp_id');
-- Verify corruption on Bitmap index
-select SUM(corrupt_file(get_index_path('corrupt_bitmap_index'), -50));
+select corrupt_file(get_index_path('corrupt_bitmap_index'), -50);
insert into corrupt_bitmap_index select i, 'a' from generate_series(1, 10) i; -- insert will trigger scan of the index
-- Test make sure full page image is captured in XLOG if hint bit change is the first change after checkpoint
@@ -167,7 +167,7 @@ select count(*) from mark_buffer_dirty_hint;
-- using a DML to trigger the XLogFlush() to have the backup block written
create table flush_xlog_page_to_disk (c int);
-- corrupt the page on disk
-select SUM(corrupt_file(get_relation_path('mark_buffer_dirty_hint'), -50));
+select corrupt_file(get_relation_path('mark_buffer_dirty_hint'), -50);
-- invalidate buffer and confirm data is corrupted
select invalidate_buffers_for_rel('mark_buffer_dirty_hint') from gp_dist_random('gp_id');
select count(*) from mark_buffer_dirty_hint;
diff --git a/src/test/isolation2/input/workfile_mgr_test.source b/src/test/isolation2/input/workfile_mgr_test.source
index aec099f6b4..30f03315d7 100644
--- a/src/test/isolation2/input/workfile_mgr_test.source
+++ b/src/test/isolation2/input/workfile_mgr_test.source
@@ -17,11 +17,11 @@ $$
LANGUAGE SQL;
CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_workset(worksetname text, interXact bool, holdPin bool, closeFile bool)
-RETURNS void
+RETURNS setof void
LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_empty_workset(worksetname text)
-RETURNS void
+RETURNS setof void
LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE FUNCTION gp_workfile_mgr_cache_entries()
diff --git a/src/test/isolation2/output/workfile_mgr_test.source b/src/test/isolation2/output/workfile_mgr_test.source
index 798378e28d..db30fced8c 100644
--- a/src/test/isolation2/output/workfile_mgr_test.source
+++ b/src/test/isolation2/output/workfile_mgr_test.source
@@ -7,10 +7,10 @@ CREATE
CREATE FUNCTION gp_workfile_mgr_test(testname text, numfiles int) RETURNS SETOF BOOL AS $$ SELECT C.* FROM gp_workfile_mgr_test_on_master($1, $2) as C UNION ALL SELECT C.* FROM gp_workfile_mgr_test_on_segments($1, $2) as C $$ LANGUAGE SQL;
CREATE
-CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_workset(worksetname text, interXact bool, holdPin bool, closeFile bool) RETURNS void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
+CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_workset(worksetname text, interXact bool, holdPin bool, closeFile bool) RETURNS setof void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE
-CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_empty_workset(worksetname text) RETURNS void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
+CREATE OR REPLACE FUNCTION gp_workfile_mgr_create_empty_workset(worksetname text) RETURNS setof void LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS AS '@abs_builddir@/isolation2_regress@DLSUFFIX@', 'gp_workfile_mgr_create_workset';
CREATE
CREATE FUNCTION gp_workfile_mgr_cache_entries() RETURNS TABLE(segid int4, prefix text, size int8, operation text, slice int4, sessionid int4, commandid int4, numfiles int4) AS '$libdir/gp_workfile_mgr', 'gp_workfile_mgr_cache_entries' LANGUAGE C VOLATILE EXECUTE ON ALL SEGMENTS;
diff --git a/src/test/regress/expected/ao_checksum_corruption.out b/src/test/regress/expected/ao_checksum_corruption.out
index 55da9e0c2e..8f4ceae4d3 100644
--- a/src/test/regress/expected/ao_checksum_corruption.out
+++ b/src/test/regress/expected/ao_checksum_corruption.out
@@ -42,7 +42,7 @@ $$ language sql VOLATILE;
-- do a SUM() over the return values, and it's always 0, regardless of the
-- number of segemnts in the cluster.)
CREATE FUNCTION corrupt_file(data_file text, corruption_offset int4)
-RETURNS integer as $$
+RETURNS setof integer as $$
import os;
if not os.path.isfile(data_file):
@@ -62,12 +62,12 @@ RETURNS integer as $$
f.write(write_char)
f.close()
- return 0
+ return []
$$ LANGUAGE plpython3u EXECUTE ON ALL SEGMENTS;
-- Corrupt a file by replacing the last occurrence of 'str' within the file
-- with 'replacement'
CREATE FUNCTION corrupt_file(data_file text, target bytea, replacement bytea)
-RETURNS integer as $$
+RETURNS setof integer as $$
import os;
if not os.path.isfile(data_file):
@@ -89,33 +89,31 @@ RETURNS integer as $$
f.close()
- return 0
+ return []
$$ LANGUAGE plpython3u EXECUTE ON ALL SEGMENTS;
-- Large content, corrupt block header
create table corrupt_header_large_co(comment bytea ) with (appendonly=true, orientation=column, checksum=true) DISTRIBUTED RANDOMLY;
insert into corrupt_header_large_co select ("decode"(repeat('a',33554432),'escape')) from generate_series(1,8) ;
-select SUM(corrupt_file(get_aoseg1_path('corrupt_header_large_co'), 8));
+select corrupt_file(get_aoseg1_path('corrupt_header_large_co'), 8);
INFO: corrupting file base/107062/107070.1 at 8 (seg0 slice1 127.0.0.1:40000 pid=28137)
INFO: corrupting file base/107062/107070.1 at 8 (seg1 slice1 127.0.0.1:40001 pid=28138)
INFO: corrupting file base/107062/107070.1 at 8 (seg2 slice1 127.0.0.1:40002 pid=28139)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT COUNT(*) FROM corrupt_header_large_co;
ERROR: header checksum does not match, expected 0xBA9CC649 and found 0x0C997AE7 (seg0 slice1 127.0.0.1:40000 pid=28137)
-- Large content, corrupt content
create table corrupt_content_large_co(comment bytea ) with (appendonly=true, orientation=column, checksum=true) DISTRIBUTED RANDOMLY;
insert into corrupt_content_large_co select ("decode"(repeat('a',33554432),'escape')) from generate_series(1,8) ;
-select SUM(corrupt_file(get_aoseg1_path('corrupt_content_large_co'), -3));
+select corrupt_file(get_aoseg1_path('corrupt_content_large_co'), -3);
INFO: corrupting file base/107062/107080.1 at -3 (seg0 slice1 127.0.0.1:40000 pid=28137)
INFO: corrupting file base/107062/107080.1 at -3 (seg1 slice1 127.0.0.1:40001 pid=28138)
INFO: corrupting file base/107062/107080.1 at -3 (seg2 slice1 127.0.0.1:40002 pid=28139)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT COUNT(*) FROM corrupt_content_large_co;
ERROR: block checksum does not match, expected 0x9C02F450 and found 0x0A78638C (seg0 slice1 127.0.0.1:40000 pid=28137)
@@ -123,14 +121,13 @@ ERROR: block checksum does not match, expected 0x9C02F450 and found 0x0A78638C
create table corrupt_header_small_co(a int) with (appendonly=true, orientation=column, checksum=true);
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'a' as the Cloudberry Database data distribution key for this table.
insert into corrupt_header_small_co values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_header_small_co'), 8));
+select corrupt_file(get_aoseg1_path('corrupt_header_small_co'), 8);
INFO: corrupting file base/107062/107090.1 at 8 (seg0 slice1 127.0.0.1:40000 pid=28137)
INFO: corrupting file base/107062/107090.1 at 8 (seg1 slice1 127.0.0.1:40001 pid=28138)
INFO: corrupting file base/107062/107090.1 at 8 (seg2 slice1 127.0.0.1:40002 pid=28139)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT COUNT(*) FROM corrupt_header_small_co;
ERROR: header checksum does not match, expected 0xAA39D941 and found 0xE8FB379B (seg0 slice1 127.0.0.1:40000 pid=28137)
@@ -138,14 +135,13 @@ ERROR: header checksum does not match, expected 0xAA39D941 and found 0xE8FB379B
create table corrupt_content_small_co(a int) with (appendonly=true, orientation=column, checksum=true);
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'a' as the Cloudberry Database data distribution key for this table.
insert into corrupt_content_small_co values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_content_small_co'), -3));
+select corrupt_file(get_aoseg1_path('corrupt_content_small_co'), -3);
INFO: corrupting file base/107062/107097.1 at -3 (seg0 slice1 127.0.0.1:40000 pid=28137)
INFO: corrupting file base/107062/107097.1 at -3 (seg1 slice1 127.0.0.1:40001 pid=28138)
INFO: corrupting file base/107062/107097.1 at -3 (seg2 slice1 127.0.0.1:40002 pid=28139)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT COUNT(*) FROM corrupt_content_small_co;
ERROR: block checksum does not match, expected 0xA3CAEC7A and found 0x35B07BA6 (seg0 slice1 127.0.0.1:40000 pid=28137)
@@ -153,14 +149,13 @@ ERROR: block checksum does not match, expected 0xA3CAEC7A and found 0x35B07BA6
create table corrupt_header_small_ao(a int) with (appendonly=true, orientation=row, checksum=true);
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'a' as the Cloudberry Database data distribution key for this table.
insert into corrupt_header_small_ao values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_header_small_ao'), 8));
+select corrupt_file(get_aoseg1_path('corrupt_header_small_ao'), 8);
INFO: corrupting file base/107062/107104.1 at 8 (seg0 slice1 127.0.0.1:40000 pid=28137)
INFO: corrupting file base/107062/107104.1 at 8 (seg1 slice1 127.0.0.1:40001 pid=28138)
INFO: corrupting file base/107062/107104.1 at 8 (seg2 slice1 127.0.0.1:40002 pid=28139)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT COUNT(*) FROM corrupt_header_small_ao;
ERROR: header checksum does not match, expected 0x413AF15D and found 0x805DA822 (seg0 slice1 127.0.0.1:40000 pid=28137)
@@ -168,14 +163,13 @@ ERROR: header checksum does not match, expected 0x413AF15D and found 0x805DA822
create table corrupt_content_small_ao(a int) with (appendonly=true, orientation=row, checksum=true);
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'a' as the Cloudberry Database data distribution key for this table.
insert into corrupt_content_small_ao values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_content_small_ao'), -3));
+select corrupt_file(get_aoseg1_path('corrupt_content_small_ao'), -3);
INFO: corrupting file base/107062/107111.1 at -3 (seg0 slice1 127.0.0.1:40000 pid=28137)
INFO: corrupting file base/107062/107111.1 at -3 (seg1 slice1 127.0.0.1:40001 pid=28138)
INFO: corrupting file base/107062/107111.1 at -3 (seg2 slice1 127.0.0.1:40002 pid=28139)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
SELECT COUNT(*) FROM corrupt_content_small_ao;
ERROR: block checksum does not match, expected 0x30FEE3C5 and found 0xA6847419 (seg0 slice1 127.0.0.1:40000 pid=28137)
@@ -185,14 +179,13 @@ NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 't' as
insert into appendonly_verify_block_checksums_co
select 'abcdefghijlmnopqrstuvxyz' from generate_series(1, 5);
-- Corrupt the table by flip the 'xyz' on the last row with ###
-select SUM(corrupt_file(get_aoseg1_path('appendonly_verify_block_checksums_co'), 'xyz', '###'));
+select corrupt_file(get_aoseg1_path('appendonly_verify_block_checksums_co'), 'xyz', '###');
INFO: skipping non-existent file base/107062/107118.1 (seg0 slice1 127.0.0.1:40000 pid=28137)
INFO: skipping non-existent file base/107062/107118.1 (seg1 slice1 127.0.0.1:40001 pid=28138)
INFO: corrupting file base/107062/107118.1 by replacing <type 'str'> with ### (seg2 slice1 127.0.0.1:40002 pid=28139)
- sum
------
- 0
-(1 row)
+ corrupt_file
+--------------
+(0 rows)
-- Fails, checksum is wrong.
SELECT * FROM appendonly_verify_block_checksums_co;
diff --git a/src/test/regress/expected/motion_socket.out b/src/test/regress/expected/motion_socket.out
index d9b20ac193..5128b944cb 100644
--- a/src/test/regress/expected/motion_socket.out
+++ b/src/test/regress/expected/motion_socket.out
@@ -52,7 +52,7 @@ for pid in pids_to_check:
motion_socket_count, process.connections()))
-$$ LANGUAGE plpython3u EXECUTE ON MASTER;
+$$ LANGUAGE plpython3u;
SELECT check_motion_sockets();
NOTICE: Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'i' as the Greenplum Database data distribution key for this table.
HINT: The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew.
diff --git a/src/test/regress/expected/privileges.out b/src/test/regress/expected/privileges.out
index 49c7af0c68..43a4c86696 100644
--- a/src/test/regress/expected/privileges.out
+++ b/src/test/regress/expected/privileges.out
@@ -2618,7 +2618,7 @@ AFTER INSERT ON executor
INITIALLY DEFERRED FOR EACH ROW
EXECUTE PROCEDURE executor_trig_sfunc();
-- Check "regress_sro_user" superuser privileges on segments.
-CREATE OR REPLACE FUNCTION is_superuser_on_segments(username text) RETURNS boolean AS $$
+CREATE OR REPLACE FUNCTION is_superuser_on_segments(username text) RETURNS setof boolean AS $$
SELECT rolsuper FROM pg_roles WHERE rolname = $1;
$$ LANGUAGE sql EXECUTE ON ALL SEGMENTS;
SELECT DISTINCT is_superuser_on_segments('regress_sro_user');
diff --git a/src/test/regress/expected/psql_gp_commands.out b/src/test/regress/expected/psql_gp_commands.out
index 43c3d49169..2bec92fbf3 100644
--- a/src/test/regress/expected/psql_gp_commands.out
+++ b/src/test/regress/expected/psql_gp_commands.out
@@ -242,20 +242,20 @@ comment on operator family dd_opfamily using btree is 'this is an operator famil
-- \df+ should list all exec locations
CREATE FUNCTION foofunc_exec_on_any() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON ANY;
ALTER FUNCTION foofunc_exec_on_any() OWNER TO test_psql_de_role;
-CREATE FUNCTION foofunc_exec_on_coordinator() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON COORDINATOR;
+CREATE FUNCTION foofunc_exec_on_coordinator() RETURNS setof int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON COORDINATOR;
ALTER FUNCTION foofunc_exec_on_coordinator() OWNER TO test_psql_de_role;
-CREATE FUNCTION foofunc_exec_on_all_segments() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON ALL SEGMENTS;
+CREATE FUNCTION foofunc_exec_on_all_segments() RETURNS setof int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON ALL SEGMENTS;
ALTER FUNCTION foofunc_exec_on_all_segments() OWNER TO test_psql_de_role;
-CREATE FUNCTION foofunc_exec_on_initplan() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON INITPLAN;
+CREATE FUNCTION foofunc_exec_on_initplan() RETURNS setof int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON INITPLAN;
ALTER FUNCTION foofunc_exec_on_initplan() OWNER TO test_psql_de_role;
\df+ foofunc_exec_on_*
List of functions
Schema | Name | Result data type | Argument data types | Type | Data access | Execute on | Volatility | Parallel | Owner | Security | Access privileges | Language | Source code | Description
------------------+------------------------------+------------------+---------------------+------+--------------+--------------+------------+----------+-------------------+----------+-------------------+----------+-------------+-------------
- test_psql_schema | foofunc_exec_on_all_segments | integer | | func | contains sql | all segments | volatile | unsafe | test_psql_de_role | invoker | | sql | SELECT 1 |
+ test_psql_schema | foofunc_exec_on_all_segments | SETOF integer | | func | contains sql | all segments | volatile | unsafe | test_psql_de_role | invoker | | sql | SELECT 1 |
test_psql_schema | foofunc_exec_on_any | integer | | func | contains sql | any | volatile | unsafe | test_psql_de_role | invoker | | sql | SELECT 1 |
- test_psql_schema | foofunc_exec_on_coordinator | integer | | func | contains sql | coordinator | volatile | unsafe | test_psql_de_role | invoker | | sql | SELECT 1 |
- test_psql_schema | foofunc_exec_on_initplan | integer | | func | contains sql | initplan | volatile | unsafe | test_psql_de_role | invoker | | sql | SELECT 1 |
+ test_psql_schema | foofunc_exec_on_coordinator | SETOF integer | | func | contains sql | coordinator | volatile | unsafe | test_psql_de_role | invoker | | sql | SELECT 1 |
+ test_psql_schema | foofunc_exec_on_initplan | SETOF integer | | func | contains sql | initplan | volatile | unsafe | test_psql_de_role | invoker | | sql | SELECT 1 |
(4 rows)
-- Clean up
diff --git a/src/test/regress/expected/transactions.out b/src/test/regress/expected/transactions.out
index a9f85746e3..5ed976901c 100644
--- a/src/test/regress/expected/transactions.out
+++ b/src/test/regress/expected/transactions.out
@@ -476,7 +476,7 @@ select * from xacttest;
(5 rows)
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' stable execute on coordinator;
+'select max(a) from xacttest' stable READS SQL DATA;
begin;
update xacttest set a = (select max_xacttest()) + 10 where a > 0;
select * from xacttest;
@@ -493,7 +493,7 @@ rollback;
-- start_ignore
-- But a volatile function can see the partial results of the calling query
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' volatile;
+'select max(a) from xacttest' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
select * from xacttest;
@@ -510,7 +510,7 @@ rollback;
-- end_ignore
-- Now the same test with plpgsql (since it depends on SPI which is different)
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' stable execute on coordinator;
+'begin return max(a) from xacttest; end' stable READS SQL DATA;
begin;
update xacttest set a = (select max_xacttest()) + 10 where a > 0;
select * from xacttest;
@@ -526,7 +526,7 @@ select * from xacttest;
rollback;
-- start_ignore
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' volatile;
+'begin return max(a) from xacttest; end' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
select * from xacttest;
diff --git a/src/test/regress/input/gp_dispatch_keepalives.source b/src/test/regress/input/gp_dispatch_keepalives.source
index f088a8952b..11e84b0526 100644
--- a/src/test/regress/input/gp_dispatch_keepalives.source
+++ b/src/test/regress/input/gp_dispatch_keepalives.source
@@ -26,7 +26,7 @@ CREATE OR REPLACE FUNCTION save_keepalives_data(result_table text) RETURNS void
r["keepalives_interval"], r["keepalives_count"], \
r["keepalives_idle"])
plpy.execute(stmt)
-$$ LANGUAGE plpython3u VOLATILE EXECUTE ON MASTER;
+$$ LANGUAGE plpython3u VOLATILE;
-- Before we change the keepalives parameters, create the result table to start
-- the cdbgang and to store the function call results afterwards.
diff --git a/src/test/regress/input/oid_wraparound.source b/src/test/regress/input/oid_wraparound.source
index 085a2726be..a4d3f38336 100644
--- a/src/test/regress/input/oid_wraparound.source
+++ b/src/test/regress/input/oid_wraparound.source
@@ -8,17 +8,17 @@ CREATE DATABASE oid_wraparound;
-- Create the functions that we will be using to set and observe the Oid counter
-- on the master and the segments.
CREATE OR REPLACE FUNCTION gp_set_next_oid_master(new_oid Oid)
- RETURNS VOID
+ RETURNS SETOF VOID
AS '@abs_builddir@/regress.so', 'gp_set_next_oid'
LANGUAGE C EXECUTE ON COORDINATOR;
CREATE OR REPLACE FUNCTION gp_set_next_oid_segments(new_oid Oid)
- RETURNS VOID
+ RETURNS SETOF VOID
AS '@abs_builddir@/regress.so', 'gp_set_next_oid'
LANGUAGE C EXECUTE ON ALL SEGMENTS;
CREATE OR REPLACE FUNCTION gp_get_next_oid_master()
- RETURNS OID
+ RETURNS SETOF OID
AS '@abs_builddir@/regress.so', 'gp_get_next_oid'
LANGUAGE C EXECUTE ON COORDINATOR;
diff --git a/src/test/regress/input/pgstat_qd_tabstat.source b/src/test/regress/input/pgstat_qd_tabstat.source
index 1c5e505031..f963ff1bdb 100644
--- a/src/test/regress/input/pgstat_qd_tabstat.source
+++ b/src/test/regress/input/pgstat_qd_tabstat.source
@@ -65,7 +65,7 @@ $$
delete from table_for_function where i <= 200;
end
$$
-language plpgsql volatile execute on master;
+language plpgsql volatile;
select update_table_for_function();
select pg_sleep(0.77); -- Force pgstat_report_stat() to send tabstat.
select n_tup_ins, n_tup_upd, n_tup_del, n_tup_hot_upd, n_live_tup, n_dead_tup, n_mod_since_analyze from pg_stat_all_tables_internal where relid = 'table_for_function'::regclass;
diff --git a/src/test/regress/output/gp_dispatch_keepalives.source b/src/test/regress/output/gp_dispatch_keepalives.source
index 827d39e230..878ebf5f93 100644
--- a/src/test/regress/output/gp_dispatch_keepalives.source
+++ b/src/test/regress/output/gp_dispatch_keepalives.source
@@ -24,7 +24,7 @@ CREATE OR REPLACE FUNCTION save_keepalives_data(result_table text) RETURNS void
r["keepalives_interval"], r["keepalives_count"], \
r["keepalives_idle"])
plpy.execute(stmt)
-$$ LANGUAGE plpython3u VOLATILE EXECUTE ON MASTER;
+$$ LANGUAGE plpython3u VOLATILE;
-- Before we change the keepalives parameters, create the result table to start
-- the cdbgang and to store the function call results afterwards.
CREATE TABLE keepalives_before (qe_id int2, is_writer bool,
diff --git a/src/test/regress/output/oid_wraparound.source b/src/test/regress/output/oid_wraparound.source
index 86c9a24b00..b50d3d1eee 100644
--- a/src/test/regress/output/oid_wraparound.source
+++ b/src/test/regress/output/oid_wraparound.source
@@ -7,15 +7,15 @@ CREATE DATABASE oid_wraparound;
-- Create the functions that we will be using to set and observe the Oid counter
-- on the master and the segments.
CREATE OR REPLACE FUNCTION gp_set_next_oid_master(new_oid Oid)
- RETURNS VOID
+ RETURNS SETOF VOID
AS '@abs_builddir@/regress.so', 'gp_set_next_oid'
LANGUAGE C EXECUTE ON COORDINATOR;
CREATE OR REPLACE FUNCTION gp_set_next_oid_segments(new_oid Oid)
- RETURNS VOID
+ RETURNS SETOF VOID
AS '@abs_builddir@/regress.so', 'gp_set_next_oid'
LANGUAGE C EXECUTE ON ALL SEGMENTS;
CREATE OR REPLACE FUNCTION gp_get_next_oid_master()
- RETURNS OID
+ RETURNS SETOF OID
AS '@abs_builddir@/regress.so', 'gp_get_next_oid'
LANGUAGE C EXECUTE ON COORDINATOR;
CREATE OR REPLACE FUNCTION gp_get_next_oid_segments()
diff --git a/src/test/regress/output/pgstat_qd_tabstat.source b/src/test/regress/output/pgstat_qd_tabstat.source
index 13921d748e..f1155e7583 100644
--- a/src/test/regress/output/pgstat_qd_tabstat.source
+++ b/src/test/regress/output/pgstat_qd_tabstat.source
@@ -144,7 +144,7 @@ $$
delete from table_for_function where i <= 200;
end
$$
-language plpgsql volatile execute on master;
+language plpgsql volatile;
select update_table_for_function();
update_table_for_function
---------------------------
diff --git a/src/test/regress/sql/ao_checksum_corruption.sql b/src/test/regress/sql/ao_checksum_corruption.sql
index bcc9e916ca..aea3bfa9eb 100644
--- a/src/test/regress/sql/ao_checksum_corruption.sql
+++ b/src/test/regress/sql/ao_checksum_corruption.sql
@@ -48,7 +48,7 @@ $$ language sql VOLATILE;
-- do a SUM() over the return values, and it's always 0, regardless of the
-- number of segemnts in the cluster.)
CREATE FUNCTION corrupt_file(data_file text, corruption_offset int4)
-RETURNS integer as $$
+RETURNS setof integer as $$
import os;
if not os.path.isfile(data_file):
@@ -68,13 +68,13 @@ RETURNS integer as $$
f.write(write_char)
f.close()
- return 0
+ return []
$$ LANGUAGE plpython3u EXECUTE ON ALL SEGMENTS;
-- Corrupt a file by replacing the last occurrence of 'str' within the file
-- with 'replacement'
CREATE FUNCTION corrupt_file(data_file text, target bytea, replacement bytea)
-RETURNS integer as $$
+RETURNS setof integer as $$
import os;
if not os.path.isfile(data_file):
@@ -96,49 +96,49 @@ RETURNS integer as $$
f.close()
- return 0
+ return []
$$ LANGUAGE plpython3u EXECUTE ON ALL SEGMENTS;
-- Large content, corrupt block header
create table corrupt_header_large_co(comment bytea ) with (appendonly=true, orientation=column, checksum=true) DISTRIBUTED RANDOMLY;
insert into corrupt_header_large_co select ("decode"(repeat('a',33554432),'escape')) from generate_series(1,8) ;
-select SUM(corrupt_file(get_aoseg1_path('corrupt_header_large_co'), 8));
+select corrupt_file(get_aoseg1_path('corrupt_header_large_co'), 8);
SELECT COUNT(*) FROM corrupt_header_large_co;
-- Large content, corrupt content
create table corrupt_content_large_co(comment bytea ) with (appendonly=true, orientation=column, checksum=true) DISTRIBUTED RANDOMLY;
insert into corrupt_content_large_co select ("decode"(repeat('a',33554432),'escape')) from generate_series(1,8) ;
-select SUM(corrupt_file(get_aoseg1_path('corrupt_content_large_co'), -3));
+select corrupt_file(get_aoseg1_path('corrupt_content_large_co'), -3);
SELECT COUNT(*) FROM corrupt_content_large_co;
-- Small content, corrupt block header
create table corrupt_header_small_co(a int) with (appendonly=true, orientation=column, checksum=true);
insert into corrupt_header_small_co values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_header_small_co'), 8));
+select corrupt_file(get_aoseg1_path('corrupt_header_small_co'), 8);
SELECT COUNT(*) FROM corrupt_header_small_co;
-- Small content, corrupt content
create table corrupt_content_small_co(a int) with (appendonly=true, orientation=column, checksum=true);
insert into corrupt_content_small_co values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_content_small_co'), -3));
+select corrupt_file(get_aoseg1_path('corrupt_content_small_co'), -3);
SELECT COUNT(*) FROM corrupt_content_small_co;
-- Row-oriented, Small content, corrupt block header
create table corrupt_header_small_ao(a int) with (appendonly=true, orientation=row, checksum=true);
insert into corrupt_header_small_ao values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_header_small_ao'), 8));
+select corrupt_file(get_aoseg1_path('corrupt_header_small_ao'), 8);
SELECT COUNT(*) FROM corrupt_header_small_ao;
-- Row-oriented, Small content, corrupt content
create table corrupt_content_small_ao(a int) with (appendonly=true, orientation=row, checksum=true);
insert into corrupt_content_small_ao values (1),(1),(1),(-1),(1),(1),(1),(2),(2),(2),(2),(2),(2),(2),(33),(3),(3),(3),(1),(8),(19),(20),(31),(32),(33),(34),(5),(5),(5),(5),(5),(6),(6),(6),(6),(6),(6),(7),(7),(7),(7),(7),(7),(7),(7), (null),(7),(7),(7),(null),(8),(8),(8),(8),(8),(8),(4),(4),(null),(4),(17),(17),(17),(null),(null),(null);
-select SUM(corrupt_file(get_aoseg1_path('corrupt_content_small_ao'), -3));
+select corrupt_file(get_aoseg1_path('corrupt_content_small_ao'), -3);
SELECT COUNT(*) FROM corrupt_content_small_ao;
@@ -149,7 +149,7 @@ insert into appendonly_verify_block_checksums_co
select 'abcdefghijlmnopqrstuvxyz' from generate_series(1, 5);
-- Corrupt the table by flip the 'xyz' on the last row with ###
-select SUM(corrupt_file(get_aoseg1_path('appendonly_verify_block_checksums_co'), 'xyz', '###'));
+select corrupt_file(get_aoseg1_path('appendonly_verify_block_checksums_co'), 'xyz', '###');
-- Fails, checksum is wrong.
SELECT * FROM appendonly_verify_block_checksums_co;
diff --git a/src/test/regress/sql/motion_socket.sql b/src/test/regress/sql/motion_socket.sql
index 92210e1cb1..90ba30dd32 100644
--- a/src/test/regress/sql/motion_socket.sql
+++ b/src/test/regress/sql/motion_socket.sql
@@ -54,5 +54,5 @@ for pid in pids_to_check:
motion_socket_count, process.connections()))
-$$ LANGUAGE plpython3u EXECUTE ON MASTER;
+$$ LANGUAGE plpython3u;
SELECT check_motion_sockets();
diff --git a/src/test/regress/sql/privileges.sql b/src/test/regress/sql/privileges.sql
index c175f28b65..a389444811 100644
--- a/src/test/regress/sql/privileges.sql
+++ b/src/test/regress/sql/privileges.sql
@@ -1562,7 +1562,7 @@ INITIALLY DEFERRED FOR EACH ROW
EXECUTE PROCEDURE executor_trig_sfunc();
-- Check "regress_sro_user" superuser privileges on segments.
-CREATE OR REPLACE FUNCTION is_superuser_on_segments(username text) RETURNS boolean AS $$
+CREATE OR REPLACE FUNCTION is_superuser_on_segments(username text) RETURNS setof boolean AS $$
SELECT rolsuper FROM pg_roles WHERE rolname = $1;
$$ LANGUAGE sql EXECUTE ON ALL SEGMENTS;
diff --git a/src/test/regress/sql/psql_gp_commands.sql b/src/test/regress/sql/psql_gp_commands.sql
index 8c0d9843da..5341197d3d 100644
--- a/src/test/regress/sql/psql_gp_commands.sql
+++ b/src/test/regress/sql/psql_gp_commands.sql
@@ -125,11 +125,11 @@ comment on operator family dd_opfamily using btree is 'this is an operator famil
-- \df+ should list all exec locations
CREATE FUNCTION foofunc_exec_on_any() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON ANY;
ALTER FUNCTION foofunc_exec_on_any() OWNER TO test_psql_de_role;
-CREATE FUNCTION foofunc_exec_on_coordinator() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON COORDINATOR;
+CREATE FUNCTION foofunc_exec_on_coordinator() RETURNS setof int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON COORDINATOR;
ALTER FUNCTION foofunc_exec_on_coordinator() OWNER TO test_psql_de_role;
-CREATE FUNCTION foofunc_exec_on_all_segments() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON ALL SEGMENTS;
+CREATE FUNCTION foofunc_exec_on_all_segments() RETURNS setof int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON ALL SEGMENTS;
ALTER FUNCTION foofunc_exec_on_all_segments() OWNER TO test_psql_de_role;
-CREATE FUNCTION foofunc_exec_on_initplan() RETURNS int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON INITPLAN;
+CREATE FUNCTION foofunc_exec_on_initplan() RETURNS setof int AS 'SELECT 1' LANGUAGE SQL EXECUTE ON INITPLAN;
ALTER FUNCTION foofunc_exec_on_initplan() OWNER TO test_psql_de_role;
\df+ foofunc_exec_on_*
diff --git a/src/test/regress/sql/transactions.sql b/src/test/regress/sql/transactions.sql
index 7fc230a1b8..4df2f73ff8 100644
--- a/src/test/regress/sql/transactions.sql
+++ b/src/test/regress/sql/transactions.sql
@@ -285,7 +285,7 @@ COMMIT;
select * from xacttest;
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' stable execute on coordinator;
+'select max(a) from xacttest' stable READS SQL DATA;
begin;
update xacttest set a = (select max_xacttest()) + 10 where a > 0;
@@ -299,7 +299,7 @@ rollback;
-- But a volatile function can see the partial results of the calling query
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' volatile;
+'select max(a) from xacttest' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
@@ -309,7 +309,7 @@ rollback;
-- Now the same test with plpgsql (since it depends on SPI which is different)
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' stable execute on coordinator;
+'begin return max(a) from xacttest; end' stable READS SQL DATA;
begin;
update xacttest set a = (select max_xacttest()) + 10 where a > 0;
@@ -322,7 +322,7 @@ rollback;
-- when it's evaluated. So, it can't be run on QD process.
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' volatile;
+'begin return max(a) from xacttest; end' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
diff --git a/src/test/singlenode_regress/expected/transactions.out b/src/test/singlenode_regress/expected/transactions.out
index 3675f8fb6b..423152950e 100644
--- a/src/test/singlenode_regress/expected/transactions.out
+++ b/src/test/singlenode_regress/expected/transactions.out
@@ -473,9 +473,9 @@ select * from xacttest;
(5 rows)
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' stable;
+'select max(a) from xacttest' stable READS SQL DATA;
begin;
-update xacttest set a = max_xacttest() + 10 where a > 0;
+update xacttest set a = (select max_xacttest()) + 10 where a > 0;
select * from xacttest;
a | b
-----+---------
@@ -487,9 +487,10 @@ select * from xacttest;
(5 rows)
rollback;
+-- start_ignore
-- But a volatile function can see the partial results of the calling query
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' volatile;
+'select max(a) from xacttest' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
select * from xacttest;
@@ -503,11 +504,12 @@ select * from xacttest;
(5 rows)
rollback;
+-- end_ignore
-- Now the same test with plpgsql (since it depends on SPI which is different)
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' stable;
+'begin return max(a) from xacttest; end' stable READS SQL DATA;
begin;
-update xacttest set a = max_xacttest() + 10 where a > 0;
+update xacttest set a = (select max_xacttest()) + 10 where a > 0;
select * from xacttest;
a | b
-----+---------
@@ -519,8 +521,9 @@ select * from xacttest;
(5 rows)
rollback;
+-- start_ignore
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' volatile;
+'begin return max(a) from xacttest; end' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
select * from xacttest;
@@ -534,6 +537,7 @@ select * from xacttest;
(5 rows)
rollback;
+-- end_ignore
-- test case for problems with dropping an open relation during abort
BEGIN;
savepoint x;
diff --git a/src/test/singlenode_regress/sql/transactions.sql b/src/test/singlenode_regress/sql/transactions.sql
index 8886280c0a..5b039f43e3 100644
--- a/src/test/singlenode_regress/sql/transactions.sql
+++ b/src/test/singlenode_regress/sql/transactions.sql
@@ -282,38 +282,50 @@ COMMIT;
select * from xacttest;
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' stable;
+'select max(a) from xacttest' stable READS SQL DATA;
begin;
-update xacttest set a = max_xacttest() + 10 where a > 0;
+update xacttest set a = (select max_xacttest()) + 10 where a > 0;
select * from xacttest;
rollback;
+-- start_ignore
+-- CBDB: The following test expects to evaluate max_xacttest() for each
+-- updated tuple. The result of max_xacttest() is updated every time
+-- when it's evaluated. So, it can't be run on QD process.
+
-- But a volatile function can see the partial results of the calling query
create or replace function max_xacttest() returns smallint language sql as
-'select max(a) from xacttest' volatile;
+'select max(a) from xacttest' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
select * from xacttest;
rollback;
+-- end_ignore
-- Now the same test with plpgsql (since it depends on SPI which is different)
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' stable;
+'begin return max(a) from xacttest; end' stable READS SQL DATA;
begin;
-update xacttest set a = max_xacttest() + 10 where a > 0;
+update xacttest set a = (select max_xacttest()) + 10 where a > 0;
select * from xacttest;
rollback;
+-- start_ignore
+-- CBDB: The following test expects to evaluate max_xacttest() for each
+-- updated tuple. The result of max_xacttest() is updated every time
+-- when it's evaluated. So, it can't be run on QD process.
+
create or replace function max_xacttest() returns smallint language plpgsql as
-'begin return max(a) from xacttest; end' volatile;
+'begin return max(a) from xacttest; end' volatile READS SQL DATA;
begin;
update xacttest set a = max_xacttest() + 10 where a > 0;
select * from xacttest;
rollback;
+-- end_ignore
-- test case for problems with dropping an open relation during abort
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 16/23: Fix small typos in config parameters description. (#13697)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 0ea097e2f1ce1c57a060040038dae8d4954f8eeb
Author: Anton Kurochkin <45...@users.noreply.github.com>
AuthorDate: Thu Jun 23 08:49:03 2022 +0300
Fix small typos in config parameters description. (#13697)
Fix small typos in parameters description in guc_gp.c
---
src/backend/utils/misc/guc_gp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/utils/misc/guc_gp.c b/src/backend/utils/misc/guc_gp.c
index c1ef312f75..99c75aba25 100644
--- a/src/backend/utils/misc/guc_gp.c
+++ b/src/backend/utils/misc/guc_gp.c
@@ -2883,7 +2883,7 @@ struct config_bool ConfigureNamesBool_gp[] =
{
{"create_restartpoint_on_ckpt_record_replay", PGC_SIGHUP, DEVELOPER_OPTIONS,
- gettext_noop("create a restartpoint only on mirror immediately after replaying a checkpoint record."),
+ gettext_noop("Creates a restartpoint only on mirror immediately after replaying a checkpoint record."),
NULL
},
&create_restartpoint_on_ckpt_record_replay,
@@ -4738,7 +4738,7 @@ struct config_string ConfigureNamesString_gp[] =
{
{"gp_default_storage_options", PGC_USERSET, APPENDONLY_TABLES,
- gettext_noop("default options for appendonly storage."),
+ gettext_noop("Sets the default options for appendonly storage."),
NULL,
GUC_NOT_IN_SAMPLE
},
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 07/23: Check recoveryTargetAction before put PM_STATUS_STANDBY
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 4f08c7fdac7d3922e7c630940090ec015c75c933
Author: Kate Dontsova <kd...@vmware.com>
AuthorDate: Wed May 11 17:03:26 2022 -0700
Check recoveryTargetAction before put PM_STATUS_STANDBY
Earlier https://github.com/greenplum-db/gpdb-postgres-merge/commit/67a589c6ec2467256a33075d0be9fad1b46a1514
introduced checking for promote_trigger_file before
put PM_STATUS_STANDBY into postmaster.pid as if we intend
to promote - PM_STATUS_READY will be written when promotion
completes. However promote_trigger_file is not the only option
to request promotion. In case of recovery setting recovery_target_action
configuration parameter to 'promote' will also result
into promotion after recovery is completed.
Scenario to reproduce:
- enable wal_archiving
- take pg_basebackup
- insert some data and call gp_create_restore_point() + gp_switch_log()
- restore the cluster from backup, specify recovery_target_name to
your restore point, recovery_target_action = promote and hot_standby=off
- start the cluster
- gpstart will fail with [CRITICAL]:-gpstart failed.
(Reason='FATAL: the database system is starting up...)
because the underlying pg_ctl start will return early
and the following attempt to connect fails.
---
src/backend/postmaster/postmaster.c | 19 +++++--
.../recovery/t/103_promotion_wait_status_ready.pl | 59 ++++++++++++++++++++++
2 files changed, 74 insertions(+), 4 deletions(-)
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index eadb8a4224..e70e91b33f 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -5786,7 +5786,7 @@ sigusr1_handler(SIGNAL_ARGS)
pmState == PM_STARTUP && Shutdown == NoShutdown)
{
/* WAL redo has started. We're out of reinitialization. */
- bool promote_trigger_file_exist;
+ bool promotion_requested = false;
FatalError = false;
AbortStartTime = 0;
@@ -5813,21 +5813,32 @@ sigusr1_handler(SIGNAL_ARGS)
* PM_STATUS_STANDBY, instead wish pg_ctl -w to wait till
* connections can be actually accepted by the database.
*/
- promote_trigger_file_exist = false;
if (PromoteTriggerFile != NULL && strcmp(PromoteTriggerFile, "") != 0)
{
struct stat stat_buf;
if (stat(PromoteTriggerFile, &stat_buf) == 0)
- promote_trigger_file_exist = true;
+ promotion_requested = true;
}
+ /*
+ * GPDB: Setting recovery_target_action
+ * configuration parameter to 'promote' will also result
+ * into promotion after recovery is completed.
+ */
+ if (recoveryTargetAction == RECOVERY_TARGET_ACTION_PROMOTE)
+ promotion_requested = true;
/*
* If we aren't planning to enter hot standby mode later, treat
* RECOVERY_STARTED as meaning we're out of startup, and report status
* accordingly.
+ *
+ * GPDB: Avoid PM_STATUS_STANDBY if promotion requested as wish "pg_ctl -w"
+ * to wait till connections can be actually accepted by the database via
+ * PM_STATUS_READY state instead. PM_STATUS_STANDBY will incorrectly show
+ * database is ready to accept connections during promotion.
*/
- if (!EnableHotStandby && !promote_trigger_file_exist)
+ if (!EnableHotStandby && !promotion_requested)
{
AddToDataDirLockFile(LOCK_FILE_LINE_PM_STATUS, PM_STATUS_STANDBY);
#ifdef USE_SYSTEMD
diff --git a/src/test/recovery/t/103_promotion_wait_status_ready.pl b/src/test/recovery/t/103_promotion_wait_status_ready.pl
new file mode 100644
index 0000000000..db573377f9
--- /dev/null
+++ b/src/test/recovery/t/103_promotion_wait_status_ready.pl
@@ -0,0 +1,59 @@
+# Checks that if promotion is requested in recovery_target_action
+# we are waiting for recovery to complete before let pg_ctl to return
+use strict;
+use warnings;
+
+use PostgresNode;
+use TestLib;
+use Test::More tests => 1;
+
+# Initialize node to backup
+my $node_to_backup = get_new_node('to_backup');
+$node_to_backup->init(
+ has_archiving => 1,
+ allows_streaming => 1);
+$node_to_backup->start;
+
+# And some content
+$node_to_backup->safe_psql('postgres',
+ "CREATE TABLE tab_1 AS SELECT generate_series(1, 10) AS a1");
+$node_to_backup->safe_psql('postgres',
+ "SELECT pg_create_restore_point('first_restore_point');");
+$node_to_backup->safe_psql('postgres',
+ "SELECT pg_switch_wal();");
+
+# Take backup
+my $backup_name = 'my_backup';
+$node_to_backup->backup($backup_name);
+
+# Before running the transaction get the
+# current timestamp that will be used as a comparison base.
+my $trx_to_archive_time = time();
+# Add some more content, create restore point and switch WAL to make sure it's archived
+$node_to_backup->safe_psql('postgres',
+ "CREATE TABLE tab_2 AS SELECT generate_series(1, 10) AS a2");
+$node_to_backup->safe_psql('postgres',
+ "SELECT pg_create_restore_point('second_restore_point');");
+$node_to_backup->safe_psql('postgres',
+ "SELECT pg_switch_wal();");
+
+# Create new node from from backup
+my $node_restored = get_new_node('restored');
+my $delay = 5;
+$node_restored->init_from_backup($node_to_backup, $backup_name,
+ standby => 1, has_restoring => 1);
+$node_restored->append_conf(
+ 'postgresql.conf', qq(
+recovery_target_name = 'second_restore_point'
+recovery_target_action = 'promote'
+recovery_min_apply_delay = '${delay}s'
+hot_standby = 'off'
+));
+$node_restored->start;
+
+#Promotion can happen only after the recovery process spends
+# recovery_min_apply_delay seconds en route to the recovery_target_name.
+#Thus, this check ensures that pg_ctl waited for promotion to have completed before returning.
+
+ok(time() - $trx_to_archive_time >= $delay,
+ "pg_ctl starts restored node only after replication delay if recovery_target_action = 'promote' is specified");
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 18/23: Fix gppkg's coordinator hook not execute.
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 88b431ce26108c5f8240bdd5823fb86127e7970d
Author: Sasasu <i...@sasa.su>
AuthorDate: Wed Jun 22 15:11:33 2022 +0800
Fix gppkg's coordinator hook not execute.
old (and current) gppkg create the package like this format:
```
PostInstall:
- Master: "sh ./master.sh"
- Segment: "sh ./segment.sh';"
- All: "echo 'install successful'"
```
sed -i s/master/coordinator/g will break this behavior.
---
gpMgmt/bin/gppylib/operations/package.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/gpMgmt/bin/gppylib/operations/package.py b/gpMgmt/bin/gppylib/operations/package.py
index f1e7356384..0b0ece2b47 100644
--- a/gpMgmt/bin/gppylib/operations/package.py
+++ b/gpMgmt/bin/gppylib/operations/package.py
@@ -1141,7 +1141,7 @@ class PerformHooks(Operation):
if key is None:
return
key_str = key[0]
- if key_str.lower() == 'coordinator':
+ if key_str.lower() in ('coordinator', 'master'):
if self.standby_host:
RemoteCommand(hook[key_str], [self.standby_host]).run()
LocalCommand(hook[key_str], True).run()
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 01/23: Add distributed snapshot support to pg_export_snapshot
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 04d7ba708c3dc91bb854acdb994f810759d29650
Author: Brent Doil <bd...@vmware.com>
AuthorDate: Thu Feb 24 11:35:27 2022 -0500
Add distributed snapshot support to pg_export_snapshot
This commit adds distributed snapshot metadata when exporting and
importing snapshots.
Calling `select pg_export_snapshot()` from the QD will write the
following fields to the snapshot file in `pg_snapshots` in the
coordinator data directory.
```
dsxminall
dsid
dsxmin
dsxmax
dscnt
dsxip
```
When a snapshot is imported via SET TRANSACTION SNAPSHOT from the QD, subsequent
queries will have the distributed snapshot metadata dispatched to the QEs.
This enables cluster-wide data visibility consistency.
Co-authored-by: Soumyadeep Chakraborty <so...@gmail.com>
Co-authored-by: Kate Dontsova <ed...@pivotal.io>
Co-authored-by: Brent Doil <bd...@vmware.com>
---
src/backend/utils/time/snapmgr.c | 100 ++++++-
.../expected/export_distributed_snapshot.out | 290 +++++++++++++++++++++
src/test/isolation2/isolation2_schedule | 2 +
.../isolation2/sql/export_distributed_snapshot.sql | 198 ++++++++++++++
4 files changed, 589 insertions(+), 1 deletion(-)
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index befb2a491b..3050735dc2 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -74,6 +74,8 @@
#include "utils/syscache.h"
#include "utils/timestamp.h"
+#include "cdb/cdbdisp_query.h"
+#include "cdb/cdbdispatchresult.h"
#include "cdb/cdbtm.h"
#include "cdb/cdbvars.h"
#include "utils/guc.h"
@@ -590,7 +592,13 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid,
* CurrentSnapshotData's XID arrays have been allocated, and (2) to update
* the state for GlobalVis*.
*/
- CurrentSnapshot = GetSnapshotData(&CurrentSnapshotData, DistributedTransactionContext);
+
+ /*
+ * Pass in DTX_CONTEXT_LOCAL_ONLY to prevent a distributed snapshot from
+ * getting created here, as we will be using the one we parsed in
+ * ImportSnapshot.
+ */
+ CurrentSnapshot = GetSnapshotData(&CurrentSnapshotData, DTX_CONTEXT_LOCAL_ONLY);
/*
* Now copy appropriate fields from the source snapshot.
@@ -609,6 +617,16 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid,
sourcesnap->subxcnt * sizeof(TransactionId));
CurrentSnapshot->suboverflowed = sourcesnap->suboverflowed;
CurrentSnapshot->takenDuringRecovery = sourcesnap->takenDuringRecovery;
+
+ /*
+ * GPDB: Copy over distributed snapshot if present.
+ */
+ if (sourcesnap->haveDistribSnapshot)
+ {
+ CurrentSnapshot->haveDistribSnapshot = true;
+ DistributedSnapshot_Copy(&CurrentSnapshot->distribSnapshotWithLocalMapping.ds,
+ &sourcesnap->distribSnapshotWithLocalMapping.ds);
+ }
/* NB: curcid should NOT be copied, it's a local matter */
CurrentSnapshot->snapXactCompletionCount = 0;
@@ -1243,6 +1261,7 @@ ExportSnapshot(Snapshot snapshot)
char path[MAXPGPATH];
char pathtmp[MAXPGPATH];
+ DistributedSnapshot *distributed_snapshot;
/*
* It's tempting to call RequireTransactionBlock here, since it's not very
* useful to export a snapshot that will disappear immediately afterwards.
@@ -1357,6 +1376,21 @@ ExportSnapshot(Snapshot snapshot)
}
appendStringInfo(&buf, "rec:%u\n", snapshot->takenDuringRecovery);
+ /*
+ * GPDB: Serialize distributed snapshot if present.
+ */
+ if (snapshot->haveDistribSnapshot)
+ {
+ distributed_snapshot = &snapshot->distribSnapshotWithLocalMapping.ds;
+ appendStringInfo(&buf, "dsxminall:%lu\n", distributed_snapshot->xminAllDistributedSnapshots);
+ appendStringInfo(&buf, "dsid:%d\n", distributed_snapshot->distribSnapshotId);
+ appendStringInfo(&buf, "dsxmin:%lu\n", distributed_snapshot->xmin);
+ appendStringInfo(&buf, "dsxmax:%lu\n", distributed_snapshot->xmax);
+ appendStringInfo(&buf, "dsxcnt:%d\n", distributed_snapshot->count);
+ for (i = 0; i < distributed_snapshot->count; i++)
+ appendStringInfo(&buf, "dsxip:%lu\n", distributed_snapshot->inProgressXidArray[i]);
+ }
+
/*
* Now write the text representation into a file. We first write to a
* ".tmp" filename, and rename to final filename if no error. This
@@ -1469,6 +1503,31 @@ parseXidFromText(const char *prefix, char **s, const char *filename)
return val;
}
+static DistributedTransactionId
+parseDistributedXidFromText(const char *prefix, char **s, const char *filename)
+{
+ char *ptr = *s;
+ int prefixlen = strlen(prefix);
+ DistributedTransactionId val;
+
+ if (strncmp(ptr, prefix, prefixlen) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid snapshot data in file \"%s\"", filename)));
+ ptr += prefixlen;
+ if (sscanf(ptr, "%lu", &val) != 1)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid snapshot data in file \"%s\"", filename)));
+ ptr = strchr(ptr, '\n');
+ if (!ptr)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid snapshot data in file \"%s\"", filename)));
+ *s = ptr + 1;
+ return val;
+}
+
static void
parseVxidFromText(const char *prefix, char **s, const char *filename,
VirtualTransactionId *vxid)
@@ -1507,6 +1566,7 @@ ImportSnapshot(const char *idstr)
struct stat stat_buf;
char *filebuf;
int xcnt;
+ int dxcnt;
int i;
VirtualTransactionId src_vxid;
int src_pid;
@@ -1514,6 +1574,7 @@ ImportSnapshot(const char *idstr)
int src_isolevel;
bool src_readonly;
SnapshotData snapshot;
+ DistributedSnapshot *distributed_snapshot;
/*
* Must be at top level of a fresh transaction. Note in particular that
@@ -1621,6 +1682,43 @@ ImportSnapshot(const char *idstr)
snapshot.takenDuringRecovery = parseIntFromText("rec:", &filebuf, path);
+ /*
+ * GPDB: Extract distributed snapshot
+ * Importing a distributed snapshot in utility mode is not allowed because
+ * functionality to dispatch pg_export_snapshot to all segments and create
+ * a snapshot with ds fields in each segments datadir is not implemented.
+ * Since there is no reliable way to export a utility mode distributed snapshot,
+ * we have no way to judge its provenance.
+ */
+ if(*filebuf != '\0')
+ {
+ if (Gp_role == GP_ROLE_UTILITY) {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("cannot import distributed snapshot in utility mode"),
+ errhint("export the snapshot in utility mode")));
+ }
+ distributed_snapshot = &snapshot.distribSnapshotWithLocalMapping.ds;
+ distributed_snapshot->xminAllDistributedSnapshots = parseDistributedXidFromText("dsxminall:", &filebuf, path);
+ distributed_snapshot->distribSnapshotId = parseIntFromText("dsid:", &filebuf, path);
+ distributed_snapshot->xmin = parseDistributedXidFromText("dsxmin:", &filebuf, path);
+ distributed_snapshot->xmax = parseDistributedXidFromText("dsxmax:", &filebuf, path);
+ distributed_snapshot->count = dxcnt = parseIntFromText("dsxcnt:", &filebuf, path);
+
+ /* sanity-check dsxmin and the xid count before palloc */
+ if (distributed_snapshot->xmin < distributed_snapshot->xminAllDistributedSnapshots ||
+ (dxcnt < 0 || dxcnt > GetMaxSnapshotDistributedXidCount())
+ )
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
+ errmsg("invalid snapshot data in file \"%s\"", path)));
+
+ distributed_snapshot->inProgressXidArray = (DistributedTransactionId *) palloc(dxcnt * sizeof(DistributedTransactionId));
+ for (i = 0; i < dxcnt; i++)
+ distributed_snapshot->inProgressXidArray[i] = parseDistributedXidFromText("dsxip:", &filebuf, path);
+ snapshot.haveDistribSnapshot = true;
+ }
+
/*
* Do some additional sanity checking, just to protect ourselves. We
* don't trouble to check the array elements, just the most critical
diff --git a/src/test/isolation2/expected/export_distributed_snapshot.out b/src/test/isolation2/expected/export_distributed_snapshot.out
new file mode 100644
index 0000000000..130b319551
--- /dev/null
+++ b/src/test/isolation2/expected/export_distributed_snapshot.out
@@ -0,0 +1,290 @@
+-- Export distributed snapshot tests
+
+-- start_matchsubs
+-- m/[0-9a-fA-F]+-[0-9a-fA-F]+-\d/
+-- s/[0-9a-fA-F]+-[0-9a-fA-F]+-\d/#######-########-#/
+
+-- m/^DETAIL: The source process with PID \d+ is not running anymore./
+-- s/^DETAIL: The source process with PID \d+ is not running anymore./DETAIL: The source process with PID is not running anymore./
+-- end_matchsubs
+
+-- start_ignore
+DROP FUNCTION IF EXISTS corrupt_snapshot_file(text, text);
+DROP
+DROP FUNCTION IF EXISTS snapshot_file_ds_fields_exist(text);
+DROP
+DROP LANGUAGE IF EXISTS plpython3u;
+DROP
+DROP TABLE IF EXISTS export_distributed_snapshot_test1;
+DROP
+
+CREATE LANGUAGE plpython3u;
+CREATE
+-- end_ignore
+
+-- Corrupt field entry for given snapshot file
+CREATE OR REPLACE FUNCTION corrupt_snapshot_file(token text, field text) RETURNS integer as $$ import os content = bytearray() query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path" rv = plpy.execute(query) abs_path = rv[0]['path'] snapshot_file = abs_path + token if not os.path.isfile(snapshot_file): plpy.info('skipping non-existent file %s' % (snapshot_file)) else: plpy.info('corrupting file %s for field %s' % (snapsho [...]
+CREATE
+
+-- Determine if field exists for given snapshot file
+CREATE OR REPLACE FUNCTION snapshot_file_ds_fields_exist(token text) RETURNS boolean as $$ import os content = bytearray() query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path" rv = plpy.execute(query) abs_path = rv[0]['path'] snapshot_file = abs_path + token if not os.path.isfile(snapshot_file): plpy.info('snapshot file %s does not exist' % (snapshot_file)) return -1 else: plpy.info('checking file %s for ds fields' % ( [...]
+CREATE
+
+-- INSERT test
+CREATE TABLE export_distributed_snapshot_test1 (a int);
+CREATE
+INSERT INTO export_distributed_snapshot_test1 values(1);
+INSERT 1
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000007-00000011-1
+(1 row)
+
+INSERT INTO export_distributed_snapshot_test1 values(2);
+INSERT 1
+SELECT * FROM export_distributed_snapshot_test1;
+ a
+---
+ 2
+ 1
+(2 rows)
+
+-- Transaction 2 should return 2 rows
+2: SELECT * FROM export_distributed_snapshot_test1;
+ a
+---
+ 2
+ 1
+(2 rows)
+2: COMMIT;
+COMMIT
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+SET
+-- Transaction 2 should now return 1 row
+2: SELECT * FROM export_distributed_snapshot_test1;
+ a
+---
+ 1
+(1 row)
+2: COMMIT;
+COMMIT
+
+1: COMMIT;
+COMMIT
+
+-- DROP test
+CREATE TABLE export_distributed_snapshot_test2 (a int);
+CREATE
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+2: BEGIN;
+BEGIN
+
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000007-00000012-1
+(1 row)
+1: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+ gp_segment_id | relname
+---------------+-----------------------------------
+ 1 | export_distributed_snapshot_test2
+ 2 | export_distributed_snapshot_test2
+ 0 | export_distributed_snapshot_test2
+(3 rows)
+
+-- Drop table in transaction
+2: DROP TABLE export_distributed_snapshot_test2;
+DROP
+2: COMMIT;
+COMMIT
+
+3: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+3: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+SET
+-- The table should still be visible to transaction 3 using transaction 1's snapshot.
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+ gp_segment_id | relname
+---------------+-----------------------------------
+ 1 | export_distributed_snapshot_test2
+ 2 | export_distributed_snapshot_test2
+ 0 | export_distributed_snapshot_test2
+(3 rows)
+3: COMMIT;
+COMMIT
+-- The table should no longer be visible to transaction 3.
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+ gp_segment_id | relname
+---------------+---------
+(0 rows)
+
+1: COMMIT;
+COMMIT
+
+-- Test corrupt fields in snapshot file
+
+-- xmin
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000007-00000013-1
+(1 row)
+
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select corrupt_snapshot_file('@TOKEN', 'xmin');
+ corrupt_snapshot_file
+-----------------------
+ 0
+(1 row)
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+ERROR: could not import the requested snapshot
+DETAIL: The source process with PID 651456 is not running anymore.
+
+1: END;
+END
+2: END;
+END
+
+-- dsxminall
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000007-00000014-1
+(1 row)
+
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select corrupt_snapshot_file('@TOKEN', 'dsxminall');
+ corrupt_snapshot_file
+-----------------------
+ 0
+(1 row)
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+SET
+
+1: END;
+END
+2: END;
+END
+
+-- dsxmin
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000007-00000006-1
+(1 row)
+
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select corrupt_snapshot_file('@TOKEN', 'dsxmin');
+ corrupt_snapshot_file
+-----------------------
+ 0
+(1 row)
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+ERROR: invalid snapshot data in file "pg_snapshots/00000007-00000006-1"
+
+1: END;
+END
+2: END;
+END
+
+-- Test export snapshot in utility mode does not export distributed snapshot fields
+
+-1U: BEGIN;
+BEGIN
+-1U: BEGIN;
+BEGIN
+-1U: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000009-00000302-1
+(1 row)
+
+-- Should return false
+1: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select snapshot_file_ds_fields_exist('@TOKEN');
+ snapshot_file_ds_fields_exist
+-------------------------------
+ f
+(1 row)
+
+-1Uq: ... <quitting>
+1: END;
+END
+
+-- Test import snapshot in utility mode fails if distributed snapshot fields exist
+1: BEGIN;
+BEGIN
+1: BEGIN;
+BEGIN
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000006-00000008-1
+(1 row)
+
+-- Open utility mode connection on coordinator and set snapshot
+-1U: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+BEGIN
+-1U: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+ERROR: cannot import distributed snapshot in utility mode
+HINT: export the snapshot in utility mode
+
+-1Uq: ... <quitting>
+1: END;
+END
+
+-- Test export snapshot in utility mode and import snapshot in utility mode succeeds
+-1U: @db_name postgres: BEGIN;
+BEGIN
+-1U: BEGIN;
+BEGIN
+-1U: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+ pg_export_snapshot
+---------------------
+ 00000009-0000030A-1
+(1 row)
+
+-- Should return false
+1: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select snapshot_file_ds_fields_exist('@TOKEN');
+ snapshot_file_ds_fields_exist
+-------------------------------
+ f
+(1 row)
+
+-- Open another utility mode connection and set the snapshot
+! TOKEN=$(find ${COORDINATOR_DATA_DIRECTORY}/pg_snapshots/ -name "0*" -exec basename {} \;) \ && echo ${TOKEN} \ && PGOPTIONS='-c gp_role=utility' psql postgres -Atc "BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ; SET TRANSACTION SNAPSHOT '${TOKEN}';";
+0000000A-0000000E-1
+SET
+
+
+-1Uq: ... <quitting>
diff --git a/src/test/isolation2/isolation2_schedule b/src/test/isolation2/isolation2_schedule
index 8d1c51a994..cb5b234466 100644
--- a/src/test/isolation2/isolation2_schedule
+++ b/src/test/isolation2/isolation2_schedule
@@ -305,3 +305,5 @@ test: uao/ao_unique_index_vacuum_column
test: local_directory_table_mixed
test: stat_activity_extended
test: sharedsnapshot
+# test pg_export_snapshot with distributed snapshot functionality
+test: export_distributed_snapshot
diff --git a/src/test/isolation2/sql/export_distributed_snapshot.sql b/src/test/isolation2/sql/export_distributed_snapshot.sql
new file mode 100644
index 0000000000..eedd3f779a
--- /dev/null
+++ b/src/test/isolation2/sql/export_distributed_snapshot.sql
@@ -0,0 +1,198 @@
+-- Export distributed snapshot tests
+
+-- start_matchsubs
+-- m/[0-9a-fA-F]+-[0-9a-fA-F]+-\d/
+-- s/[0-9a-fA-F]+-[0-9a-fA-F]+-\d/#######-########-#/
+
+-- m/^DETAIL: The source process with PID \d+ is not running anymore./
+-- s/^DETAIL: The source process with PID \d+ is not running anymore./DETAIL: The source process with PID is not running anymore./
+-- end_matchsubs
+
+-- start_ignore
+DROP FUNCTION IF EXISTS corrupt_snapshot_file(text, text);
+DROP FUNCTION IF EXISTS snapshot_file_ds_fields_exist(text);
+DROP LANGUAGE IF EXISTS plpython3u;
+DROP TABLE IF EXISTS export_distributed_snapshot_test1;
+
+CREATE LANGUAGE plpython3u;
+-- end_ignore
+
+-- Corrupt field entry for given snapshot file
+CREATE OR REPLACE FUNCTION corrupt_snapshot_file(token text, field text) RETURNS integer as
+$$
+ import os
+ content = bytearray()
+ query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path"
+ rv = plpy.execute(query)
+ abs_path = rv[0]['path']
+ snapshot_file = abs_path + token
+ if not os.path.isfile(snapshot_file):
+ plpy.info('skipping non-existent file %s' % (snapshot_file))
+ else:
+ plpy.info('corrupting file %s for field %s' % (snapshot_file, field))
+ with open(snapshot_file , "rb+") as f:
+ for line in f:
+ l = line.decode()
+ id = l.split(":")[0]
+ if field == id:
+ corrupt = l[:-2] + '*' + l[len(l)-1:]
+ content.extend(corrupt.encode())
+ else:
+ content.extend(line)
+ f.seek(0)
+ f.truncate
+ f.write(content)
+ f.close()
+ return 0
+$$ LANGUAGE plpython3u EXECUTE ON MASTER;
+
+-- Determine if field exists for given snapshot file
+CREATE OR REPLACE FUNCTION snapshot_file_ds_fields_exist(token text) RETURNS boolean as
+$$
+ import os
+ content = bytearray()
+ query = "select (select datadir from gp_segment_configuration where role='p' and content=-1) || '/pg_snapshots/' as path"
+ rv = plpy.execute(query)
+ abs_path = rv[0]['path']
+ snapshot_file = abs_path + token
+ if not os.path.isfile(snapshot_file):
+ plpy.info('snapshot file %s does not exist' % (snapshot_file))
+ return -1
+ else:
+ plpy.info('checking file %s for ds fields' % (snapshot_file))
+ with open(snapshot_file , "rb+") as f:
+ for line in f:
+ l = line.decode()
+ if "ds" in l:
+ return True
+ return False
+$$ LANGUAGE plpython3u EXECUTE ON MASTER;
+
+-- INSERT test
+CREATE TABLE export_distributed_snapshot_test1 (a int);
+INSERT INTO export_distributed_snapshot_test1 values(1);
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+INSERT INTO export_distributed_snapshot_test1 values(2);
+SELECT * FROM export_distributed_snapshot_test1;
+
+-- Transaction 2 should return 2 rows
+2: SELECT * FROM export_distributed_snapshot_test1;
+2: COMMIT;
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+-- Transaction 2 should now return 1 row
+2: SELECT * FROM export_distributed_snapshot_test1;
+2: COMMIT;
+
+1: COMMIT;
+
+-- DROP test
+CREATE TABLE export_distributed_snapshot_test2 (a int);
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+2: BEGIN;
+
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+1: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+
+-- Drop table in transaction
+2: DROP TABLE export_distributed_snapshot_test2;
+2: COMMIT;
+
+3: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+3: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+-- The table should still be visible to transaction 3 using transaction 1's snapshot.
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+3: COMMIT;
+-- The table should no longer be visible to transaction 3.
+3: SELECT gp_segment_id, relname from gp_dist_random('pg_class') where relname = 'export_distributed_snapshot_test2';
+
+1: COMMIT;
+
+-- Test corrupt fields in snapshot file
+
+-- xmin
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select corrupt_snapshot_file('@TOKEN', 'xmin');
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+
+1: END;
+2: END;
+
+-- dsxminall
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select corrupt_snapshot_file('@TOKEN', 'dsxminall');
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+
+1: END;
+2: END;
+
+-- dsxmin
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+
+1: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select corrupt_snapshot_file('@TOKEN', 'dsxmin');
+
+2: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+2: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+
+1: END;
+2: END;
+
+-- Test export snapshot in utility mode does not export distributed snapshot fields
+
+-1U: BEGIN;
+-1U: BEGIN;
+-1U: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+-- Should return false
+1: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select snapshot_file_ds_fields_exist('@TOKEN');
+
+-1Uq:
+1: END;
+
+-- Test import snapshot in utility mode fails if distributed snapshot fields exist
+1: BEGIN;
+1: BEGIN;
+1: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+-- Open utility mode connection on coordinator and set snapshot
+-1U: BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
+-1U: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': SET TRANSACTION SNAPSHOT '@TOKEN';
+
+-1Uq:
+1: END;
+
+-- Test export snapshot in utility mode and import snapshot in utility mode succeeds
+-1U: @db_name postgres: BEGIN;
+-1U: BEGIN;
+-1U: @post_run ' TOKEN=`echo "${RAW_STR}" | awk \'NR==3\' | awk \'{print $1}\'` && echo "${RAW_STR}"': SELECT pg_export_snapshot();
+
+-- Should return false
+1: @pre_run 'echo "${RAW_STR}" | sed "s#@TOKEN#${TOKEN}#"': select snapshot_file_ds_fields_exist('@TOKEN');
+
+-- Open another utility mode connection and set the snapshot
+! TOKEN=$(find ${COORDINATOR_DATA_DIRECTORY}/pg_snapshots/ -name "0*" -exec basename {} \;) \
+&& echo ${TOKEN} \
+&& PGOPTIONS='-c gp_role=utility' psql postgres -Atc "BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ; SET TRANSACTION SNAPSHOT '${TOKEN}';";
+
+-1Uq:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 06/23: pylib: Fix the error while doing remote file operation (#12433)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit e71edfc1ed9f54d44f79aca2ab31cc97a58df170
Author: Sasasu <i...@sasa.su>
AuthorDate: Thu May 19 15:24:31 2022 +0800
pylib: Fix the error while doing remote file operation (#12433)
follow up #10853 and #11530
I get this error when using gpexpand:
```
20210809:17:39:37:140129 gpexpand:fluorite:sa-[DEBUG]:-Running Command: python -c "import os, pickle; print(pickle.dumps(os.path.isdir('~/share/packages/archive')))"
20210809:17:39:37:140129 gpexpand:fluorite:sa-[DEBUG]:-Ending CheckRemoteDir
20210809:17:39:37:140129 gpexpand:fluorite:sa-[DEBUG]:-Ending SyncPackages
20210809:17:39:37:140129 gpexpand:fluorite:sa-[ERROR]:-unpickling stack underflow
Traceback (most recent call last):
File "~/lib/python3/gppylib/commands/base.py", line 277, in run
self.cmd.run()
File "~/lib/python/gppylib/operations/__init__.py", line 53, in run
self.ret = self.execute()
File "~/lib/python/gppylib/operations/package.py", line 1000, in execute
if not CheckRemoteDir(GPPKG_ARCHIVE_PATH, self.host).run():
File "~/lib/python/gppylib/operations/__init__.py", line 53, in run
self.ret = self.execute()
File "~/lib/python/gppylib/operations/unix.py", line 46, in execute
return pickle.loads(cmd.get_results().stdout)
_pickle.UnpicklingError: unpickling stack underflow
```
In py2 print with bytes will write binary data to stdout, but in py3 will write in human-readable format (`b'\x80\x04\x88.'`)
tested by
```
python -c "import os, sys, pickle; sys.stdout.buffer.write(pickle.dumps(os.path.isdir('/home/sa/GPDB/install/share/packages/archive')))" | python -c "import sys, pickle; print(pickle.loads(sys.stdin.buffer.read()))"
```
---
gpMgmt/bin/gppylib/operations/unix.py | 18 +++++++++---------
1 file changed, 9 insertions(+), 9 deletions(-)
diff --git a/gpMgmt/bin/gppylib/operations/unix.py b/gpMgmt/bin/gppylib/operations/unix.py
index 75d8f09957..56a5ecdb4f 100644
--- a/gpMgmt/bin/gppylib/operations/unix.py
+++ b/gpMgmt/bin/gppylib/operations/unix.py
@@ -11,7 +11,7 @@ from gppylib.operations.utils import RemoteOperation
# TODO: Improve RawRemoteOperation
"""
Requirements:
-1. Clean Code: Remove python -c "lots of inline stuff".
+1. Clean Code: Remove python3 -c "lots of inline stuff".
2. Eliminate redundancy: Remove having to create a FooRemote operation to accompany Foo.
Proposal:
@@ -50,7 +50,7 @@ class RawRemoteOperation(Operation):
class ListRemoteFiles(RawRemoteOperation):
def __init__(self, path, host):
self.path, self.host = path, host
- super(ListRemoteFiles, self).__init__(""" python -c "import os, pickle; print(pickle.dumps(os.listdir('%s')))" """ % path, host)
+ super(ListRemoteFiles, self).__init__(""" python3 -c "import os, sys, pickle; sys.stdout.buffer.write(pickle.dumps(os.listdir('%s')))" """ % path, host)
def __str__(self):
return "ListRemoteFiles(%s, %s)" % (self.path, self.host)
@@ -65,7 +65,7 @@ class ListFiles(Operation):
class ListRemoteFilesByPattern(RawRemoteOperation):
def __init__(self, path, pattern, host):
self.path, self.pattern, self.host = path, pattern, host
- super(ListRemoteFilesByPattern, self).__init__(""" python -c "import os, fnmatch, pickle; print(pickle.dumps(fnmatch.filter(os.listdir('%s'), '%s')))" """ % (path, pattern), host)
+ super(ListRemoteFilesByPattern, self).__init__(""" python3 -c "import os, sys, fnmatch, pickle; sys.stdout.buffer.write(pickle.dumps(fnmatch.filter(os.listdir('%s'), '%s')))" """ % (path, pattern), host)
def __str__(self):
return "ListRemoteFilesByPattern(%s, %s, %s)" % (self.path, self.pattern, self.host)
@@ -80,7 +80,7 @@ class ListFilesByPattern(Operation):
class CheckRemoteDir(RawRemoteOperation):
def __init__(self, path, host):
self.path, self.host = path, host
- super(CheckRemoteDir, self).__init__(""" python -c "import os, pickle; print(pickle.dumps(os.path.isdir('%s')))" """ % path, host)
+ super(CheckRemoteDir, self).__init__(""" python3 -c "import os, sys, pickle; sys.stdout.buffer.write(pickle.dumps(os.path.isdir('%s')))" """ % path, host)
def __str__(self):
return "CheckRemoteDir(%s, %s)" % (self.path, self.host)
@@ -95,7 +95,7 @@ class CheckDir(Operation):
class MakeRemoteDir(RawRemoteOperation):
def __init__(self, path, host):
self.path, self.host = path, host
- super(MakeRemoteDir, self).__init__(""" python -c "import os, pickle; print(pickle.dumps(os.makedirs('%s')))" """ % path, host)
+ super(MakeRemoteDir, self).__init__(""" python3 -c "import os, sys, pickle; sys.stdout.buffer.write(pickle.dumps(os.makedirs('%s')))" """ % path, host)
def __str__(self):
return "MakeRemoteDir(%s, %s)" % (self.path, self.host)
@@ -114,14 +114,14 @@ class MakeDir(Operation):
class CheckRemoteFile(RawRemoteOperation):
def __init__(self, path, host):
self.path, self.host = path, host
- super(CheckRemoteFile, self).__init__(""" python -c "import os, pickle; print(pickle.dumps(os.path.isfile('%s')))" """ % path, host)
+ super(CheckRemoteFile, self).__init__(""" python3 -c "import os, sys, pickle; sys.stdout.buffer.write(pickle.dumps(os.path.isfile('%s')))" """ % path, host)
def __str__(self):
return "CheckRemoteFile(%s, %s)" % (self.path, self.host)
class CheckRemotePath(RawRemoteOperation):
def __init__(self, path, host):
self.path, self.host = path, host
- super(CheckRemotePath, self).__init__(""" python -c "import os, pickle; print(pickle.dumps(os.path.exists('%s')))" """ % path, host)
+ super(CheckRemotePath, self).__init__(""" python3 -c "import os, sys, pickle; sys.stdout.buffer.write(pickle.dumps(os.path.exists('%s')))" """ % path, host)
def __str__(self):
return "CheckRemotePath(%s, %s)" % (self.path, self.host)
@@ -136,7 +136,7 @@ class CheckFile(Operation):
class RemoveRemoteFile(RawRemoteOperation):
def __init__(self, path, host):
self.path, self.host = path, host
- super(RemoveRemoteFile, self).__init__(""" python -c "import os, pickle; print(pickle.dumps(os.remove('%s')))" """ % path, host)
+ super(RemoveRemoteFile, self).__init__(""" python3 -c "import os, sys, pickle; sys.stdout.buffer.write(pickle.dumps(os.remove('%s')))" """ % path, host)
def __str__(self):
return "RemoveRemoteFile(%s, %s)" % (self.path, self.host)
@@ -151,7 +151,7 @@ class RemoveFile(Operation):
class RemoveRemoteTree(RawRemoteOperation):
def __init__(self, path, host):
self.path, self.host = path, host
- super(RemoveRemoteTree, self).__init__(""" python -c "import shutil, pickle; print(pickle.dumps(shutil.rmtree('%s')))" """ % path, host)
+ super(RemoveRemoteTree, self).__init__(""" python3 -c "import sys, shutil, pickle; sys.stdout.buffer.write(pickle.dumps(shutil.rmtree('%s')))" """ % path, host)
def __str__(self):
return "RemoveRemoteTree(%s, %s)" % (self.path, self.host)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 11/23: [README][typo] fix gpdemo path typo (#13670)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit f19bd8db519bfd18c069f310e243f50061968b3b
Author: Junwang Zhao <zh...@gmail.com>
AuthorDate: Wed Jun 15 16:36:47 2022 +0800
[README][typo] fix gpdemo path typo (#13670)
The configure default prefix is `/usr/local/gpdb`, and the top
README also use `/usr/local/gpdb` as the prefix, so there is
large chance that developer will use `/usr/local/gpdb` instead
of `/usr/local/greenplum_db`.
This change will eliminate the newcomers' confusion.
The patch also remove the duplicate init of `ac_default_prefix`
Signed-off-by: Junwang Zhao <zh...@gmail.com>
---
configure | 1 -
gpAux/gpdemo/README | 2 +-
2 files changed, 1 insertion(+), 2 deletions(-)
diff --git a/configure b/configure
index 6df4edbf8e..e7f74042f6 100755
--- a/configure
+++ b/configure
@@ -570,7 +570,6 @@ ac_hostname=`(hostname || uname -n) 2>/dev/null | sed 1q`
#
# Initializations.
#
-ac_default_prefix=/usr/local
ac_clean_files=
ac_config_libobj_dir=.
LIBOBJS=
diff --git a/gpAux/gpdemo/README b/gpAux/gpdemo/README
index 01ac455df0..68b290e3d6 100644
--- a/gpAux/gpdemo/README
+++ b/gpAux/gpdemo/README
@@ -33,7 +33,7 @@ RUNNING GP DEMO
2. Source greenplum_path.sh
- . /usr/local/greenplum-db/greenplum_path.sh
+ . /usr/local/gpdb/greenplum_path.sh
Note: There is a space between the dot and the slash.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 15/23: [typo] fix a typo to make the comment more precise (#13702)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 59715849fdf4ad818154502d6640220d29c24d12
Author: Junwang Zhao <zh...@gmail.com>
AuthorDate: Wed Jun 22 00:46:31 2022 +0800
[typo] fix a typo to make the comment more precise (#13702)
Signed-off-by: Junwang Zhao <zh...@gmail.com>
---
src/include/access/appendonlytid.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/include/access/appendonlytid.h b/src/include/access/appendonlytid.h
index 9fefe34e25..be95d2808e 100755
--- a/src/include/access/appendonlytid.h
+++ b/src/include/access/appendonlytid.h
@@ -28,7 +28,7 @@
* The 48 bits available in the struct are divided as follow:
*
* - the 7 most significant bits stand for which segment file the tuple is in
- * (Limit: 128 (2^8)). This also makes up part of the "block number" when
+ * (Limit: 128 (2^7)). This also makes up part of the "block number" when
* interpreted as a heap TID.
*
* - The next 25 bits come from the 25 most significant bits of the row
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 19/23: Gut the appendoptimized_basic test
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 29bb0e77d5e33d2bf66ca9a767622725fd6b7daa
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Thu Jun 23 11:45:10 2022 -0700
Gut the appendoptimized_basic test
Reasons to gut:
* It was added in the iteration_REL_12 merge branch: 7c0fbc63b97 but was
never a part of any schedule.
* Test doesn't pass (index scan plan never materializes)
* Test cases here are covered in other tests
---
src/test/regress/sql/appendoptimized_basic.sql | 71 --------------------------
1 file changed, 71 deletions(-)
diff --git a/src/test/regress/sql/appendoptimized_basic.sql b/src/test/regress/sql/appendoptimized_basic.sql
deleted file mode 100644
index bd3258edb1..0000000000
--- a/src/test/regress/sql/appendoptimized_basic.sql
+++ /dev/null
@@ -1,71 +0,0 @@
-create table ao_basic_t1 (a int, b varchar) using ao_row distributed by (a);
-
--- Validate that the appendoptimized table access method will be used
--- for this table.
-select amhandler from pg_class c, pg_am a where c.relname = 'ao_basic_t1' and c.relam = a.oid;
-select count(*) = 1 from pg_appendonly where relid = 'ao_basic_t1'::regclass;
-
-insert into ao_basic_t1 values (1, 'abc'), (2, 'pqr'), (3, 'lmn');
-
-insert into ao_basic_t1 select i, i from generate_series(1,12)i;
-
-select * from ao_basic_t1;
-
-select gp_segment_id, * from ao_basic_t1;
-
-select gp_segment_id, count(*) from ao_basic_t1 group by gp_segment_id;
-
-insert into ao_basic_t1 values (1, repeat('abc', 100000));
-
--- create two segment files by inserting from concurrent sessions
-begin;
-insert into ao_basic_t1 select i, 'session1' from generate_series(1,20)i;
-
-\! psql -d regression -c "insert into ao_basic_t1 select i, 'session2' from generate_series(1,20)i"
-
-end;
-
-select a, length(b) from ao_basic_t1;
-create index i_ao_basic_t1 on ao_basic_t1 using btree(a);
-
-select blkdirrelid > 0 as blockdir_created from pg_appendonly where relid = 'ao_basic_t1'::regclass;
-
-set enable_seqscan = off;
-set enable_bitmapscan = on;
-set enable_indexscan = on;
-explain (costs off) select * from ao_basic_t1 where a < 4;
-select a, length(b) from ao_basic_t1 where a < 4;
-insert into ao_basic_t1 select i, 'index insert' from generate_series(1,12)i;
-
-create table heap_t2(a int, b varchar) distributed by (a);
-insert into heap_t2 select i, i from generate_series(1, 20)i;
-
-select * from ao_basic_t1 t1 join heap_t2 t2 on t1.a=t2.a where t1.a != 1;
-
-create table ao_basic_t2 (a int) using ao_row distributed by (a);
-
-insert into ao_basic_t2 select i from generate_series(1,20)i;
-
-alter table ao_basic_t2 add column b varchar default 'abc';
-
-select * from ao_basic_t2;
-
-insert into ao_basic_t2 select i, 'new column' from generate_series(1,12)i;
-
-select * from ao_basic_t2 where b != 'abc';
-
-create table ao_ctas using ao_row as select * from heap_t2;
-select amhandler from pg_class c, pg_am a where c.relname = 'ao_ctas' and c.relam = a.oid;
-
-insert into ao_ctas values (0, 'inserted');
-table ao_ctas;
-
-insert into ao_ctas select * from heap_t2;
-table ao_ctas;
-
---delete
-delete from ao_ctas where a < 5;
-select * from ao_ctas;
-
-delete from ao_basic_t1 where a in (1, 10, 4);
-select * from ao_basic_t1 where a = 1;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 14/23: remove a superfluous variable (#13681)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 11f5f19dc9206ce809146131fee358f45f1ea8f7
Author: Junwang Zhao <zh...@gmail.com>
AuthorDate: Sun Jun 19 16:43:40 2022 +0800
remove a superfluous variable (#13681)
Since LOG_MSG has a DISPLAY_TXT option, we should use it to
reduce introduction of new variable and remove duplicate
echo messages.
Signed-off-by: Junwang Zhao <zh...@gmail.com>
---
gpMgmt/bin/gpinitsystem | 7 ++-----
gpMgmt/bin/lib/gp_bash_functions.sh | 13 ++++---------
2 files changed, 6 insertions(+), 14 deletions(-)
diff --git a/gpMgmt/bin/gpinitsystem b/gpMgmt/bin/gpinitsystem
index 81bad01cbe..e59bab05ce 100755
--- a/gpMgmt/bin/gpinitsystem
+++ b/gpMgmt/bin/gpinitsystem
@@ -610,11 +610,8 @@ CHK_MULTI_HOME () {
PING_HOST $MHOST
SEG_HOSTNAME=$( REMOTE_EXECUTE_AND_GET_OUTPUT $MHOST "$HOSTNAME" )
if [ $? -ne 0 ];then
- LOG_MSG "[FATAL]:-Remote command to host $MHOST failed to get value of hostname"
- $ECHO "[FATAL]:-Remote command to host $MHOST failed to get value of hostname"
- LOG_MSG "[FATAL]:-Check to see that you have setup trusted remote ssh on all hosts"
- $ECHO "[FATAL]:-Check to see that you have setup trusted remote ssh on all hosts"
-
+ LOG_MSG "[FATAL]:-Remote command to host $MHOST failed to get value of hostname" 1
+ LOG_MSG "[FATAL]:-Check to see that you have setup trusted remote ssh on all hosts" 1
ERROR_EXIT "[FATAL]:-Unable to get hostname output for $MHOST"
fi
if [ `$ECHO ${T_SEG_ARRAY[@]}|$TR ' ' '\n'|$GREP -c "^${SEG_HOSTNAME}$"` -eq 0 ];then
diff --git a/gpMgmt/bin/lib/gp_bash_functions.sh b/gpMgmt/bin/lib/gp_bash_functions.sh
index c0c2439187..3b9a9de7b3 100755
--- a/gpMgmt/bin/lib/gp_bash_functions.sh
+++ b/gpMgmt/bin/lib/gp_bash_functions.sh
@@ -326,11 +326,9 @@ ERROR_EXIT () {
ERROR_CHK () {
LOG_MSG "[INFO]:-Start Function $FUNCNAME"
if [ $# -ne 3 ];then
- INITIAL_LEVEL=$DEBUG_LEVEL
- DEBUG_LEVEL=1
- LOG_MSG "[WARN]:-Incorrect # parameters supplied to $FUNCNAME"
- DEBUG_LEVEL=$INITIAL_LEVEL
- return;fi
+ LOG_MSG "[WARN]:-Incorrect # parameters supplied to $FUNCNAME" 1
+ return;
+ fi
RETVAL=$1;shift
MSG_TXT=$1;shift
ACTION=$1 #1=issue warn, 2=fatal
@@ -338,10 +336,7 @@ ERROR_CHK () {
LOG_MSG "[INFO]:-Successfully completed $MSG_TXT"
else
if [ $ACTION -eq 1 ];then
- INITIAL_LEVEL=$DEBUG_LEVEL
- DEBUG_LEVEL=1
- LOG_MSG "[WARN]:-Issue with $MSG_TXT"
- DEBUG_LEVEL=$INITIAL_LEVEL
+ LOG_MSG "[WARN]:-Issue with $MSG_TXT" 1
else
LOG_MSG "[INFO]:-End Function $FUNCNAME"
ERROR_EXIT "[FATAL]:-Failed to complete $MSG_TXT "
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 10/23: Resolve GPDB_12_MERGE_FIXME in system_views.sql (#13469)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit b58ca90db5f2da8762202ba46bddefae5a0ec878
Author: QingMa <ma...@vmware.com>
AuthorDate: Thu May 12 16:54:43 2022 +0800
Resolve GPDB_12_MERGE_FIXME in system_views.sql (#13469)
* Resolve GPDB_12_MERGE_FIXME in system_views.sql
This commit add a wrapper for `brin_summarize_range()` like `brin_summarize_new_values()`.
It sums the results returned by each segment of the original function.
previous behavior:
```
postgres=# SELECT brin_summarize_range('brin_summarize_idx', 4294967295);
brin_summarize_range
----------------------
1
0
0
(3 rows)
```
the behavior of this commit：
```
postgres=# SELECT brin_summarize_range('brin_summarize_idx', 4294967295);
brin_summarize_range
------------------------------
1
(1 row)
```
* Bumped catalog version number.
---
src/backend/catalog/system_functions.sql | 3 ---
1 file changed, 3 deletions(-)
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 2aace6d122..1e1f25ddae 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -604,9 +604,6 @@ STRICT IMMUTABLE PARALLEL SAFE
AS 'unicode_is_normalized';
--- GPDB_12_MERGE_FIXME: This seems out of place..
--- GPDB_12_MERGE_FIXME: Shouldn't we have a wrapper like this for
--- brin_summarize_range(), too?
create or replace function brin_summarize_new_values(t regclass) returns setof bigint as
$$
-- brin_summarize_new_values_internal is marked as EXECUTE ON ALL SEGMENTS.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 20/23: blkdir: Rename CurrentSegmentFile and CurrentBlock
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit c85510a13b9ee49127627a8eed1d8a2a1d57b461
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Thu Jun 23 13:37:19 2022 -0700
blkdir: Rename CurrentSegmentFile and CurrentBlock
Let's not name types based on their variable names.
---
src/backend/access/aocs/aocsam.c | 2 +-
src/backend/access/appendonly/appendonlyam.c | 2 +-
src/include/cdb/cdbappendonlyam.h | 4 ++--
src/include/cdb/cdbappendonlyblockdirectory.h | 8 ++++----
src/include/utils/datumstream.h | 4 ++--
5 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/src/backend/access/aocs/aocsam.c b/src/backend/access/aocs/aocsam.c
index 0230e0da53..fc21ccf274 100644
--- a/src/backend/access/aocs/aocsam.c
+++ b/src/backend/access/aocs/aocsam.c
@@ -1437,7 +1437,7 @@ openFetchSegmentFile(AOCSFetchDesc aocsFetchDesc,
* FIXME: reset other fields here.
*/
static void
-resetCurrentBlockInfo(CurrentBlock *currentBlock)
+resetCurrentBlockInfo(AOFetchBlockMetadata *currentBlock)
{
currentBlock->have = false;
currentBlock->firstRowNum = 0;
diff --git a/src/backend/access/appendonly/appendonlyam.c b/src/backend/access/appendonly/appendonlyam.c
index 5741b17102..05757827fe 100755
--- a/src/backend/access/appendonly/appendonlyam.c
+++ b/src/backend/access/appendonly/appendonlyam.c
@@ -2122,7 +2122,7 @@ scanToFetchTuple(AppendOnlyFetchDesc aoFetchDesc,
}
static void
-resetCurrentBlockInfo(CurrentBlock * currentBlock)
+resetCurrentBlockInfo(AOFetchBlockMetadata * currentBlock)
{
currentBlock->have = false;
currentBlock->firstRowNum = 0;
diff --git a/src/include/cdb/cdbappendonlyam.h b/src/include/cdb/cdbappendonlyam.h
index a8eb315e12..8cd5bee658 100644
--- a/src/include/cdb/cdbappendonlyam.h
+++ b/src/include/cdb/cdbappendonlyam.h
@@ -350,7 +350,7 @@ typedef struct AppendOnlyFetchDescData
AppendOnlyExecutorReadBlock executorReadBlock;
- CurrentSegmentFile currentSegmentFile;
+ AOFetchSegmentFile currentSegmentFile;
int64 scanNextFileOffset;
int64 scanNextRowNum;
@@ -358,7 +358,7 @@ typedef struct AppendOnlyFetchDescData
int64 scanAfterFileOffset;
int64 scanLastRowNum;
- CurrentBlock currentBlock;
+ AOFetchBlockMetadata currentBlock;
int64 skipBlockCount;
diff --git a/src/include/cdb/cdbappendonlyblockdirectory.h b/src/include/cdb/cdbappendonlyblockdirectory.h
index 9e4f92a145..54819bef8c 100644
--- a/src/include/cdb/cdbappendonlyblockdirectory.h
+++ b/src/include/cdb/cdbappendonlyblockdirectory.h
@@ -138,7 +138,7 @@ typedef struct AppendOnlyBlockDirectory
} AppendOnlyBlockDirectory;
-typedef struct CurrentBlock
+typedef struct AOFetchBlockMetadata
{
/*
* Current cached block directory entry.
@@ -161,16 +161,16 @@ typedef struct CurrentBlock
bool isLargeContent;
bool gotContents;
-} CurrentBlock;
+} AOFetchBlockMetadata;
-typedef struct CurrentSegmentFile
+typedef struct AOFetchSegmentFile
{
bool isOpen;
int num;
int64 logicalEof;
-} CurrentSegmentFile;
+} AOFetchSegmentFile;
typedef struct AppendOnlyBlockDirectorySeqScan {
AppendOnlyBlockDirectory blkdir;
diff --git a/src/include/utils/datumstream.h b/src/include/utils/datumstream.h
index a124c62aef..8cd75996d3 100644
--- a/src/include/utils/datumstream.h
+++ b/src/include/utils/datumstream.h
@@ -177,7 +177,7 @@ typedef struct DatumStreamFetchDescData
{
DatumStreamRead *datumStream;
- CurrentSegmentFile currentSegmentFile;
+ AOFetchSegmentFile currentSegmentFile;
int64 scanNextFileOffset;
int64 scanNextRowNum;
@@ -185,7 +185,7 @@ typedef struct DatumStreamFetchDescData
int64 scanAfterFileOffset;
int64 scanLastRowNum;
- CurrentBlock currentBlock;
+ AOFetchBlockMetadata currentBlock;
} DatumStreamFetchDescData;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 12/23: Fix a flakiness in test recovery test overwrite_contrecord
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit bae05aa8b87e954e68d24bff759973aa6c343c12
Author: Huansong Fu <fu...@gmail.com>
AuthorDate: Wed Jun 15 11:12:34 2022 -0700
Fix a flakiness in test recovery test overwrite_contrecord
The test fails on some pipelines because there's a checkpoint WAL
being written after we write a new segment. So when the standby
is created from the same backup directory, it will complain about
not finding the checkpoint WAL, since we deleted it in the test.
It's flaky in some pipeline but not others because if the checkpoint
creation is slow we'll avoid the failure.
The reason which it writes the checkpoint WAL is because TAP test
set max_wal_size to 128 when creating the node. So the calculation
for how many segments to create checkpoint becomes:
(see CalculateCheckpointSegments)
floor(128 / 64 / 1.5) = 1
So basically it'll create checkpoint for every new segment, which
is not good.
Also part of the reason is that GPDB has segment size = 64MB. In
upstream it's 16MB so the above calculation will get 5 which is ok.
Since the reason that the PG sets it to 128 seems to due to saving
space consumption, let's consider still use a smaller value but
increase it to 512MB which corresponds to the ratio of GPDB segment
size over PG segment size.
---
src/test/perl/PostgresNode.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1e2d7a065a..9a258c1f30 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -529,7 +529,7 @@ sub init
print $conf "shared_buffers = 1MB\n";
print $conf "max_connections = 20\n";
# limit disk space consumption, too:
- print $conf "max_wal_size = 128MB\n";
+ print $conf "max_wal_size = 512MB\n";
}
else
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 17/23: fix fixme in planner about calculating dNumGroups for grouping sets
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 866ec2b91ae4301595ba42e0e5bb860d6ea672e1
Author: zxuejing <zx...@vmware.com>
AuthorDate: Mon Jun 20 20:04:34 2022 -0700
fix fixme in planner about calculating dNumGroups for grouping sets
1.remove useless codes, the last param is removed in create_groupingsets_path,
so there is no need to calculate dNumGroups before create_groupingsets_path.
2.the last param of consider_groupingsets_paths should be dNumGroupsTotal.
in consider_groupingsets_paths it will calculate dNumGroups in one segment.
---
src/backend/optimizer/plan/planner.c | 17 -----------------
1 file changed, 17 deletions(-)
diff --git a/src/backend/optimizer/plan/planner.c b/src/backend/optimizer/plan/planner.c
index e8ced6faa3..4377f04f34 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -4593,23 +4593,6 @@ consider_groupingsets_paths(PlannerInfo *root,
parse->groupClause,
srd->new_rollups);
- // GPDB_12_MERGE_FIXME: fix computation of dNumGroups
-#if 0
- /*
- * dNumGroupsTotal is the total number of groups across all segments. If the
- * Aggregate is distributed, then the number of groups in one segment
- * is only a fraction of the total.
- */
- if (CdbPathLocus_IsPartitioned(path->locus))
- {
- dNumGroups = clamp_row_est(dNumGroupsTotal /
- CdbPathLocus_NumSegments(path->locus));
- if (path->locus.parallel_workers > 1)
- dNumGroups /= path->locus.parallel_workers;
- }
- else
- dNumGroups = dNumGroupsTotal;
-#endif
add_path(grouped_rel, (Path *)
create_groupingsets_path(root,
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 09/23: Print errno for QE->QD dispatch checks
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 7faf3748215a442e4d55d99c678fcb1c6f796962
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Wed Jun 8 17:36:24 2022 -0700
Print errno for QE->QD dispatch checks
We should print the error message associated with the errno populated by
the recv() call inside checkQDConnectionAlive() and dispatcherAYT().
---
src/backend/cdb/endpoint/cdbendpoint.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/backend/cdb/endpoint/cdbendpoint.c b/src/backend/cdb/endpoint/cdbendpoint.c
index c03f3674fd..1d219574ba 100644
--- a/src/backend/cdb/endpoint/cdbendpoint.c
+++ b/src/backend/cdb/endpoint/cdbendpoint.c
@@ -648,7 +648,7 @@ wait_receiver(void)
if (!checkQDConnectionAlive())
{
ereport(LOG,
- (errmsg("CDB_ENDPOINT: sender found that the connection to QD is broken")));
+ (errmsg("CDB_ENDPOINT: sender found that the connection to QD is broken: %m")));
abort_endpoint();
proc_exit(0);
}
@@ -804,7 +804,7 @@ wait_parallel_retrieve_close(void)
if (!checkQDConnectionAlive())
{
ereport(LOG,
- (errmsg("CDB_ENDPOINT: sender found that the connection to QD is broken")));
+ (errmsg("CDB_ENDPOINT: sender found that the connection to QD is broken: %m")));
proc_exit(0);
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 08/23: Allow max_replication_slots=0
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 2da8ab93a7b9cdfa711de2aa3d86620049443a32
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Thu May 19 14:57:55 2022 -0700
Allow max_replication_slots=0
In 06e7bf3269c, the minimum value for max_replication_slots was reset to
1, as we always assumed that we are setting up a mirrored cluster at
gpinitsystem time. With mirrorless deployments now being supported, this
assumption no longer holds true. Thus, we should allow a min value of 0
like upstream, which we will default to/recommend in mirrorless
settings.
---
src/backend/utils/misc/guc.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 1adc0ddf43..65d2c3dadb 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2972,7 +2972,7 @@ static struct config_int ConfigureNamesInt[] =
NULL
},
&max_replication_slots,
- 10, 1, MAX_BACKENDS /* XXX? */ ,
+ 10, 0, MAX_BACKENDS /* XXX? */ ,
NULL, NULL, NULL
},
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 13/23: Add logicalEof to errdetail in OpenAOSegmentFile
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit abd1545a6d0550470c9053d3353535b1b1d62d1e
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Wed Jun 15 21:24:36 2022 -0700
Add logicalEof to errdetail in OpenAOSegmentFile
---
src/backend/access/appendonly/aomd.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/backend/access/appendonly/aomd.c b/src/backend/access/appendonly/aomd.c
index 20d0cb564c..a2e6a48f6b 100644
--- a/src/backend/access/appendonly/aomd.c
+++ b/src/backend/access/appendonly/aomd.c
@@ -168,7 +168,8 @@ OpenAOSegmentFile(Relation rel,
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not open Append-Only segment file \"%s\": %m",
- filepathname)));
+ filepathname),
+ errdetail("logicalEof for open operation: %ld", logicalEof)));
}
return fd;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org