You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by ma...@apache.org on 2025/01/14 01:01:13 UTC

(cloudberry) branch main updated (8cf1abafb4 -> 79b8ace965)

This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git


    from 8cf1abafb4 Refactor ALTER TABLE SET AM to be align with 2970799
     new c48753031a Bump minimum requirement of zstd to 1.4.0
     new 32d33bf966 Move a FIXME from postgresql.conf.sample to guc.c
     new b1aebd3afe Remove deadcode cdbpullup_isExprCoveredByTargetlist.
     new 745b89a8e8 Refactor cdbpullup_missingVarWalker.
     new f0fd4bd229 CI fix: don't log too much interconnect events (#15761)
     new eba05de737 Resolve FIXMEs in datetime.c
     new 5eb5061024 Add missing nodes for function raw_expression_tree_walker().
     new f917aa3446 Remove FIXME in relation_open() (#15786)
     new 2de0e285f5 Revert "Refactor cdbpullup_missingVarWalker."
     new 2f11cacb74 Only super user can set the GUC gp_resource_group_bypass.
     new 81d4dd8656 Enable `wal_compression` by default
     new 37c0b696a6 Update comment of createMotionLayerState() (#15800)
     new 6a2c96da05 Replace 'egrep' with 'grep -E'. (#15804)
     new 21404af330 Post fix: replace 'egrep' with 'grep -E' in gpMgmt (#15810)
     new e02db30aa4 Clean up comments.
     new d3c84a9a81 Remove MemoryContext s_tupSerMemCtxt (#15779)
     new 2d61bf5266 starting segments in execute mode post recovery (#15599)
     new 7f68241e2c Yet further fixes for multi-row VALUES lists for updatable views.
     new cdad9dc391 Fix multi-row DEFAULT handling for INSERT ... SELECT rules.
     new 79b8ace965 resource_queue.sql: Add pg_locks sanity check

The 20 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 configure                                          |  22 ++--
 configure.ac                                       |   4 +-
 contrib/Makefile                                   |   2 +-
 contrib/interconnect/udp/ic_udpifc.c               |   4 +-
 gpMgmt/bin/gpinitsystem                            |   4 +-
 gpMgmt/bin/lib/gp_bash_functions.sh                |   1 -
 gpMgmt/sbin/gpsegrecovery.py                       |   2 +-
 gpMgmt/test/behave/mgmt_utils/gpinitsystem.feature |   2 +-
 gpMgmt/test/behave/mgmt_utils/gprecoverseg.feature |   9 ++
 gpMgmt/test/behave/mgmt_utils/steps/mgmt_utils.py  |   6 +-
 .../mgmt_utils/steps/tablespace_mgmt_utils.py      |   1 -
 src/backend/access/common/relation.c               |   1 -
 src/backend/cdb/cdbpullup.c                        |  47 --------
 src/backend/cdb/motion/cdbmotion.c                 |   2 +-
 src/backend/cdb/motion/tupser.c                    |  35 +-----
 src/backend/nodes/nodeFuncs.c                      |   2 +
 src/backend/rewrite/rewriteHandler.c               | 125 ++++++++++++++-------
 src/backend/rewrite/rewriteManip.c                 |  11 +-
 src/backend/utils/adt/datetime.c                   |  68 +----------
 src/backend/utils/misc/guc.c                       |   3 +-
 src/backend/utils/misc/guc_gp.c                    |   2 +-
 src/backend/utils/misc/postgresql.conf.sample      |   4 +-
 src/include/cdb/cdbpullup.h                        |  20 ----
 .../expected/resgroup/resgroup_bypass.out          |   6 +
 src/test/isolation2/expected/resource_queue.out    |   8 +-
 .../expected/segwalrep/select_throttle.out         |  10 +-
 .../isolation2/sql/resgroup/resgroup_bypass.sql    |   5 +
 src/test/isolation2/sql/resource_queue.sql         |   5 +-
 .../isolation2/sql/segwalrep/select_throttle.sql   |   2 +-
 src/test/regress/expected/updatable_views.out      |  41 +++++++
 .../regress/expected/updatable_views_optimizer.out |  41 +++++++
 src/test/regress/sql/updatable_views.sql           |  27 +++++
 32 files changed, 272 insertions(+), 250 deletions(-)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 04/20: Refactor cdbpullup_missingVarWalker.

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 745b89a8e83f8f4ebcbf19dd8b65b47db4c2e7f2
Author: Zhenghua Lyu <ka...@gmail.com>
AuthorDate: Mon Jun 5 14:23:32 2023 +0000

    Refactor cdbpullup_missingVarWalker.
    
    Previously the API cdbpullup_missingVarWalker() is used to check if an
    expression can be represented by a targetlist. There are two problems
    for the previous design:
      * the API return bool type, the walker help function also return
        bool, but the walker's return value is to control
        expression_tree_walker's termination, it is not good to use
        walker's return value as the API's return value.
      * how to handle the case if there are no Vars in the node expression.
    
    This commit fixes the above two issues:
      * use context to store result needed by users, not the walker's
        return value
      * if there are no vars in the node expression, take this case as
        missing (cannot represented via targetlist).
    
    Fix Github Issue #15079.
---
 src/backend/cdb/cdbpullup.c                        | 112 +++++++++++++++------
 src/test/regress/expected/bfv_subquery.out         |  10 ++
 .../regress/expected/bfv_subquery_optimizer.out    |  10 ++
 src/test/regress/sql/bfv_subquery.sql              |   5 +
 4 files changed, 104 insertions(+), 33 deletions(-)

diff --git a/src/backend/cdb/cdbpullup.c b/src/backend/cdb/cdbpullup.c
index c6f243ae08..ae5945d0b6 100644
--- a/src/backend/cdb/cdbpullup.c
+++ b/src/backend/cdb/cdbpullup.c
@@ -27,9 +27,18 @@
 #include "cdb/cdbpullup.h"		/* me */
 
 
-static bool cdbpullup_missingVarWalker(Expr *node, void *targetlist);
+static bool  cdbpullup_missingVarCheck(Expr *node, List *targetlist);
+static List *cdbpullup_getall_vars(Node *node);
+static bool  cdbpullup_getall_vars_walker(Node *node, void *context);
+static bool  cdbpullup_missingVarCheckWalker(Node *node, void *context);
 static Expr *cdbpullup_make_expr(Index varno, AttrNumber varattno, Expr *oldexpr, bool modifyOld);
 
+
+typedef struct MissingVarContext
+{
+	List   *vars;
+	List   *findvars;
+} MissingVarContext;
 /*
  * cdbpullup_expr
  *
@@ -317,10 +326,8 @@ cdbpullup_findEclassInTargetList(EquivalenceClass *eclass, List *targetlist,
 
 		/* Return this item if all referenced Vars are in targetlist. */
 		if (!IsA(key, Var) &&
-			!cdbpullup_missingVarWalker(key, targetlist))
-		{
+			!cdbpullup_missingVarCheck(key, targetlist))
 			return key;
-		}
 	}
 
 	return NULL;
@@ -414,46 +421,85 @@ cdbpullup_make_expr(Index varno, AttrNumber varattno, Expr *oldexpr, bool modify
 	}
 }								/* cdbpullup_make_var */
 
-
 /*
- * cdbpullup_missingVarWalker
- *
- * Returns true if some Var in expr is not in targetlist.
- * 'targetlist' is either a List of TargetEntry, or a plain List of Expr.
+ * cdbpullup_missingVarCheck
+ *   Given an expression and a targetlist, check if the expression
+ *   can be represented by the exprs in the targetlist.
  *
- * NB:  A Var in the expr is considered as matching a Var in the targetlist
- * without regard for whether or not there is a RelabelType node atop the
- * targetlist Var.
+ *   This function return true when Expr node cannot be represented
+ *   based on targetlist (which matches the name that we have missing vars).
  *
+ *   The implementaion is as below:
+ *     1. build the list of all Vars in the Expr node, it is a set (created
+ *        by list_append_unique_ptr in the helper function cdbpullup_getall_vars)
+ *     2. walk the targetlist to build a set (created by list_append_unique_ptr)
+ *        for all Vars in targetlist that is member of Varset built in 1st step,
+ *        here membership test is using list_member which call equal to test.
+ *     3. if the size of the 2nd step's var set is smaller than the 1st step's,
+ *        it means the targetlist misses some vars.
  */
 static bool
-cdbpullup_missingVarWalker(Expr *node, void *targetlist)
+cdbpullup_missingVarCheck(Expr *node, List *targetlist)
 {
-	if (!node)
+	List             *vars    = NIL;
+	MissingVarContext context;
+	bool              missing;
+
+	vars = cdbpullup_getall_vars((Node *) node);
+	if (list_length(vars) == 0)
+		return true;
+
+	context.vars = vars;
+	context.findvars = NIL;
+	(void) expression_tree_walker((Node *) targetlist,
+								  cdbpullup_missingVarCheckWalker,
+								  (void *) &context);
+	missing = (list_length(context.findvars) < list_length(vars));
+	list_free(vars);
+	list_free(context.findvars);
+	return missing;
+}
+
+static List *
+cdbpullup_getall_vars(Node *node)
+{
+	List *vars = NIL;
+
+	(void) expression_tree_walker(node, cdbpullup_getall_vars_walker, (void *) &vars);
+	return vars;
+}
+
+static bool
+cdbpullup_getall_vars_walker(Node *node, void *context)
+{
+	List **vars = context;
+
+	if (node == NULL)
 		return false;
 
-	/*
-	 * Should also consider PlaceHolderVar in the targetlist.
-	 * See github issue: https://github.com/greenplum-db/gpdb/issues/10315
-	 */
-	if (IsA(node, Var) || IsA(node, PlaceHolderVar))
+	if (IsA(node, Var))
 	{
-		if (!targetlist)
-			return true;
+		*vars = list_append_unique_ptr(*vars, node);
+		return false;
+	}
 
-		/* is targetlist a List of TargetEntry? */
-		if (IsA(linitial(targetlist), TargetEntry))
-		{
-			if (tlist_member_ignore_relabel(node, (List *) targetlist))
-				return false;	/* this Var ok - go on to check rest of expr */
-		}
+	return expression_tree_walker(node, cdbpullup_getall_vars_walker, (void *) vars);
+}
+
+static bool
+cdbpullup_missingVarCheckWalker(Node *node, void *context)
+{
+	MissingVarContext *ctx = context;
 
-		/* targetlist must be a List of Expr */
-		else if (list_member((List *) targetlist, node))
-			return false;		/* this Var ok - go on to check rest of expr */
+	if (node == NULL)
+		return false;
 
-		return true;			/* Var is not in targetlist - quit the walk */
+	if (IsA(node, Var))
+	{
+		if (list_member(ctx->vars, node))
+			ctx->findvars = list_append_unique_ptr(ctx->findvars, node);
+		return false;
 	}
 
-	return expression_tree_walker((Node *) node, cdbpullup_missingVarWalker, targetlist);
-}								/* cdbpullup_missingVarWalker */
+	return expression_tree_walker(node, cdbpullup_missingVarCheckWalker, (void *) ctx);
+}
diff --git a/src/test/regress/expected/bfv_subquery.out b/src/test/regress/expected/bfv_subquery.out
index 741211b3b3..ff9f04adc2 100644
--- a/src/test/regress/expected/bfv_subquery.out
+++ b/src/test/regress/expected/bfv_subquery.out
@@ -555,3 +555,13 @@ limit 15;
  2 |   102
 (15 rows)
 
+-- test case for Github Issue 15079
+create table if not exists issue_15079(c0 real, c1 boolean);
+NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'c0' as the Greenplum Database data distribution key for this table.
+HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew.
+select * from (select bool_or(issue_15079.c1) from issue_15079 group by random()) as result;
+ bool_or 
+---------
+(0 rows)
+
+drop table issue_15079;
diff --git a/src/test/regress/expected/bfv_subquery_optimizer.out b/src/test/regress/expected/bfv_subquery_optimizer.out
index f89dd20935..db858dd038 100644
--- a/src/test/regress/expected/bfv_subquery_optimizer.out
+++ b/src/test/regress/expected/bfv_subquery_optimizer.out
@@ -550,3 +550,13 @@ limit 15;
  2 |   102
 (15 rows)
 
+-- test case for Github Issue 15079
+create table if not exists issue_15079(c0 real, c1 boolean);
+NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'c0' as the Greenplum Database data distribution key for this table.
+HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew.
+select * from (select bool_or(issue_15079.c1) from issue_15079 group by random()) as result;
+ bool_or 
+---------
+(0 rows)
+
+drop table issue_15079;
diff --git a/src/test/regress/sql/bfv_subquery.sql b/src/test/regress/sql/bfv_subquery.sql
index e03fcad15b..3e968e3e74 100644
--- a/src/test/regress/sql/bfv_subquery.sql
+++ b/src/test/regress/sql/bfv_subquery.sql
@@ -325,3 +325,8 @@ select with_test2.* from with_test2
 where value < all (select total from my_group_sum where my_group_sum.i = with_test2.i)
 order by 1,2
 limit 15;
+
+-- test case for Github Issue 15079
+create table if not exists issue_15079(c0 real, c1 boolean);
+select * from (select bool_or(issue_15079.c1) from issue_15079 group by random()) as result;
+drop table issue_15079;


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 06/20: Resolve FIXMEs in datetime.c

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit eba05de737eea2e62926749bdb729acc55dc6bfb
Author: Huansong Fu <fu...@gmail.com>
AuthorDate: Thu Jun 8 13:17:11 2023 -0700

    Resolve FIXMEs in datetime.c
    
    The old GPDB version of the calculation (added in 059ebb1da10b1f15487ba74dd1bada185093bbf2)
    and the new upstream version (added in aa2387e2fd53) are essentially the
    same: both were trying to optimize away sprintf and just do a byte-by-byte
    conversion. Similar to what upstream does for aa2387e2fd53, I did a
    comparison between the old GPDB version and the upstream version:
    
    ```
    -- pre-requisite:
    create table t(a int, b timestamp);
    insert into t select 0, '2018-01-01 10:10:00.1'::timestamp from generate_series(1,10000000);
    \timing on
    -- and measure the copy performance:
    copy t to '/tmp/copyto.csv';
    ```
    
    This would just test function AppendSeconds() but the other two functions
    should have similar results.
    
    1. Old AppendSeconds() (note: added back a missing function TrimTrailingZeros())
    ```
    postgres=# copy t to '/tmp/copyto.csv';
    COPY 10000000
    Time: 7232.761 ms (00:07.233)
    postgres=# copy t to '/tmp/copyto.csv';
    COPY 10000000
    Time: 7137.877 ms (00:07.138)
    postgres=# copy t to '/tmp/copyto.csv';
    COPY 10000000
    Time: 7189.453 ms (00:07.189)
    ```
    average 7186 ms
    
    2. New AppendSeconds()
    ```
    postgres=# copy t to '/tmp/copyto.csv';
    COPY 10000000
    Time: 7281.391 ms (00:07.281)
    postgres=# copy t to '/tmp/copyto.csv';
    COPY 10000000
    Time: 7231.939 ms (00:07.232)
    postgres=# copy t to '/tmp/copyto.csv';
    COPY 10000000
    Time: 7211.126 ms (00:07.211)
    ```
    average 7241 ms
    
    The difference is almost negligible.
    
    In addition, the old AppendSeconds() has some limitations:
    * Suprisingly, it does not even handle precision at all. It used to handle
    it for non-HAVE_INT64_TIMESTAMP case but since that case has gone in b9d092c962ea3262930e3c31a8c3d79b66ce9d43
    the precision parameter is completely ignored.
    * It makes an assumption about a two-digit seconds. It might not be a big
    deal in most cases but considering the many callsites of AppendSeconds()
    it's still better to make it more robust.
    
    Using upstream version is also also better in terms of getting more
    upstream optimization in future.
---
 src/backend/utils/adt/datetime.c | 68 +---------------------------------------
 1 file changed, 1 insertion(+), 67 deletions(-)

diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 841013b5d3..dc06cbfd7d 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -446,31 +446,6 @@ AppendSeconds(char *cp, int sec, fsec_t fsec, int precision, bool fillzeros)
 {
 	Assert(precision >= 0);
 
-	/* GPDB_96_MERGE_FIXME: We had this faster version in GPDB. PostgreSQL
-	 * also added faster versions in commit aa2387e2fd. Performance test is
-	 * the old GPDB variants are even faster, or if we could drop the diff
-	 * and just use upstream code. For now, the GPDB version is disabled
-	 * and we use the upstream code.
-	 */
-#if 0
-		int			j = 0;
-
-		if (fillzeros || abs(sec)  > 9)
-			cp[j++] = abs(sec)  / 10 + '0';
-		cp[j++] = abs(sec)  % 10 + '0';
-		cp[j++] = '.';
-		cp[j++] =  ((int) Abs(fsec) )/ 100000 + '0';
-		cp[j++] = ((int) Abs(fsec) ) / 10000 % 10 + '0';
-		cp[j++] = ((int) Abs(fsec) ) / 1000 % 10 + '0';
-		cp[j++] = ((int) Abs(fsec) ) / 100 % 10 + '0';
-		cp[j++] = ((int) Abs(fsec) ) / 10 % 10 + '0';
-		cp[j++] = ((int) Abs(fsec) ) % 10 + '0';
-		cp[j] = '\0';
-
-#endif
-
-	/* fsec_t is just an int32 */
-
 	if (fillzeros)
 		cp = pg_ultostr_zeropad(cp, Abs(sec), 2);
 	else
@@ -3978,31 +3953,13 @@ EncodeDateOnly(struct pg_tm *tm, int style, char *str)
 		case USE_ISO_DATES:
 		case USE_XSD_DATES:
 			/* compatible with ISO date formats */
-			/* GPDB_96_MERGE_FIXME: We had this faster version in GPDB. PostgreSQL
-			 * also added faster versions in commit aa2387e2fd. Performance test is
-			 * the old GPDB variants are even faster, or if we could drop the diff
-			 * and just use upstream code. For now, the GPDB version is disabled
-			 * and we use the upstream code.
-			 */
-#if 0
-			if (tm->tm_year > 0)
-			{
-				//				sprintf(str, "%04d-%02d-%02d",
-				//		tm->tm_year, tm->tm_mon, tm->tm_mday);
-				int j = 0;
-				fast_encode_date(tm, str, &j);
-			}
-			else
-				sprintf(str, "%04d-%02d-%02d %s",
-						-(tm->tm_year - 1), tm->tm_mon, tm->tm_mday, "BC");
-#else
 			str = pg_ultostr_zeropad(str,
 									 (tm->tm_year > 0) ? tm->tm_year : -(tm->tm_year - 1), 4);
 			*str++ = '-';
 			str = pg_ultostr_zeropad(str, tm->tm_mon, 2);
 			*str++ = '-';
 			str = pg_ultostr_zeropad(str, tm->tm_mday, 2);
-#endif
+
 			break;
 
 		case USE_SQL_DATES:
@@ -4075,28 +4032,6 @@ EncodeDateOnly(struct pg_tm *tm, int style, char *str)
 void
 EncodeTimeOnly(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, int style, char *str)
 {
-	/* GPDB_96_MERGE_FIXME: We had this faster version in GPDB. PostgreSQL
-	 * also added faster versions in commit aa2387e2fd. Performance test is
-	 * the old GPDB variants are even faster, or if we could drop the diff
-	 * and just use upstream code. For now, the GPDB version is disabled
-	 * and we use the upstream code.
-	 *
-	 * If we still need the old GPDB version, make sure it was actually correct.
-	 * It seems to ignore the 'print_tz' argument...
-	 */
-#if 0
-	str[0] = tm->tm_hour/10 + '0';
-	str[1] = tm->tm_hour % 10 + '0';
-	str[2] = ':';
-	str[3] = tm->tm_min/10 + '0';
-	str[4] = tm->tm_min % 10 + '0';
-	str[5] = ':';
-	str[6] = '\0';
-	str += strlen(str);
-
-	AppendSeconds(str, tm->tm_sec, fsec, MAX_TIME_PRECISION, true);
-#else
-
 	str = pg_ultostr_zeropad(str, tm->tm_hour, 2);
 	*str++ = ':';
 	str = pg_ultostr_zeropad(str, tm->tm_min, 2);
@@ -4105,7 +4040,6 @@ EncodeTimeOnly(struct pg_tm *tm, fsec_t fsec, bool print_tz, int tz, int style,
 	if (print_tz)
 		str = EncodeTimezone(str, tz, style);
 	*str = '\0';
-#endif
 }
 
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 13/20: Replace 'egrep' with 'grep -E'. (#15804)

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 6a2c96da0507f5fcff2cf609c2fe1efcc9fff50b
Author: Xing Guo <hi...@gmail.com>
AuthorDate: Mon Jun 19 17:21:34 2023 +0800

    Replace 'egrep' with 'grep -E'. (#15804)
    
    Each time we use 'egrep' there's a warning message emitted:
    'egrep: warning: egrep is obsolescent; using grep -E'. This patch
    replaces 'egrep' with 'grep -E'.
    
    Reference:
    - https://www.shellcheck.net/wiki/SC2196
---
 gpMgmt/bin/gpinitsystem             | 4 ++--
 gpMgmt/bin/lib/gp_bash_functions.sh | 1 -
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/gpMgmt/bin/gpinitsystem b/gpMgmt/bin/gpinitsystem
index 394504cd79..72a847a72a 100755
--- a/gpMgmt/bin/gpinitsystem
+++ b/gpMgmt/bin/gpinitsystem
@@ -1330,7 +1330,7 @@ CREATE_QD_DB () {
 		export PGPORT=$GP_PORT;$PG_CTL -w -l $GP_DIR/log/startup.log -D $GP_DIR -o "-i -p $GP_PORT -c gp_role=utility \
 		-m" start >> ${LOG_FILE} 2>&1
 		RET_TEXT="`$PG_CTL status -D $GP_DIR`"
-		RUNNING=`$ECHO $RET_TEXT|$EGREP -c "not running|neither"`
+		RUNNING=`$ECHO $RET_TEXT|$GREP -E -c "not running|neither"`
 		if [ $RUNNING -ne 0 ]; then
 				$CAT ${GP_DIR}.log|$TEE -a $LOG_FILE
 				ERROR_EXIT "[FATAL]:-Failed to start the Coordinator database in admin mode"
@@ -1649,7 +1649,7 @@ SCAN_LOG () {
 	LOG_MSG "[INFO]:-Start Function $FUNCNAME"
 	LOG_MSG "[INFO]:-Scanning utility log file for any warning messages" 1
 	SCAN_STRING="\[WARN\]|invalid|warning:|fatal|error"
-	if [ `$EGREP -i -w "$SCAN_STRING" $LOG_FILE|$GREP -v "\[INFO\]"|$GREP -v "[Start|End] Function ERROR_CHK"|$GREP -v "warning: enabling \"trust\" authentication for local connections"|$GREP -v "to the list of known hosts"|$GREP -v "LOG"|$WC -w` -ne 0 ];then
+	if [ `$GREP -E -i -w "$SCAN_STRING" $LOG_FILE|$GREP -v "\[INFO\]"|$GREP -v "[Start|End] Function ERROR_CHK"|$GREP -v "warning: enabling \"trust\" authentication for local connections"|$GREP -v "to the list of known hosts"|$GREP -v "LOG"|$WC -w` -ne 0 ];then
 		LOG_MSG "[WARN]:-*******************************************************" 1
 		LOG_MSG "[WARN]:-Scan of log file indicates that some warnings or errors" 1
 		LOG_MSG "[WARN]:-were generated during the array creation" 1
diff --git a/gpMgmt/bin/lib/gp_bash_functions.sh b/gpMgmt/bin/lib/gp_bash_functions.sh
index 6c97005e84..ee01ee8147 100755
--- a/gpMgmt/bin/lib/gp_bash_functions.sh
+++ b/gpMgmt/bin/lib/gp_bash_functions.sh
@@ -82,7 +82,6 @@ DIRNAME=`findCmdInPath dirname`
 ECHO=`findCmdInPath echo`
 FIND=`findCmdInPath find`
 GREP=`findCmdInPath grep`
-EGREP=`findCmdInPath egrep`
 HEAD=`findCmdInPath head`
 HOSTNAME=`findCmdInPath hostname`
 IP=`findCmdInPath ip`


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 17/20: starting segments in execute mode post recovery (#15599)

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 2d61bf5266dea8c786113886a19ca8986ceafaf6
Author: Piyush Chandwadkar <65...@users.noreply.github.com>
AuthorDate: Fri Jun 23 17:37:17 2023 +0530

    starting segments in execute mode post recovery (#15599)
    
    After recovery, the segment should start in execute mode.
    Currently, gprecoverseg starts the segment in utility mode after the recovery.
    With these changes make sure the segment is started in execute mode.
    Added tests to confirm segments are in execute mode after recovery.
    Currently testing if the segment is in utility mode by connecting to
    individual segments and checking the error thrown.
---
 gpMgmt/sbin/gpsegrecovery.py                                 | 2 +-
 gpMgmt/test/behave/mgmt_utils/gprecoverseg.feature           | 9 +++++++++
 gpMgmt/test/behave/mgmt_utils/steps/tablespace_mgmt_utils.py | 1 -
 3 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/gpMgmt/sbin/gpsegrecovery.py b/gpMgmt/sbin/gpsegrecovery.py
index 6b9f31bf3e..811a73ccbb 100644
--- a/gpMgmt/sbin/gpsegrecovery.py
+++ b/gpMgmt/sbin/gpsegrecovery.py
@@ -100,7 +100,7 @@ def start_segment(recovery_info, logger, era):
         , numContentsInCluster=0
         , era=era
         , mirrormode="mirror"
-        , utilityMode=True)
+        , utilityMode=False)
     logger.info(str(cmd))
     cmd.run(validateAfter=True)
 
diff --git a/gpMgmt/test/behave/mgmt_utils/gprecoverseg.feature b/gpMgmt/test/behave/mgmt_utils/gprecoverseg.feature
index 0e3e2501f5..3ba2d27e97 100644
--- a/gpMgmt/test/behave/mgmt_utils/gprecoverseg.feature
+++ b/gpMgmt/test/behave/mgmt_utils/gprecoverseg.feature
@@ -10,6 +10,7 @@ Feature: gprecoverseg tests
          Then gprecoverseg should return a return code of 0
           And the segments are synchronized
           And the tablespace is valid
+          And the database segments are in execute mode
 
         Given another tablespace is created with data
          When the user runs "gprecoverseg -ra"
@@ -17,6 +18,12 @@ Feature: gprecoverseg tests
           And the segments are synchronized
           And the tablespace is valid
           And the other tablespace is valid
+          And the database segments are in execute mode
+      Examples:
+        | scenario     | args               |
+        | incremental  | -a                 |
+        | differential | -a --differential  |
+        | full         | -aF                |
 
     Scenario: full recovery works with tablespaces
         Given the database is running
@@ -368,6 +375,7 @@ Feature: gprecoverseg tests
     Then gprecoverseg should return a return code of 0
     And the segments are synchronized
     And the tablespace is valid
+    And the database segments are in execute mode
 
     Given another tablespace is created with data
     When the user runs "gprecoverseg -ra"
@@ -375,6 +383,7 @@ Feature: gprecoverseg tests
     And the segments are synchronized
     And the tablespace is valid
     And the other tablespace is valid
+    And the database segments are in execute mode
 
   @concourse_cluster
   Scenario: full recovery works with tablespaces on a multi-host environment
diff --git a/gpMgmt/test/behave/mgmt_utils/steps/tablespace_mgmt_utils.py b/gpMgmt/test/behave/mgmt_utils/steps/tablespace_mgmt_utils.py
index 19043b2a59..9829ba4a12 100644
--- a/gpMgmt/test/behave/mgmt_utils/steps/tablespace_mgmt_utils.py
+++ b/gpMgmt/test/behave/mgmt_utils/steps/tablespace_mgmt_utils.py
@@ -197,7 +197,6 @@ def impl(context):
 def impl(context):
     context.tablespaces["myspace"].verify()
 
-
 @then('the tablespace is valid after gpexpand')
 def impl(context):
     for _, tbs in list(context.tablespaces.items()):


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 08/20: Remove FIXME in relation_open() (#15786)

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit f917aa344681029d2b73d7324b5265f80f1942e8
Author: Yongtao Huang <99...@users.noreply.github.com>
AuthorDate: Fri Jun 16 19:16:44 2023 +0800

    Remove FIXME in relation_open() (#15786)
    
    Long log:
    
    The error and errdetail are both valid. Please refer to
    [issue 15095](https://github.com/greenplum-db/gpdb/issues/15095).
    
    The mini repo below will trigger this errdetail
    ```
    create table t1(c1 int);
    create temporary table t2(c1 int);
    create index i0 on t1(c1);
    create index i0 on t2(c1);
    discard all;
    drop index i0;
    vacuum t1;
    ```
    So revome this fixme.
    
    Signed-off-by: Yongtao Huang <yo...@vmware.com>
---
 src/backend/access/common/relation.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/src/backend/access/common/relation.c b/src/backend/access/common/relation.c
index fb3f53b717..3b6ac578a1 100644
--- a/src/backend/access/common/relation.c
+++ b/src/backend/access/common/relation.c
@@ -58,7 +58,6 @@ relation_open(Oid relationId, LOCKMODE lockmode)
 	/* The relcache does all the real work... */
 	r = RelationIdGetRelation(relationId);
 
-	/* GPDB_12_AFTER_MERGE_FIXME: We had added the errdetail in GPDB. Is it still valid? */
 	if (!RelationIsValid(r))
 		ereport(ERROR,
 				(errcode(ERRCODE_UNDEFINED_TABLE),


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 09/20: Revert "Refactor cdbpullup_missingVarWalker."

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 2de0e285f5e8f2b580cbe4f3e4e200f0855f5e68
Author: Zhenghua Lyu <ka...@gmail.com>
AuthorDate: Sat Jun 17 02:30:51 2023 +0000

    Revert "Refactor cdbpullup_missingVarWalker."
    
    This reverts commit aff2a34333ab31d86e65505cefd3e6144dbe67da.
    
    The commit aff2a343  fixes some issues but introduces some new
    failures:
    
      * Github Issue 15794
      * Github Issue 15767
      * Github Issue 15793
    
    I think it is better to revert it now and do a new design and
    and refactor later.
---
 src/backend/cdb/cdbpullup.c                        | 112 ++++++---------------
 src/test/regress/expected/bfv_subquery.out         |  10 --
 .../regress/expected/bfv_subquery_optimizer.out    |  10 --
 src/test/regress/sql/bfv_subquery.sql              |   5 -
 4 files changed, 33 insertions(+), 104 deletions(-)

diff --git a/src/backend/cdb/cdbpullup.c b/src/backend/cdb/cdbpullup.c
index ae5945d0b6..c6f243ae08 100644
--- a/src/backend/cdb/cdbpullup.c
+++ b/src/backend/cdb/cdbpullup.c
@@ -27,18 +27,9 @@
 #include "cdb/cdbpullup.h"		/* me */
 
 
-static bool  cdbpullup_missingVarCheck(Expr *node, List *targetlist);
-static List *cdbpullup_getall_vars(Node *node);
-static bool  cdbpullup_getall_vars_walker(Node *node, void *context);
-static bool  cdbpullup_missingVarCheckWalker(Node *node, void *context);
+static bool cdbpullup_missingVarWalker(Expr *node, void *targetlist);
 static Expr *cdbpullup_make_expr(Index varno, AttrNumber varattno, Expr *oldexpr, bool modifyOld);
 
-
-typedef struct MissingVarContext
-{
-	List   *vars;
-	List   *findvars;
-} MissingVarContext;
 /*
  * cdbpullup_expr
  *
@@ -326,8 +317,10 @@ cdbpullup_findEclassInTargetList(EquivalenceClass *eclass, List *targetlist,
 
 		/* Return this item if all referenced Vars are in targetlist. */
 		if (!IsA(key, Var) &&
-			!cdbpullup_missingVarCheck(key, targetlist))
+			!cdbpullup_missingVarWalker(key, targetlist))
+		{
 			return key;
+		}
 	}
 
 	return NULL;
@@ -421,85 +414,46 @@ cdbpullup_make_expr(Index varno, AttrNumber varattno, Expr *oldexpr, bool modify
 	}
 }								/* cdbpullup_make_var */
 
+
 /*
- * cdbpullup_missingVarCheck
- *   Given an expression and a targetlist, check if the expression
- *   can be represented by the exprs in the targetlist.
+ * cdbpullup_missingVarWalker
+ *
+ * Returns true if some Var in expr is not in targetlist.
+ * 'targetlist' is either a List of TargetEntry, or a plain List of Expr.
  *
- *   This function return true when Expr node cannot be represented
- *   based on targetlist (which matches the name that we have missing vars).
+ * NB:  A Var in the expr is considered as matching a Var in the targetlist
+ * without regard for whether or not there is a RelabelType node atop the
+ * targetlist Var.
  *
- *   The implementaion is as below:
- *     1. build the list of all Vars in the Expr node, it is a set (created
- *        by list_append_unique_ptr in the helper function cdbpullup_getall_vars)
- *     2. walk the targetlist to build a set (created by list_append_unique_ptr)
- *        for all Vars in targetlist that is member of Varset built in 1st step,
- *        here membership test is using list_member which call equal to test.
- *     3. if the size of the 2nd step's var set is smaller than the 1st step's,
- *        it means the targetlist misses some vars.
  */
 static bool
-cdbpullup_missingVarCheck(Expr *node, List *targetlist)
+cdbpullup_missingVarWalker(Expr *node, void *targetlist)
 {
-	List             *vars    = NIL;
-	MissingVarContext context;
-	bool              missing;
-
-	vars = cdbpullup_getall_vars((Node *) node);
-	if (list_length(vars) == 0)
-		return true;
-
-	context.vars = vars;
-	context.findvars = NIL;
-	(void) expression_tree_walker((Node *) targetlist,
-								  cdbpullup_missingVarCheckWalker,
-								  (void *) &context);
-	missing = (list_length(context.findvars) < list_length(vars));
-	list_free(vars);
-	list_free(context.findvars);
-	return missing;
-}
-
-static List *
-cdbpullup_getall_vars(Node *node)
-{
-	List *vars = NIL;
-
-	(void) expression_tree_walker(node, cdbpullup_getall_vars_walker, (void *) &vars);
-	return vars;
-}
-
-static bool
-cdbpullup_getall_vars_walker(Node *node, void *context)
-{
-	List **vars = context;
-
-	if (node == NULL)
+	if (!node)
 		return false;
 
-	if (IsA(node, Var))
+	/*
+	 * Should also consider PlaceHolderVar in the targetlist.
+	 * See github issue: https://github.com/greenplum-db/gpdb/issues/10315
+	 */
+	if (IsA(node, Var) || IsA(node, PlaceHolderVar))
 	{
-		*vars = list_append_unique_ptr(*vars, node);
-		return false;
-	}
+		if (!targetlist)
+			return true;
 
-	return expression_tree_walker(node, cdbpullup_getall_vars_walker, (void *) vars);
-}
-
-static bool
-cdbpullup_missingVarCheckWalker(Node *node, void *context)
-{
-	MissingVarContext *ctx = context;
+		/* is targetlist a List of TargetEntry? */
+		if (IsA(linitial(targetlist), TargetEntry))
+		{
+			if (tlist_member_ignore_relabel(node, (List *) targetlist))
+				return false;	/* this Var ok - go on to check rest of expr */
+		}
 
-	if (node == NULL)
-		return false;
+		/* targetlist must be a List of Expr */
+		else if (list_member((List *) targetlist, node))
+			return false;		/* this Var ok - go on to check rest of expr */
 
-	if (IsA(node, Var))
-	{
-		if (list_member(ctx->vars, node))
-			ctx->findvars = list_append_unique_ptr(ctx->findvars, node);
-		return false;
+		return true;			/* Var is not in targetlist - quit the walk */
 	}
 
-	return expression_tree_walker(node, cdbpullup_missingVarCheckWalker, (void *) ctx);
-}
+	return expression_tree_walker((Node *) node, cdbpullup_missingVarWalker, targetlist);
+}								/* cdbpullup_missingVarWalker */
diff --git a/src/test/regress/expected/bfv_subquery.out b/src/test/regress/expected/bfv_subquery.out
index ff9f04adc2..741211b3b3 100644
--- a/src/test/regress/expected/bfv_subquery.out
+++ b/src/test/regress/expected/bfv_subquery.out
@@ -555,13 +555,3 @@ limit 15;
  2 |   102
 (15 rows)
 
--- test case for Github Issue 15079
-create table if not exists issue_15079(c0 real, c1 boolean);
-NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'c0' as the Greenplum Database data distribution key for this table.
-HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew.
-select * from (select bool_or(issue_15079.c1) from issue_15079 group by random()) as result;
- bool_or 
----------
-(0 rows)
-
-drop table issue_15079;
diff --git a/src/test/regress/expected/bfv_subquery_optimizer.out b/src/test/regress/expected/bfv_subquery_optimizer.out
index db858dd038..f89dd20935 100644
--- a/src/test/regress/expected/bfv_subquery_optimizer.out
+++ b/src/test/regress/expected/bfv_subquery_optimizer.out
@@ -550,13 +550,3 @@ limit 15;
  2 |   102
 (15 rows)
 
--- test case for Github Issue 15079
-create table if not exists issue_15079(c0 real, c1 boolean);
-NOTICE:  Table doesn't have 'DISTRIBUTED BY' clause -- Using column named 'c0' as the Greenplum Database data distribution key for this table.
-HINT:  The 'DISTRIBUTED BY' clause determines the distribution of data. Make sure column(s) chosen are the optimal data distribution key to minimize skew.
-select * from (select bool_or(issue_15079.c1) from issue_15079 group by random()) as result;
- bool_or 
----------
-(0 rows)
-
-drop table issue_15079;
diff --git a/src/test/regress/sql/bfv_subquery.sql b/src/test/regress/sql/bfv_subquery.sql
index 3e968e3e74..e03fcad15b 100644
--- a/src/test/regress/sql/bfv_subquery.sql
+++ b/src/test/regress/sql/bfv_subquery.sql
@@ -325,8 +325,3 @@ select with_test2.* from with_test2
 where value < all (select total from my_group_sum where my_group_sum.i = with_test2.i)
 order by 1,2
 limit 15;
-
--- test case for Github Issue 15079
-create table if not exists issue_15079(c0 real, c1 boolean);
-select * from (select bool_or(issue_15079.c1) from issue_15079 group by random()) as result;
-drop table issue_15079;


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 01/20: Bump minimum requirement of zstd to 1.4.0

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit c48753031ac492dc21dff4df2c3f62a7a4678841
Author: Marbin Tan <tm...@vmware.com>
AuthorDate: Wed Jun 7 14:42:27 2023 -0700

    Bump minimum requirement of zstd to 1.4.0
    
    zstd 1.4.0 introduces stabilization for some of the advanced APIs.
    It also has some performance enhancements compared to the older version.
    Given that the release has been out since 2019, we should follow suit
    with Postgres and upgrade to 1.4.0.
    
    gpdb-dev discussion: https://groups.google.com/a/greenplum.org/g/gpdb-dev/c/0PyQvNB2aNI
    
    Reviewed-by: Soumyadeep Chakraborty <so...@gmail.com>
    Reviewed-by: Ashwin Agrawal <aa...@vmware.com>
---
 configure    | 22 +++++++++++-----------
 configure.ac |  4 ++--
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/configure b/configure
index 6a7e4103ee..7ee6e2de51 100755
--- a/configure
+++ b/configure
@@ -10344,19 +10344,19 @@ $as_echo "$with_zstd" >&6; }
 if test "$with_zstd" = yes; then
 
 pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd >= 1.1.1" >&5
-$as_echo_n "checking for libzstd >= 1.1.1... " >&6; }
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for libzstd >= 1.4.0" >&5
+$as_echo_n "checking for libzstd >= 1.4.0... " >&6; }
 
 if test -n "$ZSTD_CFLAGS"; then
     pkg_cv_ZSTD_CFLAGS="$ZSTD_CFLAGS"
  elif test -n "$PKG_CONFIG"; then
     if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd >= 1.1.1\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libzstd >= 1.1.1") 2>&5
+    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd >= 1.4.0\""; } >&5
+  ($PKG_CONFIG --exists --print-errors "libzstd >= 1.4.0") 2>&5
   ac_status=$?
   $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
   test $ac_status = 0; }; then
-  pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd >= 1.1.1" 2>/dev/null`
+  pkg_cv_ZSTD_CFLAGS=`$PKG_CONFIG --cflags "libzstd >= 1.4.0" 2>/dev/null`
 		      test "x$?" != "x0" && pkg_failed=yes
 else
   pkg_failed=yes
@@ -10368,12 +10368,12 @@ if test -n "$ZSTD_LIBS"; then
     pkg_cv_ZSTD_LIBS="$ZSTD_LIBS"
  elif test -n "$PKG_CONFIG"; then
     if test -n "$PKG_CONFIG" && \
-    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd >= 1.1.1\""; } >&5
-  ($PKG_CONFIG --exists --print-errors "libzstd >= 1.1.1") 2>&5
+    { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libzstd >= 1.4.0\""; } >&5
+  ($PKG_CONFIG --exists --print-errors "libzstd >= 1.4.0") 2>&5
   ac_status=$?
   $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
   test $ac_status = 0; }; then
-  pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd >= 1.1.1" 2>/dev/null`
+  pkg_cv_ZSTD_LIBS=`$PKG_CONFIG --libs "libzstd >= 1.4.0" 2>/dev/null`
 		      test "x$?" != "x0" && pkg_failed=yes
 else
   pkg_failed=yes
@@ -10394,14 +10394,14 @@ else
         _pkg_short_errors_supported=no
 fi
         if test $_pkg_short_errors_supported = yes; then
-	        ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd >= 1.1.1" 2>&1`
+	        ZSTD_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libzstd >= 1.4.0" 2>&1`
         else
-	        ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd >= 1.1.1" 2>&1`
+	        ZSTD_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libzstd >= 1.4.0" 2>&1`
         fi
 	# Put the nasty error message in config.log where it belongs
 	echo "$ZSTD_PKG_ERRORS" >&5
 
-	as_fn_error $? "Package requirements (libzstd >= 1.1.1) were not met:
+	as_fn_error $? "Package requirements (libzstd >= 1.4.0) were not met:
 
 $ZSTD_PKG_ERRORS
 
diff --git a/configure.ac b/configure.ac
index aee14b35da..f5ad58194c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1264,8 +1264,8 @@ AC_MSG_RESULT([$with_zstd])
 AC_SUBST(with_zstd)
 
 if test "$with_zstd" = yes; then
-  dnl zstd_errors.h was renamed from error_public.h in v1.1.1
-  PKG_CHECK_MODULES([ZSTD], [libzstd >= 1.1.1])
+  dnl zstd_errors.h was renamed from error_public.h in v1.4.0
+  PKG_CHECK_MODULES([ZSTD], [libzstd >= 1.4.0])
 fi
 
 #


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 10/20: Only super user can set the GUC gp_resource_group_bypass.

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 2f11cacb742d752c6c272369dbef5279e97a7768
Author: Zhenghua Lyu <ka...@gmail.com>
AuthorDate: Tue May 30 03:33:00 2023 +0000

    Only super user can set the GUC gp_resource_group_bypass.
---
 src/backend/utils/misc/guc_gp.c                           | 2 +-
 src/test/isolation2/expected/resgroup/resgroup_bypass.out | 6 ++++++
 src/test/isolation2/sql/resgroup/resgroup_bypass.sql      | 5 +++++
 3 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/src/backend/utils/misc/guc_gp.c b/src/backend/utils/misc/guc_gp.c
index f37086e59f..e92de86dc3 100644
--- a/src/backend/utils/misc/guc_gp.c
+++ b/src/backend/utils/misc/guc_gp.c
@@ -2839,7 +2839,7 @@ struct config_bool ConfigureNamesBool_gp[] =
 	},
 
 	{
-		{"gp_resource_group_bypass", PGC_USERSET, RESOURCES,
+		{"gp_resource_group_bypass", PGC_SUSET, RESOURCES,
 			gettext_noop("If the value is true, the query in this session will not be limited by resource group."),
 			NULL
 		},
diff --git a/src/test/isolation2/expected/resgroup/resgroup_bypass.out b/src/test/isolation2/expected/resgroup/resgroup_bypass.out
index 5cff41d745..090be7983d 100644
--- a/src/test/isolation2/expected/resgroup/resgroup_bypass.out
+++ b/src/test/isolation2/expected/resgroup/resgroup_bypass.out
@@ -19,6 +19,12 @@ SET ROLE role_bypass;
 SET
 CREATE TABLE t_bypass(a int) distributed by (a);
 CREATE
+
+-- gp_resource_group_bypass can only be set by super user
+-- below set statement will error out
+set gp_resource_group_bypass = 1;
+ERROR:  permission denied to set parameter "gp_resource_group_bypass"
+
 RESET ROLE;
 RESET
 
diff --git a/src/test/isolation2/sql/resgroup/resgroup_bypass.sql b/src/test/isolation2/sql/resgroup/resgroup_bypass.sql
index 19c50771f7..d4d97fbd6b 100644
--- a/src/test/isolation2/sql/resgroup/resgroup_bypass.sql
+++ b/src/test/isolation2/sql/resgroup/resgroup_bypass.sql
@@ -12,6 +12,11 @@ CREATE ROLE role_bypass RESOURCE GROUP rg_bypass;
 
 SET ROLE role_bypass;
 CREATE TABLE t_bypass(a int) distributed by (a);
+
+-- gp_resource_group_bypass can only be set by super user
+-- below set statement will error out
+set gp_resource_group_bypass = 1;
+
 RESET ROLE;
 
 -- Session1: pure-catalog query will be unassigned and bypassed.


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 12/20: Update comment of createMotionLayerState() (#15800)

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 37c0b696a63568ec98bd89b38faa531e33772b0e
Author: Yongtao Huang <99...@users.noreply.github.com>
AuthorDate: Sun Jun 18 11:06:02 2023 +0800

    Update comment of createMotionLayerState() (#15800)
    
    Function `setExpectedReceivers()` is replaced by `UpdateMotionExpectedReceivers()`
    in this commit https://github.com/greenplum-db/gpdb/commit/b0353e0ad5
    Update the comment.
    
    Signed-off-by: Yongtao Huang <yo...@vmware.com>
---
 src/backend/cdb/motion/cdbmotion.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/backend/cdb/motion/cdbmotion.c b/src/backend/cdb/motion/cdbmotion.c
index 59e1cf80c0..dae5115926 100644
--- a/src/backend/cdb/motion/cdbmotion.c
+++ b/src/backend/cdb/motion/cdbmotion.c
@@ -222,7 +222,7 @@ createMotionLayerState(int maxMotNodeID)
 
 		/*
 		 * we'll just set this to 0.  later, ml_ipc will call
-		 * setExpectedReceivers() to set this if we are a "Receiving" motion node.
+		 * UpdateMotionExpectedReceivers() to set this if we are a "Receiving" motion node.
 		 */
 		pEntry->num_senders = 0;
 	}


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 15/20: Clean up comments.

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit e02db30aa424c51a99df6a13267f0fce307cce11
Author: Jianghua Yang <yj...@gmail.com>
AuthorDate: Mon Jan 13 09:49:49 2025 +0800

    Clean up comments.
---
 contrib/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/Makefile b/contrib/Makefile
index 84138c41c3..6600cc91c4 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -5,7 +5,7 @@ top_builddir = ..
 include $(top_builddir)/src/Makefile.global
 
 # Cloudberry Database specific changes to the targetlist:
-# tsearch2 and postgres_fdw are currently disabled due to test failures.
+# tsearch2 is currently disabled due to test failures.
 # cube and the related earthdistance are disabled since GPDB define CUBE
 # as a keyword. lo is disabled since large objects aren't supported.
 # adminpack is disabled since the functionality has been imported into


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 14/20: Post fix: replace 'egrep' with 'grep -E' in gpMgmt (#15810)

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 21404af3303034a10af3840b03bc8bc9d331f02b
Author: Yongtao Huang <99...@users.noreply.github.com>
AuthorDate: Mon Jun 19 23:13:18 2023 +0800

    Post fix: replace 'egrep' with 'grep -E' in gpMgmt (#15810)
    
    This is a post commit of [62bcf77](https://github.com/greenplum-db/gpdb/commit/62bcf771aef01841903abe96bfd9b2c3b6a5d91d)
    
    Signed-off-by: Yongtao Huang <yo...@vmware.com>
---
 gpMgmt/test/behave/mgmt_utils/gpinitsystem.feature | 2 +-
 gpMgmt/test/behave/mgmt_utils/steps/mgmt_utils.py  | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/gpMgmt/test/behave/mgmt_utils/gpinitsystem.feature b/gpMgmt/test/behave/mgmt_utils/gpinitsystem.feature
index 3b666cc3e0..c112c7aecd 100644
--- a/gpMgmt/test/behave/mgmt_utils/gpinitsystem.feature
+++ b/gpMgmt/test/behave/mgmt_utils/gpinitsystem.feature
@@ -256,7 +256,7 @@ Feature: gpinitsystem tests
         When the user runs command "source $GPHOME/greenplum_path.sh; __DCA_VERSION_FILE__=/tmp/gpinitsystem/gpdb-appliance-version $GPHOME/bin/gpinitsystem -a -c ../gpAux/gpdemo/clusterConfigFile"
         Then gpinitsystem should return a return code of 0
         # the log file must have the entry indicating that DCA specific configuration has been set
-        And the user runs command "egrep 'Setting DCA specific configuration values' ~/gpAdminLogs/gpinitsystem*log"
+        And the user runs command "grep -E 'Setting DCA specific configuration values' ~/gpAdminLogs/gpinitsystem*log"
 
     Scenario: gpinitsystem uses the system locale if no locale is specified
         Given the database is not running
diff --git a/gpMgmt/test/behave/mgmt_utils/steps/mgmt_utils.py b/gpMgmt/test/behave/mgmt_utils/steps/mgmt_utils.py
index e836abc0f9..d2f52f33b2 100644
--- a/gpMgmt/test/behave/mgmt_utils/steps/mgmt_utils.py
+++ b/gpMgmt/test/behave/mgmt_utils/steps/mgmt_utils.py
@@ -646,7 +646,7 @@ def impl(context, process_name, secs):
 @when('the user asynchronously sets up to end {process_name} process when {log_msg} is printed in the logs')
 def impl(context, process_name, log_msg):
     command = "while sleep 0.1; " \
-              "do if egrep --quiet %s  ~/gpAdminLogs/%s*log ; " \
+              "do if grep -E --quiet %s  ~/gpAdminLogs/%s*log ; " \
               "then ps ux | grep bin/%s |awk '{print $2}' | xargs kill ;break 2; " \
               "fi; done" % (log_msg, process_name, process_name)
     run_async_command(context, command)
@@ -654,7 +654,7 @@ def impl(context, process_name, log_msg):
 @then('the user asynchronously sets up to end {kill_process_name} process when {log_msg} is printed in the {logfile_name} logs')
 def impl(context, kill_process_name, log_msg, logfile_name):
     command = "while sleep 0.1; " \
-              "do if egrep --quiet %s  ~/gpAdminLogs/%s*log ; " \
+              "do if grep -E --quiet %s  ~/gpAdminLogs/%s*log ; " \
               "then ps ux | grep bin/%s |awk '{print $2}' | xargs kill -2 ;break 2; " \
               "fi; done" % (log_msg, logfile_name, kill_process_name)
     run_async_command(context, command)
@@ -1255,7 +1255,7 @@ def impl(context, options):
 @then('gpintsystem logs should {contain} lines about running backout script')
 def impl(context, contain):
     string_to_find = 'Run command bash .*backout_gpinitsystem.* on coordinator to remove these changes$'
-    command = "egrep '{}' ~/gpAdminLogs/gpinitsystem*log".format(string_to_find)
+    command = "grep -E '{}' ~/gpAdminLogs/gpinitsystem*log".format(string_to_find)
     run_command(context, command)
     if contain == "contain":
         if has_exception(context):


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 02/20: Move a FIXME from postgresql.conf.sample to guc.c

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 32d33bf966bb3d55a98000c8a95b52a596ca6677
Author: Huansong Fu <fu...@gmail.com>
AuthorDate: Thu Jun 8 07:29:16 2023 -0700

    Move a FIXME from postgresql.conf.sample to guc.c
    
    The sample configuration file gets copied over to the installation
    directory which is visible to users. So it is unnecessary to show a
    FIXME message there. Move it to guc.c instead.
    
    We will be tracking the actual fix of the FIXME separately.
---
 src/backend/utils/misc/guc.c                  | 1 +
 src/backend/utils/misc/postgresql.conf.sample | 2 --
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 59cfd86cec..d661ecffc8 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -3715,6 +3715,7 @@ static struct config_real ConfigureNamesReal[] =
 		DEFAULT_CPU_OPERATOR_COST, 0, DBL_MAX,
 		NULL, NULL, NULL
 	},
+	/* GPDB_96_MERGE_FIXME: figure out the appropriate default values for the two parallel gucs below. */
 	{
 		{"parallel_tuple_cost", PGC_USERSET, QUERY_TUNING_COST,
 			gettext_noop("Sets the planner's estimate of the cost of "
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 635d0ba3df..c44dc92899 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -406,8 +406,6 @@ max_prepared_transactions = 250		# can be 0 or more
 #cpu_tuple_cost = 0.01			# same scale as above
 #cpu_index_tuple_cost = 0.005		# same scale as above
 #cpu_operator_cost = 0.0025		# same scale as above
-
-# GPDB_96_MERGE_FIXME: figure out the appropriate values for the parallel gucs
 #parallel_tuple_cost = 0.1		# same scale as above
 #parallel_setup_cost = 1000.0	# same scale as above
 #parallel_tuple_cost = 0.1		# same scale as above


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 19/20: Fix multi-row DEFAULT handling for INSERT ... SELECT rules.

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit cdad9dc3916e2b342846c9589a21f995132a08af
Author: Dean Rasheed <de...@gmail.com>
AuthorDate: Thu Feb 23 10:57:46 2023 +0000

    Fix multi-row DEFAULT handling for INSERT ... SELECT rules.
    
    Given an updatable view with a DO ALSO INSERT ... SELECT rule, a
    multi-row INSERT ... VALUES query on the view fails if the VALUES list
    contains any DEFAULTs that are not replaced by view defaults. This
    manifests as an "unrecognized node type" error, or an Assert failure,
    in an assert-enabled build.
    
    The reason is that when RewriteQuery() attempts to replace the
    remaining DEFAULT items with NULLs in any product queries, using
    rewriteValuesRTEToNulls(), it assumes that the VALUES RTE is located
    at the same rangetable index in each product query. However, if the
    product query is an INSERT ... SELECT, then the VALUES RTE is actually
    in the SELECT part of that query (at the same index), rather than the
    top-level product query itself.
    
    Fix, by descending to the SELECT in such cases. Note that we can't
    simply use getInsertSelectQuery() for this, since that expects to be
    given a raw rule action with OLD and NEW placeholder entries, so we
    duplicate its logic instead.
    
    While at it, beef up the checks in getInsertSelectQuery() by checking
    that the jointree->fromlist node is indeed a RangeTblRef, and that the
    RTE it points to has rtekind == RTE_SUBQUERY.
    
    Per bug #17803, from Alexander Lakhin. Back-patch to all supported
    branches.
    
    Dean Rasheed, reviewed by Tom Lane.
    
    Discussion: https://postgr.es/m/17803-53c63ed4ecb4eac6%40postgresql.org
    (cherry picked from commit 98b83b7349821b05134e6f50f516ecac878cb91d)
---
 src/backend/rewrite/rewriteHandler.c               | 32 ++++++++++++++++++++--
 src/backend/rewrite/rewriteManip.c                 | 11 +++++---
 src/test/regress/expected/updatable_views.out      | 19 +++++++++++++
 .../regress/expected/updatable_views_optimizer.out | 19 +++++++++++++
 src/test/regress/sql/updatable_views.sql           | 10 +++++++
 5 files changed, 84 insertions(+), 7 deletions(-)

diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 4b98f8f28c..33868f5b4e 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1637,7 +1637,6 @@ rewriteValuesRTEToNulls(Query *parsetree, RangeTblEntry *rte)
 	List	   *newValues;
 	ListCell   *lc;
 
-	Assert(rte->rtekind == RTE_VALUES);
 	newValues = NIL;
 	foreach(lc, rte->values_lists)
 	{
@@ -3894,12 +3893,39 @@ RewriteQuery(Query *parsetree, List *rewrite_events)
 			/*
 			 * Each product query has its own copy of the VALUES RTE at the
 			 * same index in the rangetable, so we must finalize each one.
+			 *
+			 * Note that if the product query is an INSERT ... SELECT, then
+			 * the VALUES RTE will be at the same index in the SELECT part of
+			 * the product query rather than the top-level product query
+			 * itself.
 			 */
 			foreach(n, product_queries)
 			{
 				Query	   *pt = (Query *) lfirst(n);
-				RangeTblEntry *values_rte = rt_fetch(values_rte_index,
-													 pt->rtable);
+				RangeTblEntry *values_rte;
+
+				if (pt->commandType == CMD_INSERT &&
+					pt->jointree && IsA(pt->jointree, FromExpr) &&
+					list_length(pt->jointree->fromlist) == 1)
+				{
+					Node	   *jtnode = (Node *) linitial(pt->jointree->fromlist);
+
+					if (IsA(jtnode, RangeTblRef))
+					{
+						int			rtindex = ((RangeTblRef *) jtnode)->rtindex;
+						RangeTblEntry *src_rte = rt_fetch(rtindex, pt->rtable);
+
+						if (src_rte->rtekind == RTE_SUBQUERY &&
+							src_rte->subquery &&
+							IsA(src_rte->subquery, Query) &&
+							src_rte->subquery->commandType == CMD_SELECT)
+							pt = src_rte->subquery;
+					}
+				}
+
+				values_rte = rt_fetch(values_rte_index, pt->rtable);
+				if (values_rte->rtekind != RTE_VALUES)
+					elog(ERROR, "failed to find VALUES RTE in product query");
 
 				rewriteValuesRTEToNulls(pt, values_rte);
 			}
diff --git a/src/backend/rewrite/rewriteManip.c b/src/backend/rewrite/rewriteManip.c
index 50a641945d..ecd7a3ff06 100644
--- a/src/backend/rewrite/rewriteManip.c
+++ b/src/backend/rewrite/rewriteManip.c
@@ -1000,12 +1000,15 @@ getInsertSelectQuery(Query *parsetree, Query ***subquery_ptr)
 	if (list_length(parsetree->jointree->fromlist) != 1)
 		elog(ERROR, "expected to find SELECT subquery");
 	rtr = (RangeTblRef *) linitial(parsetree->jointree->fromlist);
-	Assert(IsA(rtr, RangeTblRef));
+	if (!IsA(rtr, RangeTblRef))
+		elog(ERROR, "expected to find SELECT subquery");
 	selectrte = rt_fetch(rtr->rtindex, parsetree->rtable);
-	selectquery = selectrte->subquery;
-	if (!(selectquery && IsA(selectquery, Query) &&
-		  selectquery->commandType == CMD_SELECT))
+	if (!(selectrte->rtekind == RTE_SUBQUERY &&
+		  selectrte->subquery &&
+		  IsA(selectrte->subquery, Query) &&
+		  selectrte->subquery->commandType == CMD_SELECT))
 		elog(ERROR, "expected to find SELECT subquery");
+	selectquery = selectrte->subquery;
 	if (list_length(selectquery->rtable) >= 2 &&
 		strcmp(rt_fetch(PRS2_OLD_VARNO, selectquery->rtable)->eref->aliasname,
 			   "old") == 0 &&
diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out
index 9a46eb0e21..e6642dc4e4 100644
--- a/src/test/regress/expected/updatable_views.out
+++ b/src/test/regress/expected/updatable_views.out
@@ -3048,6 +3048,25 @@ select * from base_tab_def order by a, c NULLS LAST;
     | View default  |               | View default | 
 (22 rows)
 
+-- Test a DO ALSO INSERT ... SELECT rule
+drop rule base_tab_def_view_ins_rule on base_tab_def_view;
+create rule base_tab_def_view_ins_rule as on insert to base_tab_def_view
+  do also insert into base_tab_def (a, b, e) select new.a, new.b, 'xxx';
+truncate base_tab_def;
+insert into base_tab_def_view values (1, default, default, default, default);
+insert into base_tab_def_view values (2, default, default, default, default),
+                                     (3, default, default, default, default);
+select * from base_tab_def order by a, e nulls first;
+ a |      b       |       c       |      d       |  e  
+---+--------------+---------------+--------------+-----
+ 1 | View default | Table default | View default | 
+ 1 | View default | Table default |              | xxx
+ 2 | View default | Table default | View default | 
+ 2 | View default | Table default |              | xxx
+ 3 | View default | Table default | View default | 
+ 3 | View default | Table default |              | xxx
+(6 rows)
+
 drop view base_tab_def_view;
 drop table base_tab_def;
 -- Test defaults with array assignments
diff --git a/src/test/regress/expected/updatable_views_optimizer.out b/src/test/regress/expected/updatable_views_optimizer.out
index 35838b814d..4f7e45cad0 100644
--- a/src/test/regress/expected/updatable_views_optimizer.out
+++ b/src/test/regress/expected/updatable_views_optimizer.out
@@ -3076,6 +3076,25 @@ select * from base_tab_def order by a, c NULLS LAST;
     | View default  |               | View default | 
 (22 rows)
 
+-- Test a DO ALSO INSERT ... SELECT rule
+drop rule base_tab_def_view_ins_rule on base_tab_def_view;
+create rule base_tab_def_view_ins_rule as on insert to base_tab_def_view
+  do also insert into base_tab_def (a, b, e) select new.a, new.b, 'xxx';
+truncate base_tab_def;
+insert into base_tab_def_view values (1, default, default, default, default);
+insert into base_tab_def_view values (2, default, default, default, default),
+                                     (3, default, default, default, default);
+select * from base_tab_def order by a, e nulls first;
+ a |      b       |       c       |      d       |  e  
+---+--------------+---------------+--------------+-----
+ 1 | View default | Table default | View default | 
+ 1 | View default | Table default |              | xxx
+ 2 | View default | Table default | View default | 
+ 2 | View default | Table default |              | xxx
+ 3 | View default | Table default | View default | 
+ 3 | View default | Table default |              | xxx
+(6 rows)
+
 drop view base_tab_def_view;
 drop table base_tab_def;
 -- Test defaults with array assignments
diff --git a/src/test/regress/sql/updatable_views.sql b/src/test/regress/sql/updatable_views.sql
index bbfbf4f1cc..4e3f3c88e5 100644
--- a/src/test/regress/sql/updatable_views.sql
+++ b/src/test/regress/sql/updatable_views.sql
@@ -1600,6 +1600,16 @@ insert into base_tab_def_view values (15, default, default, default, default),
 insert into base_tab_def_view values (17), (default);
 select * from base_tab_def order by a, c NULLS LAST;
 
+-- Test a DO ALSO INSERT ... SELECT rule
+drop rule base_tab_def_view_ins_rule on base_tab_def_view;
+create rule base_tab_def_view_ins_rule as on insert to base_tab_def_view
+  do also insert into base_tab_def (a, b, e) select new.a, new.b, 'xxx';
+truncate base_tab_def;
+insert into base_tab_def_view values (1, default, default, default, default);
+insert into base_tab_def_view values (2, default, default, default, default),
+                                     (3, default, default, default, default);
+select * from base_tab_def order by a, e nulls first;
+
 drop view base_tab_def_view;
 drop table base_tab_def;
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 11/20: Enable `wal_compression` by default

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 81d4dd86561583159a9ac709c682e00c468ecd97
Author: Marbin Tan <tm...@vmware.com>
AuthorDate: Tue May 9 14:22:28 2023 -0700

    Enable `wal_compression` by default
    
    Based on recent findinds, `wal_compression` may give huge
    benefits when workloads generates large amounts of FPIs (Full Page
    Images). With `wal_compression`, the WAL generated can be reduced by
    20-30% and even higher when loading high volumes of data causes
    checkpoints to occur too frequently; checkpoints occuring too frequently
    causes more FPIs to be written to WAL.
    
    When the amount of WAL generated is reduced, this will inherently reduce
    the amount of data being transferred via interconnect to the mirrors.
    Furthermore, there will be a reduction in disk I/O as less data are generated.
    
    In a multi-node with mirrored environment, the overall duration of load will be
    reduced by 6-15%. `wal_compression` does not directly effect AO/CO,
    however, as part of AO/CO is comprised of auxiliary heap tables, there
    is a small gain in performance. This can be seen with a 1TB TPC-DS AO/CO
    load duration speed-up of 4%.
    
    Note that `full_page_writes` must be on to see any benefits mentioned
    above.
    
    ZSTD allocates its own memory. Although this is usually a concern, it is
    not in our case since a single backend is only compressing a 32K page at
    a given time.
    We may have multiple backends in a single host, but this will not
    significally increase memory usage.
    
    For further details of the discussion, see gpdb-dev mailing list:
    https://groups.google.com/a/greenplum.org/g/gpdb-dev/c/0PyQvNB2aNI
    
    -----------
    Test update
    -----------
    Due to enabling `wal_compression`, the test to throttle no longer
    triggers correctly; increase the amount of rows to trigger the expected
    behavior again.
    
    Reviewed-by: Soumyadeep Chakraborty <so...@gmail.com>
    Reviewed-by: Ashwin Agrawal <aa...@vmware.com>
---
 src/backend/utils/misc/guc.c                               |  2 +-
 src/backend/utils/misc/postgresql.conf.sample              |  2 +-
 src/test/isolation2/expected/segwalrep/select_throttle.out | 10 +++++-----
 src/test/isolation2/sql/segwalrep/select_throttle.sql      |  2 +-
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index d661ecffc8..521489c3b7 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -1387,7 +1387,7 @@ static struct config_bool ConfigureNamesBool[] =
 			NULL
 		},
 		&wal_compression,
-		false,
+		true,
 		NULL, NULL, NULL
 	},
 
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index c44dc92899..0908c41223 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -223,7 +223,7 @@ max_prepared_transactions = 250		# can be 0 or more
 #full_page_writes = on			# recover from partial page writes
 #wal_log_hints = off			# also do full page writes of non-critical updates
 					# (change requires restart)
-#wal_compression = off			# enable compression of full-page writes
+#wal_compression = on			# enable compression of full-page writes
 #wal_init_zero = on			# zero-fill new WAL files
 #wal_recycle = on			# recycle WAL files
 #wal_buffers = -1			# min 32kB, -1 sets based on shared_buffers
diff --git a/src/test/isolation2/expected/segwalrep/select_throttle.out b/src/test/isolation2/expected/segwalrep/select_throttle.out
index 1eea7e35d1..1d72208593 100644
--- a/src/test/isolation2/expected/segwalrep/select_throttle.out
+++ b/src/test/isolation2/expected/segwalrep/select_throttle.out
@@ -26,8 +26,8 @@ INSERT INTO select_no_throttle SELECT generate_series (1, 10);
 INSERT 10
 CREATE TABLE select_throttle(a int) DISTRIBUTED BY (a);
 CREATE
-INSERT INTO select_throttle SELECT generate_series (1, 100000);
-INSERT 100000
+INSERT INTO select_throttle SELECT generate_series (1, 900000);
+INSERT 900000
 
 -- Enable tuple hints so that buffer will be marked dirty upon a hint bit change
 -- (so that we don't have to wait for the tuple to age. See logic in markDirty)
@@ -82,9 +82,9 @@ SELECT gp_inject_fault_infinite('wal_sender_loop', 'reset', dbid) FROM gp_segmen
 -- after this, system continue to proceed
 
 1U<:  <... completed>
- count 
--------
- 33327 
+ count  
+--------
+ 299393 
 (1 row)
 
 SELECT wait_until_all_segments_synchronized();
diff --git a/src/test/isolation2/sql/segwalrep/select_throttle.sql b/src/test/isolation2/sql/segwalrep/select_throttle.sql
index 5094e2b1ed..a0223ed868 100644
--- a/src/test/isolation2/sql/segwalrep/select_throttle.sql
+++ b/src/test/isolation2/sql/segwalrep/select_throttle.sql
@@ -18,7 +18,7 @@ SELECT pg_reload_conf();
 CREATE TABLE select_no_throttle(a int) DISTRIBUTED BY (a);
 INSERT INTO select_no_throttle SELECT generate_series (1, 10);
 CREATE TABLE select_throttle(a int) DISTRIBUTED BY (a);
-INSERT INTO select_throttle SELECT generate_series (1, 100000);
+INSERT INTO select_throttle SELECT generate_series (1, 900000);
 
 -- Enable tuple hints so that buffer will be marked dirty upon a hint bit change
 -- (so that we don't have to wait for the tuple to age. See logic in markDirty)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 05/20: CI fix: don't log too much interconnect events (#15761)

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit f0fd4bd2298b3d9a327fce283633750c6fa63d12
Author: Adam Lee <ad...@vmware.com>
AuthorDate: Tue Jun 13 21:42:38 2023 +0800

    CI fix: don't log too much interconnect events (#15761)
    
    Align this teardown event logging with interconnect TCP.
---
 contrib/interconnect/udp/ic_udpifc.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/interconnect/udp/ic_udpifc.c b/contrib/interconnect/udp/ic_udpifc.c
index a381b4ea5b..2fe2426d3d 100644
--- a/contrib/interconnect/udp/ic_udpifc.c
+++ b/contrib/interconnect/udp/ic_udpifc.c
@@ -3748,8 +3748,8 @@ TeardownUDPIFCInterconnect_Internal(ChunkTransportState *transportStates,
 		pfree(transportStates);
 	}
 
-	if (gp_log_interconnect >= GPVARS_VERBOSITY_TERSE)
-		elog(DEBUG1, "TeardownUDPIFCInterconnect successful");
+	if (gp_log_interconnect >= GPVARS_VERBOSITY_DEBUG)
+		elog(DEBUG4, "TeardownUDPIFCInterconnect successful");
 
 	RESUME_INTERRUPTS();
 }


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 03/20: Remove deadcode cdbpullup_isExprCoveredByTargetlist.

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit b1aebd3afe01557ceea40801082f9cddb6429a97
Author: Zhenghua Lyu <ka...@gmail.com>
AuthorDate: Mon Jun 5 13:35:36 2023 +0000

    Remove deadcode cdbpullup_isExprCoveredByTargetlist.
    
    Commit 93abe741cd67 remove code that calling this function.
---
 src/backend/cdb/cdbpullup.c | 47 ---------------------------------------------
 src/include/cdb/cdbpullup.h | 20 -------------------
 2 files changed, 67 deletions(-)

diff --git a/src/backend/cdb/cdbpullup.c b/src/backend/cdb/cdbpullup.c
index cf1e318ae9..c6f243ae08 100644
--- a/src/backend/cdb/cdbpullup.c
+++ b/src/backend/cdb/cdbpullup.c
@@ -351,52 +351,6 @@ cdbpullup_truncatePathKeysForTargetList(List *pathkeys, List *targetlist)
 	return new_pathkeys;
 }
 
-/*
- * cdbpullup_isExprCoveredByTargetlist
- *
- * Returns true if 'expr' is in 'targetlist', or if 'expr' contains no
- * Var node of the current query level that is not in 'targetlist'.
- *
- * If 'expr' is a List, returns false if the above condition is false for
- * some member of the list.
- *
- * 'targetlist' is a List of TargetEntry.
- *
- * NB:  A Var in the expr is considered as matching a Var in the targetlist
- * without regard for whether or not there is a RelabelType node atop the
- * targetlist Var.
- *
- * See also: cdbpullup_missing_var_walker
- */
-bool
-cdbpullup_isExprCoveredByTargetlist(Expr *expr, List *targetlist)
-{
-	ListCell   *cell;
-
-	/* List of Expr?  Verify that all items are covered. */
-	if (IsA(expr, List))
-	{
-		foreach(cell, (List *) expr)
-		{
-			Expr	   *item = (Expr *) lfirst(cell);
-
-			/* The whole expr or all of its Vars must be in targetlist. */
-			if (!tlist_member_ignore_relabel(item, targetlist) &&
-				cdbpullup_missingVarWalker(item, targetlist))
-				return false;
-		}
-	}
-
-	/* The whole expr or all of its Vars must be in targetlist. */
-	else if (!tlist_member_ignore_relabel(expr, targetlist) &&
-			 cdbpullup_missingVarWalker(expr, targetlist))
-		return false;
-
-	/* expr is evaluable on rows projected thru targetlist */
-	return true;
-}								/* cdbpullup_isExprCoveredByTlist */
-
-
 /*
  * cdbpullup_make_var
  *
@@ -471,7 +425,6 @@ cdbpullup_make_expr(Index varno, AttrNumber varattno, Expr *oldexpr, bool modify
  * without regard for whether or not there is a RelabelType node atop the
  * targetlist Var.
  *
- * See also: cdbpullup_isExprCoveredByTargetlist
  */
 static bool
 cdbpullup_missingVarWalker(Expr *node, void *targetlist)
diff --git a/src/include/cdb/cdbpullup.h b/src/include/cdb/cdbpullup.h
index cf5a9a0ff9..535b2987ce 100644
--- a/src/include/cdb/cdbpullup.h
+++ b/src/include/cdb/cdbpullup.h
@@ -59,24 +59,4 @@ extern Expr *cdbpullup_findEclassInTargetList(EquivalenceClass *eclass, List *ta
 
 extern List *cdbpullup_truncatePathKeysForTargetList(List *pathkeys, List *targetlist);
 
-/*
- * cdbpullup_isExprCoveredByTargetlist
- *
- * Returns true if 'expr' is in 'targetlist', or if 'expr' contains no
- * Var node of the current query level that is not in 'targetlist'.
- *
- * If 'expr' is a List, returns false if the above condition is false for
- * some member of the list.
- *
- * 'targetlist' is a List of TargetEntry.
- *
- * NB:  A Var in the expr is considered as matching a Var in the targetlist
- * without regard for whether or not there is a RelabelType node atop the 
- * targetlist Var.
- *
- * See also: cdbpullup_missing_var_walker
- */
-bool
-cdbpullup_isExprCoveredByTargetlist(Expr *expr, List *targetlist);
-
 #endif   /* CDBPULLUP_H */


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 18/20: Yet further fixes for multi-row VALUES lists for updatable views.

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 7f68241e2c6a2f023ab5fd84d00f2d7da6d2c844
Author: Tom Lane <tg...@sss.pgh.pa.us>
AuthorDate: Tue Oct 11 18:24:15 2022 -0400

    Yet further fixes for multi-row VALUES lists for updatable views.
    
    DEFAULT markers appearing in an INSERT on an updatable view
    could be mis-processed if they were in a multi-row VALUES clause.
    This would lead to strange errors such as "cache lookup failed
    for type NNNN", or in older branches even to crashes.
    
    The cause is that commit 41531e42d tried to re-use rewriteValuesRTE()
    to remove any SetToDefault nodes (that hadn't previously been replaced
    by the view's own default values) appearing in "product" queries,
    that is DO ALSO queries.  That's fundamentally wrong because the
    DO ALSO queries might not even be INSERTs; and even if they are,
    their targetlists don't necessarily match the view's column list,
    so that almost all the logic in rewriteValuesRTE() is inapplicable.
    
    What we want is a narrow focus on replacing any such nodes with NULL
    constants.  (That is, in this context we are interpreting the defaults
    as being strictly those of the view itself; and we already replaced
    any that aren't NULL.)  We could add still more !force_nulls tests
    to further lobotomize rewriteValuesRTE(); but it seems cleaner to
    split out this case to a new function, restoring rewriteValuesRTE()
    to the charter it had before.
    
    Per bug #17633 from jiye_sw.  Patch by me, but thanks to
    Richard Guo and Japin Li for initial investigation.
    Back-patch to all supported branches, as the previous fix was.
    
    Discussion: https://postgr.es/m/17633-98cc85e1fa91e905@postgresql.org
    (cherry picked from commit abc510fa2a34eba00406e9ce330b77e1d9c4ccdf)
---
 src/backend/rewrite/rewriteHandler.c               | 95 +++++++++++++---------
 src/test/regress/expected/updatable_views.out      | 22 +++++
 .../regress/expected/updatable_views_optimizer.out | 22 +++++
 src/test/regress/sql/updatable_views.sql           | 17 ++++
 4 files changed, 119 insertions(+), 37 deletions(-)

diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 27e65a0b1d..4b98f8f28c 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -81,8 +81,9 @@ static TargetEntry *process_matched_tle(TargetEntry *src_tle,
 static Node *get_assignment_input(Node *node);
 static Bitmapset *findDefaultOnlyColumns(RangeTblEntry *rte);
 static bool rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
-							 Relation target_relation, bool force_nulls,
+							 Relation target_relation,
 							 Bitmapset *unused_cols);
+static void rewriteValuesRTEToNulls(Query *parsetree, RangeTblEntry *rte);
 static void markQueryForLocking(Query *qry, Node *jtnode,
 								LockClauseStrength strength, LockWaitPolicy waitPolicy,
 								bool pushedDown);
@@ -1424,27 +1425,6 @@ findDefaultOnlyColumns(RangeTblEntry *rte)
  * all DEFAULT items are replaced, and if the target relation doesn't have a
  * default, the value is explicitly set to NULL.
  *
- * Additionally, if force_nulls is true, the target relation's defaults are
- * ignored and all DEFAULT items in the VALUES list are explicitly set to
- * NULL, regardless of the target relation's type.  This is used for the
- * product queries generated by DO ALSO rules attached to an auto-updatable
- * view, for which we will have already called this function with force_nulls
- * false.  For these product queries, we must then force any remaining DEFAULT
- * items to NULL to provide concrete values for the rule actions.
- * Essentially, this is a mix of the 2 cases above --- the original query is
- * an insert into an auto-updatable view, and the product queries are inserts
- * into a rule-updatable view.
- *
- * Finally, if a DEFAULT item is found in a column mentioned in unused_cols,
- * it is explicitly set to NULL.  This happens for columns in the VALUES RTE
- * whose corresponding targetlist entries have already been replaced with the
- * relation's default expressions, so that any values in those columns of the
- * VALUES RTE are no longer used.  This can happen for identity and generated
- * columns (if INSERT ... OVERRIDING USER VALUE is used, or all the values to
- * be inserted are DEFAULT).  In principle we could replace all entries in
- * such a column with NULL, whether DEFAULT or not; but it doesn't seem worth
- * the trouble.
- *
  * Note that we may have subscripted or field assignment targetlist entries,
  * as well as more complex expressions from already-replaced DEFAULT items if
  * we have recursed to here for an auto-updatable view. However, it ought to
@@ -1458,7 +1438,7 @@ findDefaultOnlyColumns(RangeTblEntry *rte)
  */
 static bool
 rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
-				 Relation target_relation, bool force_nulls,
+				 Relation target_relation,
 				 Bitmapset *unused_cols)
 {
 	List	   *newValues;
@@ -1468,15 +1448,16 @@ rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
 	int			numattrs;
 	int		   *attrnos;
 
+	/* Steps below are not sensible for non-INSERT queries */
+	Assert(parsetree->commandType == CMD_INSERT);
+	Assert(rte->rtekind == RTE_VALUES);
+
 	/*
 	 * Rebuilding all the lists is a pretty expensive proposition in a big
 	 * VALUES list, and it's a waste of time if there aren't any DEFAULT
 	 * placeholders.  So first scan to see if there are any.
-	 *
-	 * We skip this check if force_nulls is true, because we know that there
-	 * are DEFAULT items present in that case.
 	 */
-	if (!force_nulls && !searchForDefault(rte))
+	if (!searchForDefault(rte))
 		return true;			/* nothing to do */
 
 	/*
@@ -1510,12 +1491,10 @@ rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
 	/*
 	 * Check if the target relation is an auto-updatable view, in which case
 	 * unresolved defaults will be left untouched rather than being set to
-	 * NULL.  If force_nulls is true, we always set DEFAULT items to NULL, so
-	 * skip this check in that case --- it isn't an auto-updatable view.
+	 * NULL.
 	 */
 	isAutoUpdatableView = false;
-	if (!force_nulls &&
-		target_relation->rd_rel->relkind == RELKIND_VIEW &&
+	if (target_relation->rd_rel->relkind == RELKIND_VIEW &&
 		!view_has_instead_trigger(target_relation, CMD_INSERT))
 	{
 		List	   *locks;
@@ -1589,9 +1568,10 @@ rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
 
 				if (attrno == 0)
 					elog(ERROR, "cannot set value in column %d to DEFAULT", i);
+				Assert(attrno > 0 && attrno <= target_relation->rd_att->natts);
 				att_tup = TupleDescAttr(target_relation->rd_att, attrno - 1);
 
-				if (!force_nulls && !att_tup->attisdropped)
+				if (!att_tup->attisdropped)
 					new_expr = build_column_default(target_relation, attrno);
 				else
 					new_expr = NULL;	/* force a NULL if dropped */
@@ -1641,6 +1621,50 @@ rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
 	return allReplaced;
 }
 
+/*
+ * Mop up any remaining DEFAULT items in the given VALUES RTE by
+ * replacing them with NULL constants.
+ *
+ * This is used for the product queries generated by DO ALSO rules attached to
+ * an auto-updatable view.  The action can't depend on the "target relation"
+ * since the product query might not have one (it needn't be an INSERT).
+ * Essentially, such queries are treated as being attached to a rule-updatable
+ * view.
+ */
+static void
+rewriteValuesRTEToNulls(Query *parsetree, RangeTblEntry *rte)
+{
+	List	   *newValues;
+	ListCell   *lc;
+
+	Assert(rte->rtekind == RTE_VALUES);
+	newValues = NIL;
+	foreach(lc, rte->values_lists)
+	{
+		List	   *sublist = (List *) lfirst(lc);
+		List	   *newList = NIL;
+		ListCell   *lc2;
+
+		foreach(lc2, sublist)
+		{
+			Node	   *col = (Node *) lfirst(lc2);
+
+			if (IsA(col, SetToDefault))
+			{
+				SetToDefault *def = (SetToDefault *) col;
+
+				newList = lappend(newList, makeNullConst(def->typeId,
+														 def->typeMod,
+														 def->collation));
+			}
+			else
+				newList = lappend(newList, col);
+		}
+		newValues = lappend(newValues, newList);
+	}
+	rte->values_lists = newValues;
+}
+
 
 /*
  * Record in target_rte->extraUpdatedCols the indexes of any generated columns
@@ -3796,7 +3820,7 @@ RewriteQuery(Query *parsetree, List *rewrite_events)
 															&unused_values_attrnos);
 				/* ... and the VALUES expression lists */
 				if (!rewriteValuesRTE(parsetree, values_rte, values_rte_index,
-									  rt_entry_relation, false,
+									  rt_entry_relation,
 									  unused_values_attrnos))
 					defaults_remaining = true;
 			}
@@ -3877,10 +3901,7 @@ RewriteQuery(Query *parsetree, List *rewrite_events)
 				RangeTblEntry *values_rte = rt_fetch(values_rte_index,
 													 pt->rtable);
 
-				rewriteValuesRTE(pt, values_rte, values_rte_index,
-								 rt_entry_relation,
-								 true,	/* Force remaining defaults to NULL */
-								 NULL);
+				rewriteValuesRTEToNulls(pt, values_rte);
 			}
 		}
 
diff --git a/src/test/regress/expected/updatable_views.out b/src/test/regress/expected/updatable_views.out
index e02471ad24..9a46eb0e21 100644
--- a/src/test/regress/expected/updatable_views.out
+++ b/src/test/regress/expected/updatable_views.out
@@ -442,8 +442,30 @@ EXPLAIN (costs off) DELETE FROM rw_view1 WHERE a=5;
  Optimizer: Postgres query optimizer
 (4 rows)
 
+-- it's still updatable if we add a DO ALSO rule
+CREATE TABLE base_tbl_hist(ts timestamptz default now(), a int, b text);
+CREATE RULE base_tbl_log AS ON INSERT TO rw_view1 DO ALSO
+  INSERT INTO base_tbl_hist(a,b) VALUES(new.a, new.b);
+SELECT table_name, is_updatable, is_insertable_into
+  FROM information_schema.views
+ WHERE table_name = 'rw_view1';
+ table_name | is_updatable | is_insertable_into 
+------------+--------------+--------------------
+ rw_view1   | YES          | YES
+(1 row)
+
+-- Check behavior with DEFAULTs (bug #17633)
+INSERT INTO rw_view1 VALUES (9, DEFAULT), (10, DEFAULT);
+SELECT a, b FROM base_tbl_hist;
+ a  | b 
+----+---
+  9 | 
+ 10 | 
+(2 rows)
+
 DROP TABLE base_tbl CASCADE;
 NOTICE:  drop cascades to view rw_view1
+DROP TABLE base_tbl_hist;
 -- view on top of view
 CREATE TABLE base_tbl (a int PRIMARY KEY, b text DEFAULT 'Unspecified');
 INSERT INTO base_tbl SELECT i, 'Row ' || i FROM generate_series(-2, 2) g(i);
diff --git a/src/test/regress/expected/updatable_views_optimizer.out b/src/test/regress/expected/updatable_views_optimizer.out
index c276f00254..35838b814d 100644
--- a/src/test/regress/expected/updatable_views_optimizer.out
+++ b/src/test/regress/expected/updatable_views_optimizer.out
@@ -448,8 +448,30 @@ EXPLAIN (costs off) DELETE FROM rw_view1 WHERE a=5;
  Optimizer: Pivotal Optimizer (GPORCA)
 (4 rows)
 
+-- it's still updatable if we add a DO ALSO rule
+CREATE TABLE base_tbl_hist(ts timestamptz default now(), a int, b text);
+CREATE RULE base_tbl_log AS ON INSERT TO rw_view1 DO ALSO
+  INSERT INTO base_tbl_hist(a,b) VALUES(new.a, new.b);
+SELECT table_name, is_updatable, is_insertable_into
+  FROM information_schema.views
+ WHERE table_name = 'rw_view1';
+ table_name | is_updatable | is_insertable_into 
+------------+--------------+--------------------
+ rw_view1   | YES          | YES
+(1 row)
+
+-- Check behavior with DEFAULTs (bug #17633)
+INSERT INTO rw_view1 VALUES (9, DEFAULT), (10, DEFAULT);
+SELECT a, b FROM base_tbl_hist;
+ a  | b 
+----+---
+  9 | 
+ 10 | 
+(2 rows)
+
 DROP TABLE base_tbl CASCADE;
 NOTICE:  drop cascades to view rw_view1
+DROP TABLE base_tbl_hist;
 -- view on top of view
 CREATE TABLE base_tbl (a int PRIMARY KEY, b text DEFAULT 'Unspecified');
 INSERT INTO base_tbl SELECT i, 'Row ' || i FROM generate_series(-2, 2) g(i);
diff --git a/src/test/regress/sql/updatable_views.sql b/src/test/regress/sql/updatable_views.sql
index db21ffbda1..bbfbf4f1cc 100644
--- a/src/test/regress/sql/updatable_views.sql
+++ b/src/test/regress/sql/updatable_views.sql
@@ -154,7 +154,24 @@ SELECT * FROM base_tbl;
 EXPLAIN (costs off) UPDATE rw_view1 SET a=6 WHERE a=5;
 EXPLAIN (costs off) DELETE FROM rw_view1 WHERE a=5;
 
+-- it's still updatable if we add a DO ALSO rule
+
+CREATE TABLE base_tbl_hist(ts timestamptz default now(), a int, b text);
+
+CREATE RULE base_tbl_log AS ON INSERT TO rw_view1 DO ALSO
+  INSERT INTO base_tbl_hist(a,b) VALUES(new.a, new.b);
+
+SELECT table_name, is_updatable, is_insertable_into
+  FROM information_schema.views
+ WHERE table_name = 'rw_view1';
+
+-- Check behavior with DEFAULTs (bug #17633)
+
+INSERT INTO rw_view1 VALUES (9, DEFAULT), (10, DEFAULT);
+SELECT a, b FROM base_tbl_hist;
+
 DROP TABLE base_tbl CASCADE;
+DROP TABLE base_tbl_hist;
 
 -- view on top of view
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 07/20: Add missing nodes for function raw_expression_tree_walker().

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 5eb5061024b58acdcadcb0421e49213e187c7b23
Author: wenru yan <yw...@vmware.com>
AuthorDate: Tue Jun 13 09:23:31 2023 +0000

    Add missing nodes for function raw_expression_tree_walker().
---
 src/backend/nodes/nodeFuncs.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index e1538de8b9..a8e514d80a 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -4290,6 +4290,8 @@ raw_expression_tree_walker(Node *node,
 		case T_CommonTableExpr:
 			/* search_clause and cycle_clause are not interesting here */
 			return walker(((CommonTableExpr *) node)->ctequery, context);
+		case T_TableValueExpr:
+			return walker(((TableValueExpr *) node)->subquery, context);
 		default:
 			elog(ERROR, "unrecognized node type: %d",
 				 (int) nodeTag(node));


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 16/20: Remove MemoryContext s_tupSerMemCtxt (#15779)

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit d3c84a9a81ba7a706c0a7bc4688119899e1e3069
Author: Lei (Alexandra) Wang <al...@gmail.com>
AuthorDate: Wed Jun 21 13:29:22 2023 -0700

    Remove MemoryContext s_tupSerMemCtxt (#15779)
    
    Addressed the following GPDB_12_MERGE_FIXME in tupser.c:
    
    /*
     * GPDB_12_MERGE_FIXME: This function does not use this context. This context
     * is only used in SerializeRecordCacheIntoChunks(). We need to find a better
     * place for resetting it, or eliminating the needs for this context.
     */
    MemoryContextReset(s_tupSerMemCtxt);
    
    As the FIXME stated, post 12 merge, the allocation of memory under
    this memory context is more temporary and localized, so remove it
    altogether and instead just allocate the memory under the current
    memory context motion_layer_mctx and immediately call pfree() once
    the memory is no long needed.
    
    Reviewed-by: Hongxu Ma <in...@outlook.com>
---
 src/backend/cdb/motion/tupser.c | 35 ++---------------------------------
 1 file changed, 2 insertions(+), 33 deletions(-)

diff --git a/src/backend/cdb/motion/tupser.c b/src/backend/cdb/motion/tupser.c
index b0a7bb065f..f94d0c5e1a 100644
--- a/src/backend/cdb/motion/tupser.c
+++ b/src/backend/cdb/motion/tupser.c
@@ -40,13 +40,6 @@
  */
 #define RECORD_CACHE_MAGIC_TUPLEN	-1
 
-/* A MemoryContext used within the tuple serialize code, so that freeing of
- * space is SUPAFAST.  It is initialized in the first call to InitSerTupInfo()
- * since that must be called before any tuple serialization or deserialization
- * work can be done.
- */
-static MemoryContext s_tupSerMemCtxt = NULL;
-
 static void addByteStringToChunkList(TupleChunkList tcList, char *data, int datalen, TupleChunkListCache *cache);
 
 #define addCharToChunkList(tcList, x, c)							\
@@ -82,18 +75,6 @@ InitSerTupInfo(TupleDesc tupdesc, SerTupInfo *pSerInfo)
 	AssertArg(tupdesc != NULL);
 	AssertArg(pSerInfo != NULL);
 
-	if (s_tupSerMemCtxt == NULL)
-	{
-		/* Create tuple-serialization memory context. */
-		s_tupSerMemCtxt =
-			AllocSetContextCreate(TopMemoryContext,
-								  "TupSerMemCtxt",
-								  ALLOCSET_DEFAULT_INITSIZE,	/* always have some
-																 * memory */
-								  ALLOCSET_DEFAULT_INITSIZE,
-								  ALLOCSET_DEFAULT_MAXSIZE);
-	}
-
 	/* Set contents to all 0, just to make things clean and easy. */
 	memset(pSerInfo, 0, sizeof(SerTupInfo));
 
@@ -273,7 +254,6 @@ SerializeRecordCacheIntoChunks(SerTupInfo *pSerInfo,
 							   int32 sent_record_typmod)
 {
 	TupleChunkListItem tcItem = NULL;
-	MemoryContext oldCtxt;
 	List	   *typelist = NULL;
 	int			size = -1;
 	char	   *buf = NULL;
@@ -295,8 +275,6 @@ SerializeRecordCacheIntoChunks(SerTupInfo *pSerInfo,
 	tcItem->chunk_length = TUPLE_CHUNK_HEADER_SIZE;
 	appendChunkToTCList(tcList, tcItem);
 
-	AssertState(s_tupSerMemCtxt != NULL);
-
 	/*
 	 * To avoid inconsistency of record cache between sender and receiver in
 	 * the same motion, send the serialized record cache to receiver before
@@ -304,10 +282,9 @@ SerializeRecordCacheIntoChunks(SerTupInfo *pSerInfo,
 	 * the records to its own local cache and remapping the typmod of tuples
 	 * sent by sender.
 	 */
-	oldCtxt = MemoryContextSwitchTo(s_tupSerMemCtxt);
 	typelist = build_tuple_node_list(sent_record_typmod);
 	buf = serializeNode((Node *) typelist, &size, NULL);
-	MemoryContextSwitchTo(oldCtxt);
+	list_free_deep(typelist);
 
 	/*
 	 * we use magic tuplen to identify that this chunk (or list of chunks)
@@ -321,6 +298,7 @@ SerializeRecordCacheIntoChunks(SerTupInfo *pSerInfo,
 	addByteStringToChunkList(tcList, (char *) &size, sizeof(int),
 							 &pSerInfo->chunkCache);
 	addByteStringToChunkList(tcList, buf, size, &pSerInfo->chunkCache);
+	pfree(buf);
 
 	/*
 	 * if we have more than 1 chunk we have to set the chunk types on our
@@ -502,18 +480,9 @@ SerializeTuple(TupleTableSlot *slot, SerTupInfo *pSerInfo, struct directTranspor
 	tcItem->chunk_length = TUPLE_CHUNK_HEADER_SIZE;
 	appendChunkToTCList(tcList, tcItem);
 
-	AssertState(s_tupSerMemCtxt != NULL);
-
 	addByteStringToChunkList(tcList, (char *) &tupbodylen, sizeof(tupbodylen), &pSerInfo->chunkCache);
 	addByteStringToChunkList(tcList, tupbody, tupbodylen, &pSerInfo->chunkCache);
 
-	/*
-	 * GPDB_12_MERGE_FIXME: This function does not use this context. This context
-	 * is only used in SerializeRecordCacheIntoChunks(). We need to find a better
-	 * place for resetting it, or eliminating the needs for this context.
-	 */
-	MemoryContextReset(s_tupSerMemCtxt);
-
 	/*
 	 * if we have more than 1 chunk we have to set the chunk types on our
 	 * first chunk and last chunk


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org


(cloudberry) 20/20: resource_queue.sql: Add pg_locks sanity check

Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit 79b8ace965ad334f0317238ff25d1446389ed806
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Mon Jun 26 14:05:12 2023 -0700

    resource_queue.sql: Add pg_locks sanity check
    
    This simply ensures that pg_locks does not have any resource queue locks
    upon tramsaction completion, in the test file.
---
 src/test/isolation2/expected/resource_queue.out | 8 +++++++-
 src/test/isolation2/sql/resource_queue.sql      | 5 ++++-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/src/test/isolation2/expected/resource_queue.out b/src/test/isolation2/expected/resource_queue.out
index 51eaca4a7e..df438a16f2 100644
--- a/src/test/isolation2/expected/resource_queue.out
+++ b/src/test/isolation2/expected/resource_queue.out
@@ -39,7 +39,7 @@ SET
 PREPARE
 3&:EXECUTE fooplan;  <waiting ...>
 
--- Check pg_stat_activity and pg_locks again.
+-- Check pg_stat_activity and pg_locks.
 0:SELECT wait_event_type, wait_event from pg_stat_activity where query = 'EXECUTE fooplan;';
  wait_event_type | wait_event    
 -----------------+---------------
@@ -73,6 +73,12 @@ END
 3:END;
 END
 
+-- Sanity check: Ensure that all locks were released.
+0:SELECT granted, locktype, mode FROM pg_locks where locktype = 'resource queue' and pid != pg_backend_pid();
+ granted | locktype | mode 
+---------+----------+------
+(0 rows)
+
 -- Sanity check: Ensure that the resource queue is now empty.
 0: SELECT rsqcountlimit, rsqcountvalue from pg_resqueue_status WHERE rsqname = 'rq_concurrency_test';
  rsqcountlimit | rsqcountvalue 
diff --git a/src/test/isolation2/sql/resource_queue.sql b/src/test/isolation2/sql/resource_queue.sql
index a49f289cc8..61bdabec08 100644
--- a/src/test/isolation2/sql/resource_queue.sql
+++ b/src/test/isolation2/sql/resource_queue.sql
@@ -22,7 +22,7 @@
 3:PREPARE fooplan AS SELECT 1;
 3&:EXECUTE fooplan;
 
--- Check pg_stat_activity and pg_locks again.
+-- Check pg_stat_activity and pg_locks.
 0:SELECT wait_event_type, wait_event from pg_stat_activity where query = 'EXECUTE fooplan;';
 0:SELECT granted, locktype, mode FROM pg_locks where locktype = 'resource queue' and pid != pg_backend_pid();
 
@@ -34,6 +34,9 @@
 3<:
 3:END;
 
+-- Sanity check: Ensure that all locks were released.
+0:SELECT granted, locktype, mode FROM pg_locks where locktype = 'resource queue' and pid != pg_backend_pid();
+
 -- Sanity check: Ensure that the resource queue is now empty.
 0: SELECT rsqcountlimit, rsqcountvalue from pg_resqueue_status WHERE rsqname = 'rq_concurrency_test';
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org