You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by ma...@apache.org on 2025/01/02 06:17:31 UTC
(cloudberry) branch main updated (a12d2d4b15 -> 1fb1701ab7)
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
from a12d2d4b15 Fix test cases of Cherry-pick.
new 99bd992a1c FIX BUG: COPY FORM doesn't throw out ERROR: extra data after last expected column (7X) (#15612)
new aec3384edd Fix related code in copyfrom.c
new 1fb1701ab7 Move temp files to results dir to avoid untracked files. (#15832)
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
src/backend/commands/copyfrom.c | 11 +++--
src/test/regress/data/copy.data | 1 +
src/test/regress/input/copy.source | 45 ++++++++++++++++++
src/test/regress/output/copy.source | 91 +++++++++++++++++++++++++++++++++++++
4 files changed, 145 insertions(+), 3 deletions(-)
create mode 100644 src/test/regress/data/copy.data
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 03/03: Move temp files to results dir to avoid untracked files. (#15832)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 1fb1701ab77100fc86ad5ae6c8dd6072b6ab67ac
Author: Zhang Mingli <av...@gmail.com>
AuthorDate: Fri Jun 30 20:31:39 2023 +0800
Move temp files to results dir to avoid untracked files. (#15832)
Commit 5cae067e41e introduces test file under data dir for copy case. That will generate untracked file after regression tests. Untracked files:
```
(use "git add <file>..." to include in what will be committed)
src/test/regress/data/copy_no_cols.data
nothing added to commit but untracked files present (use "git add" to track)
```
Move temp date files to reuslts dir to avoid that.
---
src/test/regress/input/copy.source | 4 ++--
src/test/regress/output/copy.source | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/test/regress/input/copy.source b/src/test/regress/input/copy.source
index c450395ebc..36f70b6412 100644
--- a/src/test/regress/input/copy.source
+++ b/src/test/regress/input/copy.source
@@ -345,8 +345,8 @@ DROP TABLE table_3_cols_dist_c3;
CREATE TABLE table_no_cols();
INSERT INTO table_no_cols DEFAULT VALUES;
SELECT count(*) FROM table_no_cols;
-COPY table_no_cols TO '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
-COPY table_no_cols FROM '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
+COPY table_no_cols TO '@abs_srcdir@/results/copy_no_cols.data' DELIMITER ',';
+COPY table_no_cols FROM '@abs_srcdir@/results/copy_no_cols.data' DELIMITER ',';
SELECT count(*) FROM table_no_cols;
COPY table_no_cols FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
DROP TABLE table_no_cols;
diff --git a/src/test/regress/output/copy.source b/src/test/regress/output/copy.source
index c5dcfeba3d..5e037d77ec 100644
--- a/src/test/regress/output/copy.source
+++ b/src/test/regress/output/copy.source
@@ -344,8 +344,8 @@ SELECT count(*) FROM table_no_cols;
1
(1 row)
-COPY table_no_cols TO '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
-COPY table_no_cols FROM '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
+COPY table_no_cols TO '@abs_srcdir@/results/copy_no_cols.data' DELIMITER ',';
+COPY table_no_cols FROM '@abs_srcdir@/results/copy_no_cols.data' DELIMITER ',';
SELECT count(*) FROM table_no_cols;
count
-------
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 01/03: FIX BUG: COPY FORM doesn't throw out ERROR: extra data after last expected column (7X) (#15612)
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit 99bd992a1c9cc87d0f878743572aa6860c52792a
Author: Yongtao Huang <99...@users.noreply.github.com>
AuthorDate: Thu Jun 8 15:42:04 2023 +0800
FIX BUG: COPY FORM doesn't throw out ERROR: extra data after last expected column (7X) (#15612)
This commit can fix bug reported in [issue 15625]
(https://github.com/greenplum-db/gpdb/issues/15625)
**Solution:**
In pg12.12, there is no function `InitCopyFromDispatchSplit()`,
so attr_count must be > 0. In GP7, we may get `[NULL]`
in QE when all fields are processed in the QD.
So the condition checking should change
from `attr_count > 0` to `attr_count >= 0`.
But before it, `attr_count = list_length(attnumlist);`,
we can ensure that `attr_count >= 0` consistently,
so delete this bug-related condition `attr_count > 0`.
Signed-off-by: Yongtao Huang <yo...@vmware.com>
Co-authored-by: Xiaoxiao He <hx...@vmware.com>
---
src/test/regress/data/copy.data | 1 +
src/test/regress/input/copy.source | 45 ++++++++++++++++++
src/test/regress/output/copy.source | 91 +++++++++++++++++++++++++++++++++++++
3 files changed, 137 insertions(+)
diff --git a/src/test/regress/data/copy.data b/src/test/regress/data/copy.data
new file mode 100644
index 0000000000..4a16c4207a
--- /dev/null
+++ b/src/test/regress/data/copy.data
@@ -0,0 +1 @@
+1,text1
diff --git a/src/test/regress/input/copy.source b/src/test/regress/input/copy.source
index 159cc1156d..c450395ebc 100644
--- a/src/test/regress/input/copy.source
+++ b/src/test/regress/input/copy.source
@@ -305,3 +305,48 @@ copy tab_progress_reporting from '@abs_srcdir@/data/emp.data'
drop trigger check_after_tab_progress_reporting on tab_progress_reporting;
drop function notice_after_tab_progress_reporting();
drop table tab_progress_reporting;
+-- check COPY behavior
+-- there are 2 fields in copy.data
+-- table_1_col_dist_c1: 1 col in table < 2 fields in file
+-- table_2_cols_dist_c1 and
+-- table_2_cols_dist_c2: 2 cols in table = 2 fields in file
+-- table_3_cols_dist_c1 and
+-- table_3_cols_dist_c2 and
+-- table_3_cols_dist_c3: 3 cols in table > 2 fields in file
+CREATE TABLE table_1_col_dist_c1(c1 text) distributed by (c1);
+CREATE TABLE table_2_cols_dist_c1(c1 text, c2 text) distributed by (c1);
+CREATE TABLE table_2_cols_dist_c2(c1 text, c2 text) distributed by (c2);
+CREATE TABLE table_3_cols_dist_c1(c1 text, c2 text, c3 text) distributed by (c1);
+CREATE TABLE table_3_cols_dist_c2(c1 text, c2 text, c3 text) distributed by (c2);
+CREATE TABLE table_3_cols_dist_c3(c1 text, c2 text, c3 text) distributed by (c3);
+
+COPY table_1_col_dist_c1 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+COPY table_2_cols_dist_c1 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+COPY table_2_cols_dist_c2 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+COPY table_3_cols_dist_c1 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+COPY table_3_cols_dist_c2 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+COPY table_3_cols_dist_c3 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+
+SELECT count(*) FROM table_1_col_dist_c1;
+SELECT count(*) FROM table_2_cols_dist_c1;
+SELECT count(*) FROM table_2_cols_dist_c2;
+SELECT count(*) FROM table_3_cols_dist_c1;
+SELECT count(*) FROM table_3_cols_dist_c2;
+SELECT count(*) FROM table_3_cols_dist_c3;
+
+DROP TABLE table_1_col_dist_c1;
+DROP TABLE table_2_cols_dist_c1;
+DROP TABLE table_2_cols_dist_c2;
+DROP TABLE table_3_cols_dist_c1;
+DROP TABLE table_3_cols_dist_c2;
+DROP TABLE table_3_cols_dist_c3;
+
+-- special case, tables without columns
+CREATE TABLE table_no_cols();
+INSERT INTO table_no_cols DEFAULT VALUES;
+SELECT count(*) FROM table_no_cols;
+COPY table_no_cols TO '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
+COPY table_no_cols FROM '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
+SELECT count(*) FROM table_no_cols;
+COPY table_no_cols FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+DROP TABLE table_no_cols;
diff --git a/src/test/regress/output/copy.source b/src/test/regress/output/copy.source
index 2eae9eca82..c5dcfeba3d 100644
--- a/src/test/regress/output/copy.source
+++ b/src/test/regress/output/copy.source
@@ -265,3 +265,94 @@ drop trigger check_after_tab_progress_reporting on tab_progress_reporting;
ERROR: trigger "check_after_tab_progress_reporting" for table "tab_progress_reporting" does not exist
drop function notice_after_tab_progress_reporting();
drop table tab_progress_reporting;
+-- check COPY behavior
+-- there are 2 fields in copy.data
+-- table_1_col_dist_c1: 1 col in table < 2 fields in file
+-- table_2_cols_dist_c1 and
+-- table_2_cols_dist_c2: 2 cols in table = 2 fields in file
+-- table_3_cols_dist_c1 and
+-- table_3_cols_dist_c2 and
+-- table_3_cols_dist_c3: 3 cols in table > 2 fields in file
+CREATE TABLE table_1_col_dist_c1(c1 text) distributed by (c1);
+CREATE TABLE table_2_cols_dist_c1(c1 text, c2 text) distributed by (c1);
+CREATE TABLE table_2_cols_dist_c2(c1 text, c2 text) distributed by (c2);
+CREATE TABLE table_3_cols_dist_c1(c1 text, c2 text, c3 text) distributed by (c1);
+CREATE TABLE table_3_cols_dist_c2(c1 text, c2 text, c3 text) distributed by (c2);
+CREATE TABLE table_3_cols_dist_c3(c1 text, c2 text, c3 text) distributed by (c3);
+COPY table_1_col_dist_c1 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+ERROR: extra data after last expected column
+CONTEXT: COPY table_1_col_dist_c1, line 1: "1,text1"
+COPY table_2_cols_dist_c1 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+COPY table_2_cols_dist_c2 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+COPY table_3_cols_dist_c1 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+ERROR: missing data for column "c3"
+CONTEXT: COPY table_3_cols_dist_c1, line 1: "1,text1"
+COPY table_3_cols_dist_c2 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+ERROR: missing data for column "c3"
+CONTEXT: COPY table_3_cols_dist_c2, line 1: "1,text1"
+COPY table_3_cols_dist_c3 FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+ERROR: missing data for column "c3"
+CONTEXT: COPY table_3_cols_dist_c3, line 1: "1,text1"
+SELECT count(*) FROM table_1_col_dist_c1;
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*) FROM table_2_cols_dist_c1;
+ count
+-------
+ 1
+(1 row)
+
+SELECT count(*) FROM table_2_cols_dist_c2;
+ count
+-------
+ 1
+(1 row)
+
+SELECT count(*) FROM table_3_cols_dist_c1;
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*) FROM table_3_cols_dist_c2;
+ count
+-------
+ 0
+(1 row)
+
+SELECT count(*) FROM table_3_cols_dist_c3;
+ count
+-------
+ 0
+(1 row)
+
+DROP TABLE table_1_col_dist_c1;
+DROP TABLE table_2_cols_dist_c1;
+DROP TABLE table_2_cols_dist_c2;
+DROP TABLE table_3_cols_dist_c1;
+DROP TABLE table_3_cols_dist_c2;
+DROP TABLE table_3_cols_dist_c3;
+-- special case, tables without columns
+CREATE TABLE table_no_cols();
+INSERT INTO table_no_cols DEFAULT VALUES;
+SELECT count(*) FROM table_no_cols;
+ count
+-------
+ 1
+(1 row)
+
+COPY table_no_cols TO '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
+COPY table_no_cols FROM '@abs_srcdir@/data/copy_no_cols.data' DELIMITER ',';
+SELECT count(*) FROM table_no_cols;
+ count
+-------
+ 2
+(1 row)
+
+COPY table_no_cols FROM '@abs_srcdir@/data/copy.data' DELIMITER ',';
+ERROR: extra data after last expected column
+CONTEXT: COPY table_no_cols, line 1: "1,text1"
+DROP TABLE table_no_cols;
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
(cloudberry) 02/03: Fix related code in copyfrom.c
Posted by ma...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git
commit aec3384edd4fd96cffbe2d5ddeee4962d1382bc9
Author: Yongtao Huang <yo...@gmail.com>
AuthorDate: Tue Dec 17 23:26:58 2024 +0800
Fix related code in copyfrom.c
Part of copy.c is split into copyfrom.c and copyto.c in PG-14
Signed-off-by: Yongtao Huang <yo...@gmail.com>
---
src/backend/commands/copyfrom.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 905fd7e68e..d7aaf8b97a 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -3327,9 +3327,14 @@ NextCopyFromX(CopyFromState cstate, ExprContext *econtext,
fldct = 0;
field_strings = cstate->raw_fields;
}
-
- /* check for overflowing fields */
- if (attr_count > 0 && fldct > attr_count)
+ /*
+ * Check for overflowing fields.
+ * GPDB: Change below condition compared to upstream to
+ * greater than or equal to 0 as in QE,
+ * attr_count may be equal to 0,
+ * when all fields are processed in the QD.
+ */
+ if (fldct > attr_count)
ereport(ERROR,
(errcode(ERRCODE_BAD_COPY_FILE_FORMAT),
errmsg("extra data after last expected column")));
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org