You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/03/27 23:02:00 UTC
[jira] [Commented] (IMPALA-11696) Incorrect warnings when creating text/sequence table with row format delimiters
[ https://issues.apache.org/jira/browse/IMPALA-11696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705711#comment-17705711 ]
ASF subversion and git services commented on IMPALA-11696:
----------------------------------------------------------
Commit b793f4197630eab665cbecc9cf5920fc8212d9c1 in impala's branch refs/heads/branch-4.1.2 from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b793f4197 ]
IMPALA-11696: Fix incorrect warnings of ignoring delimiters on text/sequence tables
IMPALA-9822 adds a warning when the customized row format delimiters in
the CreateTable statement are ignored on non-TEXT and non-SEQUENCE
tables. However, the warning also shows up for TEXT/SEQUENCE tables. The
cause is an incorrect check in the table format that all formats match
the condition.
This patch fixes the condition and adds tests to verify that no warnings
show up in such cases. Currently the test methods (e.g. AnalyzesOk) only
check expected warning messages when provided. If the provided expected
message is null, they just skip checking the warnings. This patch adds
methods like AnalyzesOkWithoutWarnings to assure no warnings are
generated.
Tests
- Run FE tests
Change-Id: I0871b94dcd2290723699c21227a576e8a6a09b5a
Reviewed-on: http://gerrit.cloudera.org:8080/19186
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Incorrect warnings when creating text/sequence table with row format delimiters
> -------------------------------------------------------------------------------
>
> Key: IMPALA-11696
> URL: https://issues.apache.org/jira/browse/IMPALA-11696
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.1.0, Impala 4.2.0, Impala 4.1.1
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
> Fix For: Impala 4.3.0
>
>
> IMPALA-9822 adds a warning when the customized row format delimiters in the CreateTable statement is ignored. However, we see the same warning when creating TEXT or SEQUENCE_FILE tables.
> {code:sql}
> [localhost:21050] default> create external table my_csv (a int, b string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' stored as textfile;
> Query: create external table my_csv (a int, b string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' stored as textfile
> +-------------------------+
> | summary |
> +-------------------------+
> | Table has been created. |
> +-------------------------+
> WARNINGS: 'ROW FORMAT DELIMITED FIELDS TERMINATED BY ','' is ignored.
> Fetched 1 row(s) in 0.11s {code}
> The following code is buggy:
> {code:java}
> if (getRowFormat() != null) {
> String fieldDelimiter = getRowFormat().getFieldDelimiter();
> String lineDelimiter = getRowFormat().getLineDelimiter();
> String escapeChar = getRowFormat().getEscapeChar();
> if (getFileFormat() != THdfsFileFormat.TEXT
> || getFileFormat() != THdfsFileFormat.SEQUENCE_FILE) { // Should use && instead
> if (fieldDelimiter != null) {
> analyzer.addWarning("'ROW FORMAT DELIMITED FIELDS TERMINATED BY '"
> + fieldDelimiter + "'' is ignored.");
> }
> if (lineDelimiter != null) {
> analyzer.addWarning("'ROW FORMAT DELIMITED LINES TERMINATED BY '"
> + lineDelimiter + "'' is ignored.");
> }
> if (escapeChar != null) {
> analyzer.addWarning(
> "'ROW FORMAT DELIMITED ESCAPED BY '" + escapeChar + "'' is ignored.");
> }
> }
> }
> {code}
> https://github.com/apache/impala/blob/8271bdd587d241cd5a61ccae7422bbb5fafcfaf7/fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java#L276
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org