You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/03/27 23:02:00 UTC
[jira] [Commented] (IMPALA-9822) Impala does not notify user that row format delimited fields is only logical when using STORED AS TEXTFILE
[ https://issues.apache.org/jira/browse/IMPALA-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705712#comment-17705712 ]
ASF subversion and git services commented on IMPALA-9822:
---------------------------------------------------------
Commit b793f4197630eab665cbecc9cf5920fc8212d9c1 in impala's branch refs/heads/branch-4.1.2 from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b793f4197 ]
IMPALA-11696: Fix incorrect warnings of ignoring delimiters on text/sequence tables
IMPALA-9822 adds a warning when the customized row format delimiters in
the CreateTable statement are ignored on non-TEXT and non-SEQUENCE
tables. However, the warning also shows up for TEXT/SEQUENCE tables. The
cause is an incorrect check in the table format that all formats match
the condition.
This patch fixes the condition and adds tests to verify that no warnings
show up in such cases. Currently the test methods (e.g. AnalyzesOk) only
check expected warning messages when provided. If the provided expected
message is null, they just skip checking the warnings. This patch adds
methods like AnalyzesOkWithoutWarnings to assure no warnings are
generated.
Tests
- Run FE tests
Change-Id: I0871b94dcd2290723699c21227a576e8a6a09b5a
Reviewed-on: http://gerrit.cloudera.org:8080/19186
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Impala does not notify user that row format delimited fields is only logical when using STORED AS TEXTFILE
> ----------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-9822
> URL: https://issues.apache.org/jira/browse/IMPALA-9822
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Alexandra Dunai
> Assignee: Shikha Asrani
> Priority: Minor
> Labels: newbie, ramp-up, usability
> Fix For: Impala 4.1.0
>
>
> When creating a table with added "ROW FORMAT DELIMITED FIELDS", Impala does not alert the user that this is only logical when using STORED AS TEXTFILE.
> You only discover that you made a mistake after trying to run a select from the table.
> Table creation:
> {code:bash}
> [adunai-1.adunai.root.hwx.site:21000] default> CREATE EXTERNAL TABLE sales_fact_1997(product_id INT,time_id INT,customer_id INT,promotion_id INT,store_id INT,store_sales DECIMAL(10,4),store_cost DECIMAL(10,4),unit_sales DECIMAL(10,4))
> > row format delimited fields terminated by '\011' STORED AS PARQUET
> > location '/user/impala/mondrian/sales_fact_1997';
> Query: CREATE EXTERNAL TABLE sales_fact_1997(product_id INT,time_id INT,customer_id INT,promotion_id INT,store_id INT,store_sales DECIMAL(10,4),store_cost DECIMAL(10,4),unit_sales DECIMAL(10,4))row format delimited fields terminated by '\011' STORED AS PARQUET location '/user/impala/mondrian/sales_fact_1997'
>
> +-------------------------+
> | summary |
> +-------------------------+
> | Table has been created. |
> +-------------------------+
> Fetched 1 row(s) in 0.10s
> {code}
>
> Select:
> {code:bash}
> [adunai-1.adunai.root.hwx.site:21000] mondrian> select count(*) from agg_c_10_sales_fact_1997;
> Query: select count(*) from agg_c_10_sales_fact_1997
> Query submitted at: 2020-06-03 11:55:06 (Coordinator: http://adunai-1.adunai.root.hwx.site:25000)
> Query progress can be monitored at: http://adunai-1.adunai.root.hwx.site:25000/query_plan?query_id=d547fafd0162da4e:872a95c100000000
> ERROR: File 'hdfs://adunai-2.adunai.root.hwx.site:8020/user/impala/mondrian/agg_c_10_sales_fact_1997/agg_c_10_sales_fact_1997.tsv' has an invalid Parquet version number: 717. Please check that it is a valid Parquet file. This error can also occur due to stale metadata. If you believe this is a valid Parquet file, try running "refresh mondrian.agg_c_10_sales_fact_1997".{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org