[jira] [Resolved] (IMPALA-10416) Testfile can't deal with non-ascii results

["Quanlong Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Quanlong Huang resolved IMPALA-10416.
-------------------------------------
    Fix Version/s: Impala 4.0
       Resolution: Fixed

> Testfile can't deal with non-ascii results
> ------------------------------------------
>
>                 Key: IMPALA-10416
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10416
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Infrastructure
>            Reporter: Quanlong Huang
>            Assignee: Quanlong Huang
>            Priority: Blocker
>             Fix For: Impala 4.0
>
>
> In the testfile, we can use non-ascii characters in the query. But when the result contains non-ascii characters, the test framework fails to deal with them.
> For instance, I'm currently on master branch (commit 5baadd1). Add a simple test query:
> {code:java}
> diff --git a/testdata/workloads/functional-query/queries/QueryTest/special-strings.test b/testdata/workloads/functional-query/queries/QueryTest/special-strings.test
> index 99a694c..9dfbc97 100644
> --- a/testdata/workloads/functional-query/queries/QueryTest/special-strings.test
> +++ b/testdata/workloads/functional-query/queries/QueryTest/special-strings.test
> @@ -24,3 +24,10 @@ select "'"
>  ---- TYPES
>  STRING
>  ====
> +---- QUERY
> +select "你好"
> +---- RESULTS
> +'你好'
> +---- TYPES
> +STRING
> +==== {code}
> Run the test
> {code:java}
> impala-py.test tests/query_test/test_exprs.py::TestExprs::test_special_strings {code}
> The failure occurs:
> {code:java}
> TestExprs.test_special_strings[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none | enable_expr_rewrites: 1] 
> tests/query_test/test_exprs.py:71: in test_special_strings
>     self.run_test_case('QueryTest/special-strings', vector)
> tests/common/impala_test_suite.py:693: in run_test_case
>     self.__verify_results_and_errors(vector, test_section, result, use_db)
> tests/common/impala_test_suite.py:529: in __verify_results_and_errors
>     replace_filenames_with_placeholder)
> tests/common/test_result_verifier.py:452: in verify_raw_results
>     actual = QueryTestResult(parse_result_rows(exec_result), actual_types,
> tests/common/test_result_verifier.py:493: in parse_result_rows
>     col = cols[i].encode('unicode_escape')
> E   UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0: ordinal not in range(128) {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org