You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Csaba Ringhofer (Jira)" <ji...@apache.org> on 2024/03/25 15:56:00 UTC
[jira] [Commented] (IMPALA-10349) Revisit constant folding on non-ASCII strings
[ https://issues.apache.org/jira/browse/IMPALA-10349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830545#comment-17830545 ]
Csaba Ringhofer commented on IMPALA-10349:
------------------------------------------
Also bumped into this related to pushing down to Kudu:
{code}
explain select count(*) from functional_kudu.alltypes where string_col = "á";
-- kudu predicates: string_col = 'á'
explain select count(*) from functional_kudu.alltypes where string_col = concat("a", "")
-- kudu predicates: string_col = 'a'
explain select count(*) from functional_kudu.alltypes where string_col = concat("á", "")
-- not pushed down to Kudu:
-- predicates: string_col = concat('á', '')
{code}
>I think we should allow folding non-ASCII strings if they are legal UTF-8 strings.
[~stigahuang] Do you why is it not possible to fold strings that are not valid UTF-8?
Currently BINARY columns also use StringLiterals, a.g cast("a" as binary) will be folded to a StringLiteral. It would be useful to also fold expressions like cast(unhex("aa") as binary) to be able to push them down to Kudu.
> Revisit constant folding on non-ASCII strings
> ---------------------------------------------
>
> Key: IMPALA-10349
> URL: https://issues.apache.org/jira/browse/IMPALA-10349
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Quanlong Huang
> Priority: Critical
>
> Constant folding may produce non-ASCII strings. In such cases, we currently abandon folding the constant. See commit message of IMPALA-1788 or codes here: [https://github.com/apache/impala/blob/9672d945963e1ca3c8699340f92d7d6ce1d91c9f/fe/src/main/java/org/apache/impala/analysis/LiteralExpr.java#L274-L282]
> I think we should allow folding non-ASCII strings if they are legal UTF-8 strings.
> Example of constant folding work:
> {code:java}
> Query: explain select * from functional.alltypes where string_col = substr('123', 1, 1)
> +-------------------------------------------------------------+
> | Explain String |
> +-------------------------------------------------------------+
> | Max Per-Host Resource Reservation: Memory=32.00KB Threads=3 |
> | Per-Host Resource Estimates: Memory=160MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 01:EXCHANGE [UNPARTITIONED] |
> | | |
> | 00:SCAN HDFS [functional.alltypes] |
> | HDFS partitions=24/24 files=24 size=478.45KB |
> | predicates: string_col = '1' |
> | row-size=89B cardinality=730 |
> +-------------------------------------------------------------+
> {code}
> Example of constant folding doesn't work:
> {code:java}
> Query: explain select * from functional.alltypes where string_col = substr('引擎', 1, 3)
> +-------------------------------------------------------------+
> | Explain String |
> +-------------------------------------------------------------+
> | Max Per-Host Resource Reservation: Memory=32.00KB Threads=3 |
> | Per-Host Resource Estimates: Memory=160MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 01:EXCHANGE [UNPARTITIONED] |
> | | |
> | 00:SCAN HDFS [functional.alltypes] |
> | HDFS partitions=24/24 files=24 size=478.45KB |
> | predicates: string_col = substr('引擎', 1, 3) |
> | row-size=89B cardinality=730 |
> +-------------------------------------------------------------+
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org