[jira] [Commented] (IMPALA-4865) Limit size of literal values from constant folding

["Tim Armstrong (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677447#comment-16677447 ] 

Tim Armstrong commented on IMPALA-4865:
---------------------------------------

I've also seen this lead to JVM OOMs from all of the huge string literals generated.

I think the right fix here is to return an error from EvalExprsWithoutRow() when the output string is above some threshold where propagating the folded value is likely to cause more problems than it solves, e.g. 1MB.

Then the callsites in the planner need to gracefully handle these failures and fall back to the non-folding path.

> Limit size of literal values from constant folding
> --------------------------------------------------
>
>                 Key: IMPALA-4865
>                 URL: https://issues.apache.org/jira/browse/IMPALA-4865
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 2.8.0
>            Reporter: Henry Robinson
>            Priority: Major
>              Labels: newbie
>
> Consider (from {{test_insert.py}}) the following query:
> {{create table foo as select repeat('AZ', 128 * 1024 * 1024) as s}}
> {{repeat()}} gets constant-folded into a 256MB expression payload, which is then transmitted to every fragment instance that evaluates the expression. This can lead to timeouts. 
> The workaround is {{ENABLE_EXPR_REWRITES=0}}.
> It's not obvious what the answer is - should the FE be aware of the serialized size of an expression when deciding whether to constant fold it?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org