You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/05/19 20:34:00 UTC
[jira] [Commented] (IMPALA-2651) codegen overhead can be extremely
large
[ https://issues.apache.org/jira/browse/IMPALA-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111510#comment-17111510 ]
Tim Armstrong commented on IMPALA-2651:
---------------------------------------
This has improved a lot over time, I think to a large degree because of reduced inlining, and maybe the LLVM upgrades help. The overhead can still be high and it is not cancellable.
> codegen overhead can be extremely large
> ---------------------------------------
>
> Key: IMPALA-2651
> URL: https://issues.apache.org/jira/browse/IMPALA-2651
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.2, Impala 2.3.0
> Reporter: Silvius Rus
> Priority: Minor
> Labels: codegen, performance
>
> We received reports of excessive codegen compilation/optimization times for very large expressions generated by visualization tools.
> We should:
> # Expose codegen optimization levels as query options. Currently there is only an all or nothing codegen query option. It's likely that overly complex expressions such as hundreds of cascading conditions take very long and benefit very little from an O2 optimization level, but they could still run significantly faster even at O0 or O1 versus interpreted.
> # Consider dropping to O1 (or turn off riskier passes individually) automatically for very large expressions.
> # Consider parameterizing the compilation duration time limits and set a reasonable default, say 10 seconds. Either disable codegen or reduce it to, say, O0 if compilation takes longer than the preset limit.
> *Workaround*
> In some cases disabling codegen can help.
> {code}
> SET disable_codegen=true;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org