You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Riza Suminto (Jira)" <ji...@apache.org> on 2021/02/26 01:03:00 UTC
[jira] [Commented] (IMPALA-10492) Lower default MAX_CNF_EXPRS query
option
[ https://issues.apache.org/jira/browse/IMPALA-10492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291322#comment-17291322 ]
Riza Suminto commented on IMPALA-10492:
---------------------------------------
I try to find out how many CNF rewrite happens in any TPC-DS/TPC-H queries. Here is my method:
1. Hack frontend to print the final value of [numCnfExprs_|https://github.com/apache/impala/blob/200d366/fe/src/main/java/org/apache/impala/rewrite/ConvertToCNFRule.java#L80] to query profile.
2. Run both tpcds and tpch workload using /bin/single_node_perf_run.py. Find out in the resulting json file whether any query execution has numCnfExprs_ > 0.
Among 99 TPC-DS and 22 TPC-H queries that is exercised, the following queries has CNF rewrite in their query compilation:
TPCDS-Q13, 168 rewrite
TPCDS-Q85, 100 rewrite
TPCDS-Q48, 34 rewrite
TPCH-Q19, 124 rewrite
TPCH-Q7, 3 rewrite
> Lower default MAX_CNF_EXPRS query option
> ----------------------------------------
>
> Key: IMPALA-10492
> URL: https://issues.apache.org/jira/browse/IMPALA-10492
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend, Frontend
> Affects Versions: Impala 4.0
> Reporter: Tamas Mate
> Priority: Minor
> Labels: ramp-up
>
> Currently MAX_CNF_EXPRS is set to unlimited by default, with a complex querie that container many predicates the CNF rewrite can lead to significant frontend memory usage and eventually OutOfMemory. A potential default value could be in the range of 1000, but it should work well with the current TCP-DS queries.
> cc.: [~amansinha], [~drorke]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org