You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2019/06/15 15:23:17 UTC
[GitHub] [incubator-druid] gianm opened a new issue #7904: SQL planning slow
with huge IN filters
gianm opened a new issue #7904: SQL planning slow with huge IN filters
URL: https://github.com/apache/incubator-druid/issues/7904
### Affected Version
0.14.2
### Description
When running a SQL query with an IN clause with ~14k elements, planning on the broker took quite a long time (over a minute). A lot of time seems to be spent in areas of Calcite code that look like the following (I saw this pattern repeat over lots of thread dumps). Maybe some algorithmic blowup when there are a lot of OR conditions, which is what a big IN would get translated to.
```
"sql[af077a4e-23e8-4440-9f12-42919d7b8b5b]" #113 daemon prio=5 os_prio=0 tid=0x00007f3142179000 nid=0x36c runnable [0x00007f3096fb3000]
java.lang.Thread.State: RUNNABLE
at org.apache.calcite.rex.RexUtil.decompose(RexUtil.java:355)
at org.apache.calcite.rex.RexUtil.gatherConstraints(RexUtil.java:366)
at org.apache.calcite.rex.RexUtil.predicateConstants(RexUtil.java:323)
at org.apache.calcite.plan.RelOptPredicateList.of(RelOptPredicateList.java:145)
at org.apache.calcite.plan.RelOptPredicateList.union(RelOptPredicateList.java:158)
at org.apache.calcite.rex.RexSimplify.simplifyOrTerms(RexSimplify.java:350)
at org.apache.calcite.rex.RexSimplify.simplifyOr(RexSimplify.java:1058)
at org.apache.calcite.rex.RexSimplify.simplify_(RexSimplify.java:183)
at org.apache.calcite.rex.RexSimplify.lambda$simplify$0(RexSimplify.java:175)
at org.apache.calcite.rex.RexSimplify$$Lambda$258/1039224919.apply(Unknown Source)
at org.apache.calcite.rex.RexSimplify.verify(RexSimplify.java:1097)
at org.apache.calcite.rex.RexSimplify.simplify(RexSimplify.java:175)
at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:2607)
at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:2567)
at org.apache.calcite.rex.RexCall.accept(RexCall.java:107)
at org.apache.calcite.rex.RexShuttle.visitList(RexShuttle.java:151)
at org.apache.calcite.rex.RexShuttle.visitCall(RexShuttle.java:100)
at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:2604)
at org.apache.calcite.rex.RexUtil$ExprSimplifier.visitCall(RexUtil.java:2567)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org