You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "James Turton (Jira)" <ji...@apache.org> on 2021/12/01 13:06:00 UTC
[jira] [Updated] (DRILL-8063) OOM planning a certain query
[ https://issues.apache.org/jira/browse/DRILL-8063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Turton updated DRILL-8063:
--------------------------------
Description:
This looks like an infinite planning bug in Calcite. To reproduce, copy the two referenced TPCH Parquet files from contrib/data/tpch-sample-data/parquet/ to dfs.tmp then run the following. Uncommenting the `magic_fix` column is just one of the changes that can be made to make the query planning succeed.
{{select
p_brand,
-- 'foobar' as magic_fix,
case
when f1 then v1
else null
end as `m_1`,
case
when f1 then v2
else null
end as `m_2`
from
(select
part.`p_brand`,
sum(t.l_extendedprice) as v1,
avg(t.l_extendedprice) as v2,
true as f1
from
dfs.tmp.`lineitem.parquet` `t`
inner join dfs.tmp.`part.parquet` part on `t`.`l_partkey` = part.`p_partkey`
group by part.`p_brand`) as `t2`
}}
Stack trace snippet:
{{
2021-12-01 13:12:15,172 [1e58a77f-0a5d-22b5-47f6-4c51bc31dbe6:foreman] ERROR o.a.drill.common.CatastrophicFailure - Cat
astrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in Foreman.
java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172)
at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:174)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.apache.calcite.rex.RexCall.appendOperands(RexCall.java:109)
at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:166)
at org.apache.calcite.rex.RexCall.toString(RexCall.java:183)
at java.base/java.lang.String.valueOf(String.java:2951)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.apache.calcite.rex.RexCall.appendOperands(RexCall.java:109)
at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:166)
at org.apache.calcite.rex.RexCall.toString(RexCall.java:183)
...
}}
was:
This looks like an infinite planning bug in Calcite. To reproduce, copy the two referenced TPCH Parquet files from contrib/data/tpch-sample-data/parquet/ to dfs.tmp then run the following. Uncommenting the `magic_fix` column is just one of the changes that can be made to make the query planning succeed.
```
select
p_brand,
-- 'foobar' as magic_fix,
case
when f1 then v1
else null
end as `m_1`,
case
when f1 then v2
else null
end as `m_2`
from
(select
part.`p_brand`,
sum(t.l_extendedprice) as v1,
avg(t.l_extendedprice) as v2,
true as f1
from
dfs.tmp.`lineitem.parquet` `t`
inner join dfs.tmp.`part.parquet` part on `t`.`l_partkey` = part.`p_partkey`
group by part.`p_brand`) as `t2`
```
Stack trace snippet:
```
2021-12-01 13:12:15,172 [1e58a77f-0a5d-22b5-47f6-4c51bc31dbe6:foreman] ERROR o.a.drill.common.CatastrophicFailure - Cat
astrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in Foreman.
java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172)
at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:174)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.apache.calcite.rex.RexCall.appendOperands(RexCall.java:109)
at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:166)
at org.apache.calcite.rex.RexCall.toString(RexCall.java:183)
at java.base/java.lang.String.valueOf(String.java:2951)
at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
at org.apache.calcite.rex.RexCall.appendOperands(RexCall.java:109)
at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:166)
at org.apache.calcite.rex.RexCall.toString(RexCall.java:183)
...
```
> OOM planning a certain query
> ----------------------------
>
> Key: DRILL-8063
> URL: https://issues.apache.org/jira/browse/DRILL-8063
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
> Affects Versions: 1.19.0
> Reporter: James Turton
> Assignee: Vitalii Diravka
> Priority: Critical
> Fix For: 1.20.0
>
>
> This looks like an infinite planning bug in Calcite. To reproduce, copy the two referenced TPCH Parquet files from contrib/data/tpch-sample-data/parquet/ to dfs.tmp then run the following. Uncommenting the `magic_fix` column is just one of the changes that can be made to make the query planning succeed.
> {{select
> p_brand,
> -- 'foobar' as magic_fix,
> case
> when f1 then v1
> else null
> end as `m_1`,
> case
> when f1 then v2
> else null
> end as `m_2`
> from
> (select
> part.`p_brand`,
> sum(t.l_extendedprice) as v1,
> avg(t.l_extendedprice) as v2,
> true as f1
> from
> dfs.tmp.`lineitem.parquet` `t`
> inner join dfs.tmp.`part.parquet` part on `t`.`l_partkey` = part.`p_partkey`
> group by part.`p_brand`) as `t2`
> }}
> Stack trace snippet:
> {{
> 2021-12-01 13:12:15,172 [1e58a77f-0a5d-22b5-47f6-4c51bc31dbe6:foreman] ERROR o.a.drill.common.CatastrophicFailure - Cat
> astrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in Foreman.
> java.lang.OutOfMemoryError: Java heap space
> at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
> at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172)
> at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538)
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:174)
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
> at org.apache.calcite.rex.RexCall.appendOperands(RexCall.java:109)
> at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:166)
> at org.apache.calcite.rex.RexCall.toString(RexCall.java:183)
> at java.base/java.lang.String.valueOf(String.java:2951)
> at java.base/java.lang.StringBuilder.append(StringBuilder.java:168)
> at org.apache.calcite.rex.RexCall.appendOperands(RexCall.java:109)
> at org.apache.calcite.rex.RexCall.computeDigest(RexCall.java:166)
> at org.apache.calcite.rex.RexCall.toString(RexCall.java:183)
> ...
> }}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)