You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Kunal Khatua (JIRA)" <ji...@apache.org> on 2017/08/02 00:48:00 UTC
[jira] [Updated] (DRILL-3794) Fragments outlive query execution in
certain out of memory cases
[ https://issues.apache.org/jira/browse/DRILL-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kunal Khatua updated DRILL-3794:
--------------------------------
Reviewer: Khurram Faraaz
[~khfaraaz] Can you verify if this issue is resolved with DRILL-5599 (Drill 1.11.0)?
> Fragments outlive query execution in certain out of memory cases
> ----------------------------------------------------------------
>
> Key: DRILL-3794
> URL: https://issues.apache.org/jira/browse/DRILL-3794
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Reporter: Rahul Challapalli
> Fix For: Future
>
> Attachments: error.log, jstack.log, lineitem.parquet
>
>
> git.commit.id.abbrev=240a455
> I created the data set by duplicating the attached lineitem.parquet (tpch0.01) file 5000 times into the lineitem folder.
> The below query fails with an out of memory error
> {code}
> 0: jdbc:drill:zk=10.10.100.190:5181> select
> . . . . . . . . . . . . . . . . . .> n.n_name,
> . . . . . . . . . . . . . . . . . .> sum(l.l_extendedprice * (1 - l.l_discount)) as revenue
> . . . . . . . . . . . . . . . . . .> from
> . . . . . . . . . . . . . . . . . .> customer c,
> . . . . . . . . . . . . . . . . . .> orders o,
> . . . . . . . . . . . . . . . . . .> lineitem l,
> . . . . . . . . . . . . . . . . . .> supplier s,
> . . . . . . . . . . . . . . . . . .> nation n,
> . . . . . . . . . . . . . . . . . .> region r
> . . . . . . . . . . . . . . . . . .> where
> . . . . . . . . . . . . . . . . . .> c.c_custkey = o.o_custkey
> . . . . . . . . . . . . . . . . . .> and l.l_orderkey = o.o_orderkey
> . . . . . . . . . . . . . . . . . .> and l.l_suppkey = s.s_suppkey
> . . . . . . . . . . . . . . . . . .> and c.c_nationkey = s.s_nationkey
> . . . . . . . . . . . . . . . . . .> and s.s_nationkey = n.n_nationkey
> . . . . . . . . . . . . . . . . . .> and n.n_regionkey = r.r_regionkey
> . . . . . . . . . . . . . . . . . .> and r.r_name = 'EUROPE'
> . . . . . . . . . . . . . . . . . .> and o.o_orderdate >= date '1997-01-01'
> . . . . . . . . . . . . . . . . . .> and o.o_orderdate < date '1997-01-01' + interval '1' year
> . . . . . . . . . . . . . . . . . .> group by
> . . . . . . . . . . . . . . . . . .> n.n_name
> . . . . . . . . . . . . . . . . . .> order by
> . . . . . . . . . . . . . . . . . .> revenue desc;
> java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or more nodes ran out of memory while executing the query.
> Fragment 5:16
> [Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> Now when I ran a jstack on the drillbit I still see that there are fragments in the waiting state. I attached the jstack output as well.
> My cluster config
> {code}
> No Of Nodes : 2
> DRILL_MAX_DIRECT_MEMORY="32G"
> DRILL_MAX_HEAP="4G"
> {code}
> Let me know if you need anything
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)