You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Rahul Challapalli (JIRA)" <ji...@apache.org> on 2015/05/14 01:15:59 UTC
[jira] [Updated] (DRILL-3070) Memory Leak when we run out of memory
[ https://issues.apache.org/jira/browse/DRILL-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rahul Challapalli updated DRILL-3070:
-------------------------------------
Description:
git.commit.id.abbrev=d10769f
No of Drillbits : 2
DRILL_MAX_DIRECT_MEMORY="1G"
I ran the below query on tpsh SF 100
{code}
select
l.l_shipmode,
sum(case
when o.o_orderpriority = '1-URGENT'
or o.o_orderpriority = '2-HIGH'
then 1
else 0
end) as high_line_count,
sum(case
when o.o_orderpriority <> '1-URGENT'
and o.o_orderpriority <> '2-HIGH'
then 1
else 0
end) as low_line_count
from
orders o,
lineitem l
where
o.o_orderkey = l.l_orderkey
and l.l_shipmode in ('TRUCK', 'REG AIR')
and l.l_commitdate < l.l_receiptdate
and l.l_shipdate < l.l_commitdate
and l.l_receiptdate >= date '1994-01-01'
and l.l_receiptdate < date '1994-01-01' + interval '1' year
group by
l.l_shipmode
order by
l.l_shipmode;
+------------+-----------------+----------------+
| l_shipmode | high_line_count | low_line_count |
+------------+-----------------+----------------+
java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: org.apache.drill.exec.memory.OutOfMemoryRuntimeException: Failure while reallocating buffer to 262152 bytes
Fragment 7:30
[Error Id: 65a09d7d-75f2-4bbb-b8f9-cfb0bdfc2655 on qa-node191.qa.lab:31010]
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Before memory :
{code}
0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
+------------+------------+--------------+------------+----------------+--------------------+------------+
| hostname | user_port | heap_current | heap_max | direct_current | jvm_direct_current | direct_max |
+------------+------------+--------------+------------+----------------+--------------------+------------+
| qa-node190.qa.lab | 31010 | 126726408 | 4151836672 | 11799087 | 100676940 | 1073741824 |
| qa-node191.qa.lab | 31010 | 299617984 | 4151836672 | 7750365 | 50345292 | 1073741824 |
+------------+------------+--------------+------------+----------------+--------------------+------------+
{code}
After Memory :
{code}
0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
+------------+------------+--------------+------------+----------------+--------------------+------------+
| hostname | user_port | heap_current | heap_max | direct_current | jvm_direct_current | direct_max |
+------------+------------+--------------+------------+----------------+--------------------+------------+
| qa-node190.qa.lab | 31010 | 446765888 | 4151836672 | 161309493 | 436221380 | 1073741824 |
| qa-node191.qa.lab | 31010 | 357580248 | 4151836672 | 199260917 | 603993536 | 1073741824 |
+------------+------------+--------------+------------+----------------+--------------------+------------+
{code}
While I expected this OOM error (as direct mem is 1GB), there seems to be a bump in directory_memory on qa-node191.qa.lab. Not sure if this is normal
- Rahul
was:
git.commit.id.abbrev=d10769f
No of Drillbits : 2
DRILL_MAX_DIRECT_MEMORY="1G"
I ran the below query on tpsh SF 100
{code}
select
l.l_shipmode,
sum(case
when o.o_orderpriority = '1-URGENT'
or o.o_orderpriority = '2-HIGH'
then 1
else 0
end) as high_line_count,
sum(case
when o.o_orderpriority <> '1-URGENT'
and o.o_orderpriority <> '2-HIGH'
then 1
else 0
end) as low_line_count
from
orders o,
lineitem l
where
o.o_orderkey = l.l_orderkey
and l.l_shipmode in ('TRUCK', 'REG AIR')
and l.l_commitdate < l.l_receiptdate
and l.l_shipdate < l.l_commitdate
and l.l_receiptdate >= date '1994-01-01'
and l.l_receiptdate < date '1994-01-01' + interval '1' year
group by
l.l_shipmode
order by
l.l_shipmode;
+------------+-----------------+----------------+
| l_shipmode | high_line_count | low_line_count |
+------------+-----------------+----------------+
java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: org.apache.drill.exec.memory.OutOfMemoryRuntimeException: Failure while reallocating buffer to 262152 bytes
Fragment 7:30
[Error Id: 65a09d7d-75f2-4bbb-b8f9-cfb0bdfc2655 on qa-node191.qa.lab:31010]
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Before memory :
{code}
0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
+------------+------------+--------------+------------+----------------+--------------------+------------+
| hostname | user_port | heap_current | heap_max | direct_current | jvm_direct_current | direct_max |
+------------+------------+--------------+------------+----------------+--------------------+------------+
| qa-node190.qa.lab | 31010 | 126726408 | 4151836672 | 11799087 | 100676940 | 1073741824 |
| qa-node191.qa.lab | 31010 | 299617984 | 4151836672 | 7750365 | 50345292 | 1073741824 |
+------------+------------+--------------+------------+----------------+--------------------+------------+
{code}
After Memory :
{code}
0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
+------------+------------+--------------+------------+----------------+--------------------+------------+
| hostname | user_port | heap_current | heap_max | direct_current | jvm_direct_current | direct_max |
+------------+------------+--------------+------------+----------------+--------------------+------------+
| qa-node190.qa.lab | 31010 | 446765888 | 4151836672 | 161309493 | 436221380 | 1073741824 |
| qa-node191.qa.lab | 31010 | 357580248 | 4151836672 | 199260917 | 603993536 | 1073741824 |
+------------+------------+--------------+------------+----------------+--------------------+------------+
{code}
There seems to be a bump in directory_memory on qa-node191.qa.lab. Not sure if this is normal
- Rahul
> Memory Leak when we run out of memory
> -------------------------------------
>
> Key: DRILL-3070
> URL: https://issues.apache.org/jira/browse/DRILL-3070
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Reporter: Rahul Challapalli
> Assignee: Chris Westin
>
> git.commit.id.abbrev=d10769f
> No of Drillbits : 2
> DRILL_MAX_DIRECT_MEMORY="1G"
> I ran the below query on tpsh SF 100
> {code}
> select
> l.l_shipmode,
> sum(case
> when o.o_orderpriority = '1-URGENT'
> or o.o_orderpriority = '2-HIGH'
> then 1
> else 0
> end) as high_line_count,
> sum(case
> when o.o_orderpriority <> '1-URGENT'
> and o.o_orderpriority <> '2-HIGH'
> then 1
> else 0
> end) as low_line_count
> from
> orders o,
> lineitem l
> where
> o.o_orderkey = l.l_orderkey
> and l.l_shipmode in ('TRUCK', 'REG AIR')
> and l.l_commitdate < l.l_receiptdate
> and l.l_shipdate < l.l_commitdate
> and l.l_receiptdate >= date '1994-01-01'
> and l.l_receiptdate < date '1994-01-01' + interval '1' year
> group by
> l.l_shipmode
> order by
> l.l_shipmode;
> +------------+-----------------+----------------+
> | l_shipmode | high_line_count | low_line_count |
> +------------+-----------------+----------------+
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: org.apache.drill.exec.memory.OutOfMemoryRuntimeException: Failure while reallocating buffer to 262152 bytes
> Fragment 7:30
> [Error Id: 65a09d7d-75f2-4bbb-b8f9-cfb0bdfc2655 on qa-node191.qa.lab:31010]
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
> Before memory :
> {code}
> 0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> | hostname | user_port | heap_current | heap_max | direct_current | jvm_direct_current | direct_max |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> | qa-node190.qa.lab | 31010 | 126726408 | 4151836672 | 11799087 | 100676940 | 1073741824 |
> | qa-node191.qa.lab | 31010 | 299617984 | 4151836672 | 7750365 | 50345292 | 1073741824 |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> {code}
> After Memory :
> {code}
> 0: jdbc:drill:schema=dfs_eea> select * from sys.memory;
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> | hostname | user_port | heap_current | heap_max | direct_current | jvm_direct_current | direct_max |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> | qa-node190.qa.lab | 31010 | 446765888 | 4151836672 | 161309493 | 436221380 | 1073741824 |
> | qa-node191.qa.lab | 31010 | 357580248 | 4151836672 | 199260917 | 603993536 | 1073741824 |
> +------------+------------+--------------+------------+----------------+--------------------+------------+
> {code}
> While I expected this OOM error (as direct mem is 1GB), there seems to be a bump in directory_memory on qa-node191.qa.lab. Not sure if this is normal
> - Rahul
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)