You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/04/30 18:58:00 UTC
[jira] [Created] (IMPALA-9712) Hit OOM on TPC-H Q19
Tim Armstrong created IMPALA-9712:
-------------------------------------
Summary: Hit OOM on TPC-H Q19
Key: IMPALA-9712
URL: https://issues.apache.org/jira/browse/IMPALA-9712
Project: IMPALA
Issue Type: Bug
Components: Backend
Reporter: Tim Armstrong
Assignee: Tim Armstrong
We saw an out-of-memory on TPC-H 1TB with mt_dop=16.
{noformat}
Query Status: Memory limit exceeded: ParquetColumnChunkReader::ReadDataPage() failed to allocate 65454 bytes for decompressed data.
HDFS_SCAN_NODE (id=0) could not allocate 63.92 KB without exceeding limit.
Error occurred on backend impala-executor-000-1:22000 by fragment 4b497d5e4237002a:94c50a5200000041
Memory left in process limit: 71.74 GB
Memory left in query limit: 52.96 KB
Query(4b497d5e4237002a:94c50a5200000000): Limit=2.35 GB Reservation=1.56 GB ReservationLimit=1.88 GB OtherMemory=801.69 MB Total=2.35 GB Peak=2.35 GB
Runtime Filter Bank: Reservation=1.00 MB ReservationLimit=1.00 MB OtherMemory=0 Total=1.00 MB Peak=1.00 MB
Fragment 4b497d5e4237002a:94c50a52000000a7: Reservation=0 OtherMemory=0 Total=0 Peak=51.91 MB
HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=48.78 MB
KrpcDataStreamSender (dst_id=4): Total=0 Peak=31.17 KB
CodeGen: Total=7.82 KB Peak=2.52 MB
Fragment 4b497d5e4237002a:94c50a52000000a8: Reservation=0 OtherMemory=0 Total=0 Peak=40.15 MB
HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=36.84 MB
KrpcDataStreamSender (dst_id=4): Total=0 Peak=31.17 KB
Fragment 4b497d5e4237002a:94c50a52000000a9: Reservation=0 OtherMemory=0 Total=0 Peak=31.79 MB
HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=28.78 MB
KrpcDataStreamSender (dst_id=4): Total=0 Peak=31.17 KB
Fragment 4b497d5e4237002a:94c50a52000000c3: Reservation=544.00 MB OtherMemory=21.12 KB Total=544.02 MB Peak=544.36 MB
EXCHANGE_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=352.00 KB
KrpcDeferredRpcs: Total=0 Peak=0
Hash Join Builder (join_node_id=2): Reservation=544.00 MB OtherMemory=13.12 KB Total=544.01 MB Peak=544.02 MB
Hash Join Builder (join_node_id=2) Exprs: Total=13.12 KB Peak=13.12 KB
Fragment 4b497d5e4237002a:94c50a5200000041: Reservation=66.00 MB OtherMemory=51.15 MB Total=117.15 MB Peak=117.29 MB
AGGREGATION_NODE (id=3): Total=43.83 MB Peak=49.86 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=6.04 MB Total=6.04 MB Peak=8.29 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB Total=67.26 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000042: Reservation=66.00 MB OtherMemory=52.06 MB Total=118.06 MB Peak=118.06 MB
AGGREGATION_NODE (id=3): Total=50.63 MB Peak=50.63 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB Peak=7.79 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB Total=67.35 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
CodeGen: Total=4.06 KB Peak=778.50 KB
Fragment 4b497d5e4237002a:94c50a5200000043: Reservation=66.00 MB OtherMemory=47.81 MB Total=113.81 MB Peak=116.01 MB
AGGREGATION_NODE (id=3): Total=45.01 MB Peak=48.67 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=1.48 MB Total=1.48 MB Peak=7.79 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB Total=67.35 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000044: Reservation=66.00 MB OtherMemory=56.25 MB Total=122.25 MB Peak=122.25 MB
AGGREGATION_NODE (id=3): Total=54.81 MB Peak=54.81 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB Peak=8.23 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB Total=67.35 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000045: Reservation=66.00 MB OtherMemory=49.68 MB Total=115.68 MB Peak=115.68 MB
AGGREGATION_NODE (id=3): Total=43.95 MB Peak=44.61 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=4.36 MB Total=4.36 MB Peak=8.29 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB Total=67.35 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000046: Reservation=66.00 MB OtherMemory=48.62 MB Total=114.62 MB Peak=115.70 MB
AGGREGATION_NODE (id=3): Total=42.33 MB Peak=48.36 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=4.92 MB Total=4.92 MB Peak=8.86 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB Total=67.35 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000047: Reservation=66.00 MB OtherMemory=49.53 MB Total=115.53 MB Peak=115.53 MB
AGGREGATION_NODE (id=3): Total=44.45 MB Peak=47.74 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=3.79 MB Total=3.79 MB Peak=8.23 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB Total=67.26 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000048: Reservation=66.00 MB OtherMemory=54.38 MB Total=120.38 MB Peak=120.38 MB
AGGREGATION_NODE (id=3): Total=53.13 MB Peak=53.13 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB Peak=8.29 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB Total=67.17 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000049: Reservation=66.00 MB OtherMemory=47.81 MB Total=113.81 MB Peak=115.08 MB
AGGREGATION_NODE (id=3): Total=45.45 MB Peak=47.73 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=1005.84 KB Total=1005.84 KB Peak=8.23 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.35 MB Total=67.35 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a520000004a: Reservation=66.00 MB OtherMemory=50.08 MB Total=116.08 MB Peak=117.15 MB
AGGREGATION_NODE (id=3): Total=44.95 MB Peak=47.73 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=5.10 MB Total=5.10 MB Peak=7.79 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=4.00 KB Total=66.00 MB Peak=67.42 MB
Exprs: Total=4.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
CodeGen: Total=53.44 KB Peak=13.23 MB
Fragment 4b497d5e4237002a:94c50a520000004b: Reservation=66.00 MB OtherMemory=47.81 MB Total=113.81 MB Peak=115.08 MB
AGGREGATION_NODE (id=3): Total=45.95 MB Peak=47.74 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=685.92 KB Total=685.92 KB Peak=8.29 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB Total=67.17 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a520000004c: Reservation=66.00 MB OtherMemory=48.34 MB Total=114.34 MB Peak=117.29 MB
AGGREGATION_NODE (id=3): Total=43.83 MB Peak=49.86 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=3.23 MB Total=3.23 MB Peak=8.29 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB Total=67.26 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a520000004d: Reservation=66.00 MB OtherMemory=45.07 MB Total=111.07 MB Peak=115.08 MB
AGGREGATION_NODE (id=3): Total=37.84 MB Peak=47.73 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=6.04 MB Total=6.04 MB Peak=7.79 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB Total=67.17 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a520000004e: Reservation=66.00 MB OtherMemory=45.81 MB Total=111.81 MB Peak=113.58 MB
AGGREGATION_NODE (id=3): Total=43.95 MB Peak=46.24 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=685.92 KB Total=685.92 KB Peak=7.73 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.17 MB Total=67.17 MB Peak=67.33 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a520000004f: Reservation=66.00 MB OtherMemory=54.59 MB Total=120.59 MB Peak=120.59 MB
AGGREGATION_NODE (id=3): Total=53.25 MB Peak=53.25 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=57.12 KB Total=57.12 KB Peak=8.29 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB Total=67.26 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
Fragment 4b497d5e4237002a:94c50a5200000050: Reservation=66.00 MB OtherMemory=51.47 MB Total=117.47 MB Peak=117.47 MB
AGGREGATION_NODE (id=3): Total=46.45 MB Peak=48.23 MB
NonGroupingAggregator 0: Total=8.00 KB Peak=8.00 KB
Exprs: Total=4.00 KB Peak=4.00 KB
HASH_JOIN_NODE (id=2): Reservation=0 OtherMemory=3.73 MB Total=3.73 MB Peak=8.86 MB
Exprs: Total=41.12 KB Peak=41.12 KB
HDFS_SCAN_NODE (id=0): Reservation=66.00 MB OtherMemory=1.26 MB Total=67.26 MB Peak=67.42 MB
Exprs: Total=8.00 KB Peak=8.00 KB
KrpcDataStreamSender (dst_id=5): Total=14.22 KB Peak=14.22 KB
{noformat}
The aggregation node is accumulating a lot of memory:
{noformat}
AGGREGATION_NODE (id=3): Total=43.83 MB Peak=49.86 MB
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)