You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Sailesh Mukil (JIRA)" <ji...@apache.org> on 2018/05/10 16:00:00 UTC
[jira] [Created] (IMPALA-7010) Various flaky tests failing with
MemLimitExceeded on S3
Sailesh Mukil created IMPALA-7010:
-------------------------------------
Summary: Various flaky tests failing with MemLimitExceeded on S3
Key: IMPALA-7010
URL: https://issues.apache.org/jira/browse/IMPALA-7010
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.0, Impala 2.13.0
Reporter: Sailesh Mukil
*test_low_mem_limit_orderby_all*
{code:java}
Error Message
query_test/test_mem_usage_scaling.py:272: in test_low_mem_limit_orderby_all self.run_primitive_query(vector, 'primitive_orderby_all') query_test/test_mem_usage_scaling.py:260: in run_primitive_query self.low_memory_limit_test(vector, query_name, self.MIN_MEM[query_name]) query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test self.run_test_case(tpch_query, new_vector) common/impala_test_suite.py:405: in run_test_case result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:620: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:341: in __execute_query self.wait_for_completion(handle) beeswax/impala_beeswax.py:361: in wait_for_completion raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003 E Memory left in process limit: 11.40 GB E Memory left in query limit: 51.61 KB E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B E CodeGen: Total=7.72 KB Peak=973.50 KB E E Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003 E Memory left in process limit: 11.40 GB E Memory left in query limit: 51.61 KB E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B E CodeGen: Total=7.72 KB Peak=973.50 KB (1 of 3 similar)
Stacktrace
query_test/test_mem_usage_scaling.py:272: in test_low_mem_limit_orderby_all
self.run_primitive_query(vector, 'primitive_orderby_all')
query_test/test_mem_usage_scaling.py:260: in run_primitive_query
self.low_memory_limit_test(vector, query_name, self.MIN_MEM[query_name])
query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test
self.run_test_case(tpch_query, new_vector)
common/impala_test_suite.py:405: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:620: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:341: in __execute_query
self.wait_for_completion(handle)
beeswax/impala_beeswax.py:361: in wait_for_completion
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer
E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit.
E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003
E Memory left in process limit: 11.40 GB
E Memory left in query limit: 51.61 KB
E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB
E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB
E Exprs: Total=4.00 KB Peak=4.00 KB
E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B
E CodeGen: Total=7.72 KB Peak=973.50 KB
E
E Memory limit exceeded: Failed to allocate tuple buffer
E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit.
E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003
E Memory left in process limit: 11.40 GB
E Memory left in query limit: 51.61 KB
E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB
E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB
E Exprs: Total=4.00 KB Peak=4.00 KB
E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B
E CodeGen: Total=7.72 KB Peak=973.50 KB (1 of 3 similar)
Standard Error
-- executing against localhost:21000
use tpch_parquet;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET default_spillable_buffer_size=256k;
SET exec_single_node_rows_threshold=0;
SET mem_limit=200m;
MainThread: Query Name:
primitive_orderby_all
-- executing against localhost:21000
-- Description : Scan a fact table and sort 50Million rows.
-- Target test case : Order by all columns in the table without returning all
-- rows to the client.
SELECT *
FROM (
SELECT Rank() OVER (
ORDER BY l_extendedprice
,l_orderkey
,l_partkey
,l_suppkey
,l_linenumber
,l_quantity
,l_discount
,l_tax
,l_returnflag
,l_linestatus
,l_shipdate
,l_commitdate
,l_receiptdate
,l_shipinstruct
,l_shipmode
,l_comment
) AS rank
FROM lineitem
WHERE l_shipdate < '1992-05-09'
) a
WHERE rank < 10;
{code}
*test_low_mem_limit_q7*
{code:java}
Error Message
query_test/test_mem_usage_scaling.py:173: in test_low_mem_limit_q7 self.low_memory_limit_test(vector, 'tpch-q7', self.MIN_MEM_FOR_TPCH['Q7']) query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test self.run_test_case(tpch_query, new_vector) common/impala_test_suite.py:405: in run_test_case result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:620: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:341: in __execute_query self.wait_for_completion(handle) beeswax/impala_beeswax.py:361: in wait_for_completion raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003 E Memory left in process limit: 11.51 GB E Memory left in query limit: 4.56 KB E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB E SORT_NODE (id=12): Total=0 Peak=0 E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0 E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB E CodeGen: Total=7.39 KB Peak=879.50 KB E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB E Exprs: Total=11.12 KB Peak=11.12 KB E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB E Exprs: Total=7.12 KB Peak=7.12 KB E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB E CodeGen: Total=107.08 KB Peak=10.86 MB E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB E CodeGen: Total=3.33 KB Peak=557.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB E CodeGen: Total=0 Peak=218.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB E E Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003 E Memory left in process limit: 11.51 GB E Memory left in query limit: 4.56 KB E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB E SORT_NODE (id=12): Total=0 Peak=0 E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0 E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB E CodeGen: Total=7.39 KB Peak=879.50 KB E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB E Exprs: Total=11.12 KB Peak=11.12 KB E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB E Exprs: Total=7.12 KB Peak=7.12 KB E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB E CodeGen: Total=107.08 KB Peak=10.86 MB E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB E CodeGen: Total=3.33 KB Peak=557.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB E CodeGen: Total=0 Peak=218.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB
Stacktrace
query_test/test_mem_usage_scaling.py:173: in test_low_mem_limit_q7
self.low_memory_limit_test(vector, 'tpch-q7', self.MIN_MEM_FOR_TPCH['Q7'])
query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test
self.run_test_case(tpch_query, new_vector)
common/impala_test_suite.py:405: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:620: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:341: in __execute_query
self.wait_for_completion(handle)
beeswax/impala_beeswax.py:361: in wait_for_completion
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer
E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit.
E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003
E Memory left in process limit: 11.51 GB
E Memory left in query limit: 4.56 KB
E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB
E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB
E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB
E SORT_NODE (id=12): Total=0 Peak=0
E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB
E Exprs: Total=51.12 KB Peak=51.12 KB
E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0
E KrpcDeferredRpcs: Total=0 Peak=0
E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB
E CodeGen: Total=7.39 KB Peak=879.50 KB
E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB
E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB
E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB
E Exprs: Total=51.12 KB Peak=51.12 KB
E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB
E Exprs: Total=11.12 KB Peak=11.12 KB
E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB
E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB
E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB
E Exprs: Total=7.12 KB Peak=7.12 KB
E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB
E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB
E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB
E Exprs: Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB
E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB
E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB
E Exprs: Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB
E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB
E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB
E Exprs: Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB
E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
E KrpcDeferredRpcs: Total=0 Peak=0
E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB
E CodeGen: Total=107.08 KB Peak=10.86 MB
E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB
E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB
E Exprs: Total=4.00 KB Peak=4.00 KB
E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB
E CodeGen: Total=3.33 KB Peak=557.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB
E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB
E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB
E CodeGen: Total=0 Peak=218.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB
E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB
E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB
E CodeGen: Total=0 Peak=220.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB
E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB
E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB
E CodeGen: Total=0 Peak=220.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB
E CodeGen: Total=0 Peak=51.00 KB
E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB
E CodeGen: Total=0 Peak=51.00 KB
E
E Memory limit exceeded: Failed to allocate tuple buffer
E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit.
E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003
E Memory left in process limit: 11.51 GB
E Memory left in query limit: 4.56 KB
E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB
E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB
E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB
E SORT_NODE (id=12): Total=0 Peak=0
E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB
E Exprs: Total=51.12 KB Peak=51.12 KB
E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0
E KrpcDeferredRpcs: Total=0 Peak=0
E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB
E CodeGen: Total=7.39 KB Peak=879.50 KB
E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB
E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB
E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB
E Exprs: Total=51.12 KB Peak=51.12 KB
E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB
E Exprs: Total=11.12 KB Peak=11.12 KB
E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB
E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB
E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB
E Exprs: Total=7.12 KB Peak=7.12 KB
E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB
E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB
E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB
E Exprs: Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB
E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB
E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB
E Exprs: Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB
E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB
E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB
E Exprs: Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB
E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB
E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
E KrpcDeferredRpcs: Total=0 Peak=0
E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
E KrpcDeferredRpcs: Total=0 Peak=0
E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB
E CodeGen: Total=107.08 KB Peak=10.86 MB
E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB
E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB
E Exprs: Total=4.00 KB Peak=4.00 KB
E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB
E CodeGen: Total=3.33 KB Peak=557.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB
E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB
E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB
E CodeGen: Total=0 Peak=218.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB
E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB
E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB
E CodeGen: Total=0 Peak=220.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB
E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB
E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB
E CodeGen: Total=0 Peak=220.00 KB
E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB
E CodeGen: Total=0 Peak=51.00 KB
E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB
E CodeGen: Total=0 Peak=51.00 KB
Standard Error
-- executing against localhost:21000
use tpch_parquet;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET default_spillable_buffer_size=256k;
SET exec_single_node_rows_threshold=0;
SET mem_limit=160m;
MainThread: Query Name:
TPCH-Q7
-- executing against localhost:21000
select
supp_nation,
cust_nation,
l_year,
sum(volume) as revenue
from (
select
n1.n_name as supp_nation,
n2.n_name as cust_nation,
year(l_shipdate) as l_year,
l_extendedprice * (1 - l_discount) as volume
from
supplier,
lineitem,
orders,
customer,
nation n1,
nation n2
where
s_suppkey = l_suppkey
and o_orderkey = l_orderkey
and c_custkey = o_custkey
and s_nationkey = n1.n_nationkey
and c_nationkey = n2.n_nationkey
and (
(n1.n_name = 'FRANCE' and n2.n_name = 'GERMANY')
or (n1.n_name = 'GERMANY' and n2.n_name = 'FRANCE')
)
and l_shipdate between '1995-01-01' and '1996-12-31'
) as shipping
group by
supp_nation,
cust_nation,
l_year
order by
supp_nation,
cust_nation,
l_year;
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)