You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Sailesh Mukil (JIRA)" <ji...@apache.org> on 2018/05/10 16:00:00 UTC
[jira] [Updated] (IMPALA-7010) Multiple flaky tests failing with
MemLimitExceeded on S3
[ https://issues.apache.org/jira/browse/IMPALA-7010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sailesh Mukil updated IMPALA-7010:
----------------------------------
Summary: Multiple flaky tests failing with MemLimitExceeded on S3 (was: Various flaky tests failing with MemLimitExceeded on S3)
> Multiple flaky tests failing with MemLimitExceeded on S3
> --------------------------------------------------------
>
> Key: IMPALA-7010
> URL: https://issues.apache.org/jira/browse/IMPALA-7010
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.0, Impala 2.13.0
> Reporter: Sailesh Mukil
> Priority: Blocker
>
> *test_low_mem_limit_orderby_all*
> {code:java}
> Error Message
> query_test/test_mem_usage_scaling.py:272: in test_low_mem_limit_orderby_all self.run_primitive_query(vector, 'primitive_orderby_all') query_test/test_mem_usage_scaling.py:260: in run_primitive_query self.low_memory_limit_test(vector, query_name, self.MIN_MEM[query_name]) query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test self.run_test_case(tpch_query, new_vector) common/impala_test_suite.py:405: in run_test_case result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:620: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:341: in __execute_query self.wait_for_completion(handle) beeswax/impala_beeswax.py:361: in wait_for_completion raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003 E Memory left in process limit: 11.40 GB E Memory left in query limit: 51.61 KB E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B E CodeGen: Total=7.72 KB Peak=973.50 KB E E Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003 E Memory left in process limit: 11.40 GB E Memory left in query limit: 51.61 KB E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B E CodeGen: Total=7.72 KB Peak=973.50 KB (1 of 3 similar)
> Stacktrace
> query_test/test_mem_usage_scaling.py:272: in test_low_mem_limit_orderby_all
> self.run_primitive_query(vector, 'primitive_orderby_all')
> query_test/test_mem_usage_scaling.py:260: in run_primitive_query
> self.low_memory_limit_test(vector, query_name, self.MIN_MEM[query_name])
> query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test
> self.run_test_case(tpch_query, new_vector)
> common/impala_test_suite.py:405: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:620: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:160: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:173: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:341: in __execute_query
> self.wait_for_completion(handle)
> beeswax/impala_beeswax.py:361: in wait_for_completion
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer
> E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit.
> E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003
> E Memory left in process limit: 11.40 GB
> E Memory left in query limit: 51.61 KB
> E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
> E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
> E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB
> E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB
> E Exprs: Total=4.00 KB Peak=4.00 KB
> E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B
> E CodeGen: Total=7.72 KB Peak=973.50 KB
> E
> E Memory limit exceeded: Failed to allocate tuple buffer
> E HDFS_SCAN_NODE (id=0) could not allocate 190.00 KB without exceeding limit.
> E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment db44c56dcd2fce95:7d746e0800000003
> E Memory left in process limit: 11.40 GB
> E Memory left in query limit: 51.61 KB
> E Query(db44c56dcd2fce95:7d746e0800000000): Limit=200.00 MB Reservation=158.50 MB ReservationLimit=160.00 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
> E Fragment db44c56dcd2fce95:7d746e0800000003: Reservation=158.50 MB OtherMemory=41.45 MB Total=199.95 MB Peak=199.95 MB
> E SORT_NODE (id=1): Reservation=9.00 MB OtherMemory=8.00 KB Total=9.01 MB Peak=22.31 MB
> E HDFS_SCAN_NODE (id=0): Reservation=149.50 MB OtherMemory=41.43 MB Total=190.93 MB Peak=192.13 MB
> E Exprs: Total=4.00 KB Peak=4.00 KB
> E KrpcDataStreamSender (dst_id=4): Total=688.00 B Peak=688.00 B
> E CodeGen: Total=7.72 KB Peak=973.50 KB (1 of 3 similar)
> Standard Error
> -- executing against localhost:21000
> use tpch_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET default_spillable_buffer_size=256k;
> SET exec_single_node_rows_threshold=0;
> SET mem_limit=200m;
> MainThread: Query Name:
> primitive_orderby_all
> -- executing against localhost:21000
> -- Description : Scan a fact table and sort 50Million rows.
> -- Target test case : Order by all columns in the table without returning all
> -- rows to the client.
> SELECT *
> FROM (
> SELECT Rank() OVER (
> ORDER BY l_extendedprice
> ,l_orderkey
> ,l_partkey
> ,l_suppkey
> ,l_linenumber
> ,l_quantity
> ,l_discount
> ,l_tax
> ,l_returnflag
> ,l_linestatus
> ,l_shipdate
> ,l_commitdate
> ,l_receiptdate
> ,l_shipinstruct
> ,l_shipmode
> ,l_comment
> ) AS rank
> FROM lineitem
> WHERE l_shipdate < '1992-05-09'
> ) a
> WHERE rank < 10;
> {code}
> *test_low_mem_limit_q7*
> {code:java}
> Error Message
> query_test/test_mem_usage_scaling.py:173: in test_low_mem_limit_q7 self.low_memory_limit_test(vector, 'tpch-q7', self.MIN_MEM_FOR_TPCH['Q7']) query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test self.run_test_case(tpch_query, new_vector) common/impala_test_suite.py:405: in run_test_case result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:620: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:341: in __execute_query self.wait_for_completion(handle) beeswax/impala_beeswax.py:361: in wait_for_completion raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003 E Memory left in process limit: 11.51 GB E Memory left in query limit: 4.56 KB E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB E SORT_NODE (id=12): Total=0 Peak=0 E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0 E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB E CodeGen: Total=7.39 KB Peak=879.50 KB E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB E Exprs: Total=11.12 KB Peak=11.12 KB E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB E Exprs: Total=7.12 KB Peak=7.12 KB E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB E CodeGen: Total=107.08 KB Peak=10.86 MB E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB E CodeGen: Total=3.33 KB Peak=557.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB E CodeGen: Total=0 Peak=218.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB E E Memory limit exceeded: Failed to allocate tuple buffer E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit. E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003 E Memory left in process limit: 11.51 GB E Memory left in query limit: 4.56 KB E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB E SORT_NODE (id=12): Total=0 Peak=0 E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0 E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB E CodeGen: Total=7.39 KB Peak=879.50 KB E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB E Exprs: Total=51.12 KB Peak=51.12 KB E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB E Exprs: Total=11.12 KB Peak=11.12 KB E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB E Exprs: Total=7.12 KB Peak=7.12 KB E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB E Exprs: Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB E KrpcDeferredRpcs: Total=0 Peak=0 E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB E CodeGen: Total=107.08 KB Peak=10.86 MB E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB E Exprs: Total=4.00 KB Peak=4.00 KB E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB E CodeGen: Total=3.33 KB Peak=557.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB E CodeGen: Total=0 Peak=218.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB E CodeGen: Total=0 Peak=220.00 KB E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB E CodeGen: Total=0 Peak=51.00 KB
> Stacktrace
> query_test/test_mem_usage_scaling.py:173: in test_low_mem_limit_q7
> self.low_memory_limit_test(vector, 'tpch-q7', self.MIN_MEM_FOR_TPCH['Q7'])
> query_test/test_mem_usage_scaling.py:114: in low_memory_limit_test
> self.run_test_case(tpch_query, new_vector)
> common/impala_test_suite.py:405: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:620: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:160: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:173: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:341: in __execute_query
> self.wait_for_completion(handle)
> beeswax/impala_beeswax.py:361: in wait_for_completion
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Memory limit exceeded: Failed to allocate tuple buffer
> E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit.
> E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003
> E Memory left in process limit: 11.51 GB
> E Memory left in query limit: 4.56 KB
> E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB
> E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB
> E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB
> E SORT_NODE (id=12): Total=0 Peak=0
> E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB
> E Exprs: Total=51.12 KB Peak=51.12 KB
> E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0
> E KrpcDeferredRpcs: Total=0 Peak=0
> E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB
> E CodeGen: Total=7.39 KB Peak=879.50 KB
> E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB
> E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB
> E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB
> E Exprs: Total=51.12 KB Peak=51.12 KB
> E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB
> E Exprs: Total=11.12 KB Peak=11.12 KB
> E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB
> E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB
> E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB
> E Exprs: Total=7.12 KB Peak=7.12 KB
> E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB
> E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB
> E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB
> E Exprs: Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB
> E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB
> E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB
> E Exprs: Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB
> E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB
> E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB
> E Exprs: Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB
> E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB
> E CodeGen: Total=107.08 KB Peak=10.86 MB
> E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB
> E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
> E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB
> E Exprs: Total=4.00 KB Peak=4.00 KB
> E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB
> E CodeGen: Total=3.33 KB Peak=557.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB
> E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB
> E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB
> E CodeGen: Total=0 Peak=218.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB
> E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB
> E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB
> E CodeGen: Total=0 Peak=220.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB
> E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB
> E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB
> E CodeGen: Total=0 Peak=220.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
> E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
> E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB
> E CodeGen: Total=0 Peak=51.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
> E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
> E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB
> E CodeGen: Total=0 Peak=51.00 KB
> E
> E Memory limit exceeded: Failed to allocate tuple buffer
> E HDFS_SCAN_NODE (id=1) could not allocate 49.00 KB without exceeding limit.
> E Error occurred on backend ec2-m2-4xlarge-centos-6-4-0e8b.vpc.cloudera.com:22001 by fragment 554fa389af3a9d36:8cd965ba00000003
> E Memory left in process limit: 11.51 GB
> E Memory left in query limit: 4.56 KB
> E Query(554fa389af3a9d36:8cd965ba00000000): Limit=160.00 MB Reservation=124.06 MB ReservationLimit=128.00 MB OtherMemory=35.93 MB Total=160.00 MB Peak=160.00 MB
> E Unclaimed reservations: Reservation=12.75 MB OtherMemory=0 Total=12.75 MB Peak=62.34 MB
> E Fragment 554fa389af3a9d36:8cd965ba00000010: Reservation=0 OtherMemory=68.92 KB Total=68.92 KB Peak=941.04 KB
> E SORT_NODE (id=12): Total=0 Peak=0
> E AGGREGATION_NODE (id=20): Total=51.12 KB Peak=51.12 KB
> E Exprs: Total=51.12 KB Peak=51.12 KB
> E EXCHANGE_NODE (id=19): Reservation=0 OtherMemory=0 Total=0 Peak=0
> E KrpcDeferredRpcs: Total=0 Peak=0
> E KrpcDataStreamSender (dst_id=21): Total=2.41 KB Peak=2.41 KB
> E CodeGen: Total=7.39 KB Peak=879.50 KB
> E Fragment 554fa389af3a9d36:8cd965ba0000000d: Reservation=20.31 MB OtherMemory=23.86 MB Total=44.18 MB Peak=44.18 MB
> E Runtime Filter Bank: Reservation=5.00 MB ReservationLimit=5.00 MB OtherMemory=0 Total=5.00 MB Peak=5.00 MB
> E AGGREGATION_NODE (id=11): Total=51.12 KB Peak=51.12 KB
> E Exprs: Total=51.12 KB Peak=51.12 KB
> E HASH_JOIN_NODE (id=10): Reservation=1.94 MB OtherMemory=58.25 KB Total=1.99 MB Peak=2.01 MB
> E Exprs: Total=11.12 KB Peak=11.12 KB
> E Hash Join Builder (join_node_id=10): Total=7.12 KB Peak=15.12 KB
> E Hash Join Builder (join_node_id=10) Exprs: Total=7.12 KB Peak=7.12 KB
> E HASH_JOIN_NODE (id=9): Reservation=1.94 MB OtherMemory=46.25 KB Total=1.98 MB Peak=2.00 MB
> E Exprs: Total=7.12 KB Peak=7.12 KB
> E Hash Join Builder (join_node_id=9): Total=7.12 KB Peak=15.12 KB
> E Hash Join Builder (join_node_id=9) Exprs: Total=7.12 KB Peak=7.12 KB
> E HASH_JOIN_NODE (id=8): Reservation=4.75 MB OtherMemory=58.25 KB Total=4.81 MB Peak=4.81 MB
> E Exprs: Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=8): Total=13.12 KB Peak=21.12 KB
> E Hash Join Builder (join_node_id=8) Exprs: Total=13.12 KB Peak=13.12 KB
> E HASH_JOIN_NODE (id=7): Reservation=1.94 MB OtherMemory=42.25 KB Total=1.98 MB Peak=1.99 MB
> E Exprs: Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=7): Total=13.12 KB Peak=21.12 KB
> E Hash Join Builder (join_node_id=7) Exprs: Total=13.12 KB Peak=13.12 KB
> E HASH_JOIN_NODE (id=6): Reservation=4.75 MB OtherMemory=42.25 KB Total=4.79 MB Peak=4.79 MB
> E Exprs: Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=6): Total=13.12 KB Peak=13.12 KB
> E Hash Join Builder (join_node_id=6) Exprs: Total=13.12 KB Peak=13.12 KB
> E EXCHANGE_NODE (id=13): Reservation=12.66 MB OtherMemory=0 Total=12.66 MB Peak=12.66 MB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=14): Reservation=7.36 MB OtherMemory=0 Total=7.36 MB Peak=7.36 MB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=15): Reservation=0 OtherMemory=0 Total=0 Peak=240.00 KB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=16): Reservation=3.44 MB OtherMemory=0 Total=3.44 MB Peak=3.44 MB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=17): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E EXCHANGE_NODE (id=18): Reservation=0 OtherMemory=0 Total=0 Peak=16.00 KB
> E KrpcDeferredRpcs: Total=0 Peak=0
> E KrpcDataStreamSender (dst_id=19): Total=7.24 KB Peak=7.24 KB
> E CodeGen: Total=107.08 KB Peak=10.86 MB
> E Fragment 554fa389af3a9d36:8cd965ba00000003: Reservation=91.00 MB OtherMemory=12.00 MB Total=103.00 MB Peak=103.00 MB
> E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
> E HDFS_SCAN_NODE (id=1): Reservation=89.00 MB OtherMemory=11.76 MB Total=100.76 MB Peak=100.76 MB
> E Exprs: Total=4.00 KB Peak=4.00 KB
> E KrpcDataStreamSender (dst_id=13): Total=91.83 KB Peak=91.83 KB
> E CodeGen: Total=3.33 KB Peak=557.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000006: Reservation=0 OtherMemory=0 Total=0 Peak=11.37 MB
> E HDFS_SCAN_NODE (id=2): Reservation=0 OtherMemory=0 Total=0 Peak=10.25 MB
> E KrpcDataStreamSender (dst_id=14): Total=0 Peak=106.57 KB
> E CodeGen: Total=0 Peak=218.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000007: Reservation=0 OtherMemory=0 Total=0 Peak=1.43 MB
> E HDFS_SCAN_NODE (id=0): Reservation=0 OtherMemory=0 Total=0 Peak=385.42 KB
> E KrpcDataStreamSender (dst_id=15): Total=0 Peak=34.90 KB
> E CodeGen: Total=0 Peak=220.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000008: Reservation=0 OtherMemory=0 Total=0 Peak=4.63 MB
> E HDFS_SCAN_NODE (id=3): Reservation=0 OtherMemory=0 Total=0 Peak=3.58 MB
> E KrpcDataStreamSender (dst_id=16): Total=0 Peak=34.90 KB
> E CodeGen: Total=0 Peak=220.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba00000009: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
> E HDFS_SCAN_NODE (id=4): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
> E KrpcDataStreamSender (dst_id=17): Total=0 Peak=20.20 KB
> E CodeGen: Total=0 Peak=51.00 KB
> E Fragment 554fa389af3a9d36:8cd965ba0000000a: Reservation=0 OtherMemory=0 Total=0 Peak=95.68 KB
> E HDFS_SCAN_NODE (id=5): Reservation=0 OtherMemory=0 Total=0 Peak=67.00 KB
> E KrpcDataStreamSender (dst_id=18): Total=0 Peak=20.20 KB
> E CodeGen: Total=0 Peak=51.00 KB
> Standard Error
> -- executing against localhost:21000
> use tpch_parquet;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET default_spillable_buffer_size=256k;
> SET exec_single_node_rows_threshold=0;
> SET mem_limit=160m;
> MainThread: Query Name:
> TPCH-Q7
> -- executing against localhost:21000
> select
> supp_nation,
> cust_nation,
> l_year,
> sum(volume) as revenue
> from (
> select
> n1.n_name as supp_nation,
> n2.n_name as cust_nation,
> year(l_shipdate) as l_year,
> l_extendedprice * (1 - l_discount) as volume
> from
> supplier,
> lineitem,
> orders,
> customer,
> nation n1,
> nation n2
> where
> s_suppkey = l_suppkey
> and o_orderkey = l_orderkey
> and c_custkey = o_custkey
> and s_nationkey = n1.n_nationkey
> and c_nationkey = n2.n_nationkey
> and (
> (n1.n_name = 'FRANCE' and n2.n_name = 'GERMANY')
> or (n1.n_name = 'GERMANY' and n2.n_name = 'FRANCE')
> )
> and l_shipdate between '1995-01-01' and '1996-12-31'
> ) as shipping
> group by
> supp_nation,
> cust_nation,
> l_year
> order by
> supp_nation,
> cust_nation,
> l_year;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org