You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Sailesh Mukil (JIRA)" <ji...@apache.org> on 2018/05/04 18:21:00 UTC
[jira] [Created] (IMPALA-6975)
TestRuntimeRowFilters.test_row_filters failing with Memory limit exceeded
Sailesh Mukil created IMPALA-6975:
-------------------------------------
Summary: TestRuntimeRowFilters.test_row_filters failing with Memory limit exceeded
Key: IMPALA-6975
URL: https://issues.apache.org/jira/browse/IMPALA-6975
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.0
Reporter: Sailesh Mukil
{code:java}
Error Message
query_test/test_runtime_filters.py:171: in test_row_filters test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS' : str(WAIT_TIME_MS)}) common/impala_test_suite.py:405: in run_test_case result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:620: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:341: in __execute_query self.wait_for_completion(handle) beeswax/impala_beeswax.py:361: in wait_for_completion raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query aborted:Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data. E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit. E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001 E Memory left in process limit: 11.14 GB E Memory left in query limit: 22.16 KB E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB E Exprs: Total=4.00 KB Peak=4.00 KB E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB E Exprs: Total=25.12 KB Peak=25.12 KB E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB E KrpcDeferredRpcs: Total=0 Peak=45.12 KB E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB E Exprs: Total=260.00 KB Peak=260.00 KB E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB E E Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data. E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit. E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001 E Memory left in process limit: 11.14 GB E Memory left in query limit: 22.16 KB E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB E Exprs: Total=4.00 KB Peak=4.00 KB E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB E Exprs: Total=25.12 KB Peak=25.12 KB E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB E KrpcDeferredRpcs: Total=0 Peak=45.12 KB E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB E Exprs: Total=260.00 KB Peak=260.00 KB E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB
Stacktrace
query_test/test_runtime_filters.py:171: in test_row_filters
test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS' : str(WAIT_TIME_MS)})
common/impala_test_suite.py:405: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:620: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:341: in __execute_query
self.wait_for_completion(handle)
beeswax/impala_beeswax.py:361: in wait_for_completion
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
E Query aborted:Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data.
E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit.
E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001
E Memory left in process limit: 11.14 GB
E Memory left in query limit: 22.16 KB
E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB
E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB
E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB
E Exprs: Total=4.00 KB Peak=4.00 KB
E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB
E Exprs: Total=25.12 KB Peak=25.12 KB
E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB
E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB
E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB
E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB
E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB
E KrpcDeferredRpcs: Total=0 Peak=45.12 KB
E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB
E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB
E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB
E Exprs: Total=260.00 KB Peak=260.00 KB
E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB
E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB
E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB
E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB
E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB
E
E Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data.
E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit.
E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001
E Memory left in process limit: 11.14 GB
E Memory left in query limit: 22.16 KB
E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB
E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB
E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB
E Exprs: Total=4.00 KB Peak=4.00 KB
E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB
E Exprs: Total=25.12 KB Peak=25.12 KB
E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB
E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB
E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB
E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB
E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB
E KrpcDeferredRpcs: Total=0 Peak=45.12 KB
E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB
E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB
E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB
E Exprs: Total=260.00 KB Peak=260.00 KB
E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB
E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB
E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB
E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB
E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org