You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Sailesh Mukil (JIRA)" <ji...@apache.org> on 2018/05/09 00:00:00 UTC
[jira] [Resolved] (IMPALA-6975)
TestRuntimeRowFilters.test_row_filters failing with Memory limit exceeded
[ https://issues.apache.org/jira/browse/IMPALA-6975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sailesh Mukil resolved IMPALA-6975.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.1.0
> TestRuntimeRowFilters.test_row_filters failing with Memory limit exceeded
> -------------------------------------------------------------------------
>
> Key: IMPALA-6975
> URL: https://issues.apache.org/jira/browse/IMPALA-6975
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.0
> Reporter: Sailesh Mukil
> Assignee: Sailesh Mukil
> Priority: Critical
> Labels: build-failure
> Fix For: Impala 3.1.0
>
>
> {code:java}
> Error Message
> query_test/test_runtime_filters.py:171: in test_row_filters test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS' : str(WAIT_TIME_MS)}) common/impala_test_suite.py:405: in run_test_case result = self.__execute_query(target_impalad_client, query, user=user) common/impala_test_suite.py:620: in __execute_query return impalad_client.execute(query, user=user) common/impala_connection.py:160: in execute return self.__beeswax_client.execute(sql_stmt, user=user) beeswax/impala_beeswax.py:173: in execute handle = self.__execute_query(query_string.strip(), user=user) beeswax/impala_beeswax.py:341: in __execute_query self.wait_for_completion(handle) beeswax/impala_beeswax.py:361: in wait_for_completion raise ImpalaBeeswaxException("Query aborted:" + error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query aborted:Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data. E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit. E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001 E Memory left in process limit: 11.14 GB E Memory left in query limit: 22.16 KB E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB E Exprs: Total=4.00 KB Peak=4.00 KB E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB E Exprs: Total=25.12 KB Peak=25.12 KB E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB E KrpcDeferredRpcs: Total=0 Peak=45.12 KB E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB E Exprs: Total=260.00 KB Peak=260.00 KB E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB E E Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data. E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit. E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001 E Memory left in process limit: 11.14 GB E Memory left in query limit: 22.16 KB E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB E Exprs: Total=4.00 KB Peak=4.00 KB E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB E Exprs: Total=25.12 KB Peak=25.12 KB E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB E KrpcDeferredRpcs: Total=0 Peak=45.12 KB E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB E Exprs: Total=260.00 KB Peak=260.00 KB E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB
> Stacktrace
> query_test/test_runtime_filters.py:171: in test_row_filters
> test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS' : str(WAIT_TIME_MS)})
> common/impala_test_suite.py:405: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:620: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:160: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:173: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:341: in __execute_query
> self.wait_for_completion(handle)
> beeswax/impala_beeswax.py:361: in wait_for_completion
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data.
> E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit.
> E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001
> E Memory left in process limit: 11.14 GB
> E Memory left in query limit: 22.16 KB
> E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB
> E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB
> E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
> E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB
> E Exprs: Total=4.00 KB Peak=4.00 KB
> E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB
> E Exprs: Total=25.12 KB Peak=25.12 KB
> E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB
> E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB
> E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB
> E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB
> E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB
> E KrpcDeferredRpcs: Total=0 Peak=45.12 KB
> E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB
> E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB
> E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
> E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB
> E Exprs: Total=260.00 KB Peak=260.00 KB
> E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB
> E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB
> E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB
> E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB
> E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB
> E
> E Memory limit exceeded: ParquetColumnReader::ReadDataPage() failed to allocate 65533 bytes for decompressed data.
> E HDFS_SCAN_NODE (id=0) could not allocate 64.00 KB without exceeding limit.
> E Error occurred on backend impala-boost-static-burst-slave-el7-1aa3.vpc.cloudera.com:22001 by fragment f9423526590ed30b:732f1db100000001
> E Memory left in process limit: 11.14 GB
> E Memory left in query limit: 22.16 KB
> E Query(f9423526590ed30b:732f1db100000000): Limit=200.00 MB Reservation=160.00 MB ReservationLimit=160.00 MB OtherMemory=39.98 MB Total=199.98 MB Peak=199.98 MB
> E Fragment f9423526590ed30b:732f1db100000007: Reservation=134.00 MB OtherMemory=30.22 MB Total=164.22 MB Peak=164.22 MB
> E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
> E AGGREGATION_NODE (id=3): Total=4.00 KB Peak=4.00 KB
> E Exprs: Total=4.00 KB Peak=4.00 KB
> E HASH_JOIN_NODE (id=2): Reservation=132.00 MB OtherMemory=198.25 KB Total=132.19 MB Peak=132.19 MB
> E Exprs: Total=25.12 KB Peak=25.12 KB
> E Hash Join Builder (join_node_id=2): Total=157.12 KB Peak=157.12 KB
> E Hash Join Builder (join_node_id=2) Exprs: Total=149.12 KB Peak=149.12 KB
> E EXCHANGE_NODE (id=4): Reservation=14.91 MB OtherMemory=67.76 KB Total=14.97 MB Peak=14.97 MB
> E KrpcDeferredRpcs: Total=67.76 KB Peak=67.76 KB
> E EXCHANGE_NODE (id=5): Reservation=15.03 MB OtherMemory=0 Total=15.03 MB Peak=15.03 MB
> E KrpcDeferredRpcs: Total=0 Peak=45.12 KB
> E KrpcDataStreamSender (dst_id=6): Total=16.00 KB Peak=16.00 KB
> E Fragment f9423526590ed30b:732f1db100000001: Reservation=26.00 MB OtherMemory=9.75 MB Total=35.75 MB Peak=35.75 MB
> E Runtime Filter Bank: Reservation=2.00 MB ReservationLimit=2.00 MB OtherMemory=0 Total=2.00 MB Peak=2.00 MB
> E HDFS_SCAN_NODE (id=0): Reservation=24.00 MB OtherMemory=9.30 MB Total=33.30 MB Peak=33.30 MB
> E Exprs: Total=260.00 KB Peak=260.00 KB
> E KrpcDataStreamSender (dst_id=4): Total=426.57 KB Peak=458.57 KB
> E KrpcDataStreamSender (dst_id=4) Exprs: Total=256.00 KB Peak=256.00 KB
> E Fragment f9423526590ed30b:732f1db100000004: Reservation=0 OtherMemory=0 Total=0 Peak=29.59 MB
> E HDFS_SCAN_NODE (id=1): Reservation=0 OtherMemory=0 Total=0 Peak=29.32 MB
> E KrpcDataStreamSender (dst_id=5): Total=0 Peak=266.57 KB
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)