You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/11/17 21:05:00 UTC
[jira] [Assigned] (IMPALA-10251) test_decimal_queries and
test_tpcds_queries may run out of memory on ASAN builds
[ https://issues.apache.org/jira/browse/IMPALA-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong reassigned IMPALA-10251:
--------------------------------------
Assignee: Tim Armstrong
> test_decimal_queries and test_tpcds_queries may run out of memory on ASAN builds
> --------------------------------------------------------------------------------
>
> Key: IMPALA-10251
> URL: https://issues.apache.org/jira/browse/IMPALA-10251
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: broken-build, flaky
>
> Encountered these test failures due to running out of memory:
> {code:java}
> query_test.test_decimal_queries.TestDecimalQueries.test_queries[protocol: beeswax | exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'false', 'decimal_v2': 'true', 'batch_size': 1} | table_format: orc/def/block]query_test.test_tpcds_queries.TestTpcdsQuery.test_tpcds_q51a[protocol: beeswax | exec_option: {'decimal_v2': 0, 'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]query_test.test_decimal_queries.TestDecimalQueries.test_queries[protocol: hs2 | exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true', 'decimal_v2': 'true', 'batch_size': 1} | table_format: parquet/none]query_test.test_tpcds_queries.TestTpcdsDecimalV2Query.test_tpcds_q18a[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]query_test.test_tpcds_queries.TestTpcdsDecimalV2Query.test_tpcds_q23_1[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]query_test.test_tpcds_queries.TestTpcdsDecimalV2Query.test_tpcds_q4[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] {code}
> The error is
> {code}
> query_test/test_decimal_queries.py:55: in test_queries
> self.run_test_case('QueryTest/decimal', vector)
> common/impala_test_suite.py:662: in run_test_case
> result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:600: in __exec_in_impala
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:909: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:365: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:386: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Failed to get minimum memory reservation of 68.01 MB on daemon impala-ec2-centos74-r5-4xlarge-ondemand-109b.vpc.cloudera.com:27001 for query 2b41e9326825c48d:15a4d93100000000 due to following error: Failed to increase reservation by 68.01 MB because it would exceed the applicable reservation limit for the "Process" ReservationTracker: reservation_limit=10.20 GB reservation=10.16 GB used_reservation=0 child_reservations=10.16 GB
> E The top 5 queries that allocated memory under this tracker are:
> E Query(ca4c47a813876f70:d626ca0c00000000): Reservation=9.60 GB ReservationLimit=9.60 GB OtherMemory=115.84 MB Total=9.71 GB Peak=9.71 GB
> E Query(3f4e984e6b54204d:2597d87f00000000): Reservation=426.77 MB ReservationLimit=9.60 GB OtherMemory=113.19 MB Total=539.95 MB Peak=540.00 MB
> E Query(6c4aca5cba7141b7:cb0d9a4000000000): Limit=512.00 MB Reservation=68.03 MB ReservationLimit=409.60 MB OtherMemory=2.59 MB Total=70.62 MB Peak=71.93 MB
> E Query(e84c2ac37bac9309:7eddf8bb00000000): Reservation=14.94 MB ReservationLimit=9.60 GB OtherMemory=5.55 MB Total=20.48 MB Peak=39.46 MB
> E Query(eb44550f4e6aebbd:4b72c39600000000): Reservation=12.31 MB ReservationLimit=9.60 GB OtherMemory=5.85 MB Total=18.16 MB Peak=18.16 MB
> E
> E
> E
> E
> E
> E Memory is likely oversubscribed. Reducing query concurrency or configuring admission control may help avoid this error.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org