You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Daniel Becker (Jira)" <ji...@apache.org> on 2022/04/05 08:42:00 UTC
[jira] [Assigned] (IMPALA-11227) FE OOM in TestParquetBloomFilter.test_fallback_from_dict_if_no_bloom_tbl_props
[ https://issues.apache.org/jira/browse/IMPALA-11227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Becker reassigned IMPALA-11227:
--------------------------------------
Assignee: Daniel Becker
> FE OOM in TestParquetBloomFilter.test_fallback_from_dict_if_no_bloom_tbl_props
> ------------------------------------------------------------------------------
>
> Key: IMPALA-11227
> URL: https://issues.apache.org/jira/browse/IMPALA-11227
> Project: IMPALA
> Issue Type: Bug
> Affects Versions: Impala 4.1.0
> Reporter: Quanlong Huang
> Assignee: Daniel Becker
> Priority: Critical
>
> The huge values clause of the insert statement in TestParquetBloomFilter.test_fallback_from_dict_if_no_bloom_tbl_props could cause FE OOM:
> [https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5524/testReport/query_test.test_parquet_bloom_filter/TestParquetBloomFilter/test_fallback_from_dict_if_no_bloom_tbl_props_protocol__beeswax___exec_option____test_replan___1___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/]
> {code:bash}
> query_test/test_parquet_bloom_filter.py:176: in test_fallback_from_dict_if_no_bloom_tbl_props
> False)
> query_test/test_parquet_bloom_filter.py:228: in _create_table_dict_overflow
> self.execute_query(insert_stmt, vector.get_value('exec_option'))
> common/impala_test_suite.py:836: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:868: in execute_query
> return self.__execute_query(self.client, query, query_options)
> common/impala_test_suite.py:961: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:212: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:189: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:365: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> beeswax/impala_beeswax.py:359: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> beeswax/impala_beeswax.py:522: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> E MESSAGE: OutOfMemoryError: GC overhead limit exceeded
> {code}
> impalad.INFO
> {code:java}
> I0404 14:24:30.203562 19115 Frontend.java:1871] 7d4c91ed04f27bc4:d32f782600000000] Analyzing query: insert into test_fallback_from_dict_if_no_bloom_tbl_props_a60c835b.fallback_from_dict values (0),(2),(4),(6),(8),(10),(12),(14),(16),(18),(20),(22),(24),(26),(28),(30),(32),(34),(36),(38),(40),(42),(44),(46),(48),(50),(52),(54),(56),(58),(60)...
> ...
> I0404 14:25:18.025733 19115 jni-util.cc:286] 7d4c91ed04f27bc4:d32f782600000000] java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
> at java.lang.StringBuilder.<init>(StringBuilder.java:89)
> at org.apache.impala.analysis.SelectListItem.toSql(SelectListItem.java:84)
> at org.apache.impala.analysis.SelectStmt.toSql(SelectStmt.java:1235)
> at org.apache.impala.analysis.StatementBase.toSql(StatementBase.java:138)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:308)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:269)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:262)
> at org.apache.impala.analysis.SetOperationStmt$SetOperand.analyze(SetOperationStmt.java:102)
> at org.apache.impala.analysis.SetOperationStmt.analyzeOperands(SetOperationStmt.java:388)
> at org.apache.impala.analysis.SetOperationStmt.analyze(SetOperationStmt.java:318)
> at org.apache.impala.analysis.UnionStmt.analyze(UnionStmt.java:49)
> at org.apache.impala.analysis.InsertStmt.analyze(InsertStmt.java:306)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:506)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:468)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2012)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1920)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1744)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164){code}
> I saw this twice in another ubuntu-16.04-dockerised-tests job:
> [https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5523/testReport/junit/query_test.test_parquet_bloom_filter/TestParquetBloomFilter/test_fallback_from_dict_if_no_bloom_tbl_props_protocol__beeswax___exec_option____test_replan___1___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org