[jira] [Created] (IMPALA-11227) FE OOM in TestParquetBloomFilter.test_fallback_from_dict_if_no_bloom_tbl_props

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-11227:
---------------------------------------

             Summary: FE OOM in TestParquetBloomFilter.test_fallback_from_dict_if_no_bloom_tbl_props
                 Key: IMPALA-11227
                 URL: https://issues.apache.org/jira/browse/IMPALA-11227
             Project: IMPALA
          Issue Type: Bug
            Reporter: Quanlong Huang


The huge values clause of the insert statement in TestParquetBloomFilter.test_fallback_from_dict_if_no_bloom_tbl_props could cause FE OOM:
[https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5524/testReport/query_test.test_parquet_bloom_filter/TestParquetBloomFilter/test_fallback_from_dict_if_no_bloom_tbl_props_protocol__beeswax___exec_option____test_replan___1___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/]
{code:bash}
query_test/test_parquet_bloom_filter.py:176: in test_fallback_from_dict_if_no_bloom_tbl_props
    False)
query_test/test_parquet_bloom_filter.py:228: in _create_table_dict_overflow
    self.execute_query(insert_stmt, vector.get_value('exec_option'))
common/impala_test_suite.py:836: in wrapper
    return function(*args, **kwargs)
common/impala_test_suite.py:868: in execute_query
    return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:961: in __execute_query
    return impalad_client.execute(query, user=user)
common/impala_connection.py:212: in execute
    return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:189: in execute
    handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:365: in __execute_query
    handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:359: in execute_query_async
    handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:522: in __do_rpc
    raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E   ImpalaBeeswaxException: ImpalaBeeswaxException:
E    INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
E    MESSAGE: OutOfMemoryError: GC overhead limit exceeded
{code}
impalad.INFO
{code:java}
I0404 14:24:30.203562 19115 Frontend.java:1871] 7d4c91ed04f27bc4:d32f782600000000] Analyzing query: insert into test_fallback_from_dict_if_no_bloom_tbl_props_a60c835b.fallback_from_dict values (0),(2),(4),(6),(8),(10),(12),(14),(16),(18),(20),(22),(24),(26),(28),(30),(32),(34),(36),(38),(40),(42),(44),(46),(48),(50),(52),(54),(56),(58),(60)...
...
I0404 14:25:18.025733 19115 jni-util.cc:286] 7d4c91ed04f27bc4:d32f782600000000] java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:68)
        at java.lang.StringBuilder.<init>(StringBuilder.java:89)
        at org.apache.impala.analysis.SelectListItem.toSql(SelectListItem.java:84)
        at org.apache.impala.analysis.SelectStmt.toSql(SelectStmt.java:1235)
        at org.apache.impala.analysis.StatementBase.toSql(StatementBase.java:138)
        at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:308)
        at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:269)
        at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:262)
        at org.apache.impala.analysis.SetOperationStmt$SetOperand.analyze(SetOperationStmt.java:102)
        at org.apache.impala.analysis.SetOperationStmt.analyzeOperands(SetOperationStmt.java:388)
        at org.apache.impala.analysis.SetOperationStmt.analyze(SetOperationStmt.java:318)
        at org.apache.impala.analysis.UnionStmt.analyze(UnionStmt.java:49)
        at org.apache.impala.analysis.InsertStmt.analyze(InsertStmt.java:306)
        at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:506)
        at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:468)
        at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2012)
        at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1920)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1744)
        at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164){code}
I saw this twice in another ubuntu-16.04-dockerised-tests job:

[https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/5523/testReport/junit/query_test.test_parquet_bloom_filter/TestParquetBloomFilter/test_fallback_from_dict_if_no_bloom_tbl_props_protocol__beeswax___exec_option____test_replan___1___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__parquet_none_/]



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org