You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Thomas Tauber-Marshall (JIRA)" <ji...@apache.org> on 2017/05/04 15:47:04 UTC
[jira] [Resolved] (IMPALA-5080) test_java_udfs: OutOfMemoryError:
PermGen space
[ https://issues.apache.org/jira/browse/IMPALA-5080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Tauber-Marshall resolved IMPALA-5080.
--------------------------------------------
Resolution: Fixed
commit 920641ffdce1618199ac4d18493151acfc96cda6
Author: Thomas Tauber-Marshall <tm...@cloudera.com>
Date: Wed May 3 11:46:38 2017 -0700
IMPALA-5080: OutOfMemory PermGen space
A previous fix for this increased the MaxPermGen size.
However, during the review process an extraneous character
was accidentally inserted that rendered the fix ineffective.
Testing:
- Ran the test, test_java_udfs, in a loop for a while.
Change-Id: If0096384e419c0a059f555d325314e46e7ebfdb8
Reviewed-on: http://gerrit.cloudera.org:8080/6790
Reviewed-by: Alex Behm <al...@cloudera.com>
Reviewed-by: Matthew Jacobs <mj...@cloudera.com>
Tested-by: Impala Public Jenkins
> test_java_udfs: OutOfMemoryError: PermGen space
> -----------------------------------------------
>
> Key: IMPALA-5080
> URL: https://issues.apache.org/jira/browse/IMPALA-5080
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.9.0
> Reporter: Michael Brown
> Assignee: Thomas Tauber-Marshall
> Priority: Blocker
> Labels: broken-build
> Fix For: Impala 2.9.0
>
>
> There was a Java OOM related to PermGen space in a release-mode UDF test. Googling, it seems this is a memory area for Java, and that one gets around it by increasing limits or other tweaks.
> The UDF code hasn't changed in a while. Was there a recent change that used up too much memory, and as a side-effect this test can fail? Have we slowly been approaching this limit all along?
> end-to-end traceback:
> {noformat}
> TestUdfExecution.test_java_udfs[exec_option: {'disable_codegen': True, 'exec_single_node_rows_threshold': 100, 'enable_expr_rewrites': True} | table_format: text/none]
> query_test/test_udfs.py:309: in test_java_udfs
> self.run_test_case('QueryTest/java-udf', vector, use_db=unique_database)
> common/impala_test_suite.py:359: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:567: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:160: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:173: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:339: in __execute_query
> self.wait_for_completion(handle)
> beeswax/impala_beeswax.py:359: in wait_for_completion
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:
> E OutOfMemoryError: PermGen space
> {noformat}
> {noformat}
> select * from
> (select max(int_col) from functional.alltypesagg
> where identity(bool_col) union all
> (select max(int_col) from functional.alltypesagg
> where identity(tinyint_col) > 1 union all
> (select max(int_col) from functional.alltypesagg
> where identity(smallint_col) > 1 union all
> (select max(int_col) from functional.alltypesagg
> where identity(int_col) > 1 union all
> (select max(int_col) from functional.alltypesagg
> where identity(bigint_col) > 1 union all
> (select max(int_col) from functional.alltypesagg
> where identity(float_col) > 1.0 union all
> (select max(int_col) from functional.alltypesagg
> where identity(double_col) > 1.0 union all
> (select max(int_col) from functional.alltypesagg
> where identity(string_col) > '1' union all
> (select max(int_col) from functional.alltypesagg
> where not identity(bool_col) union all
> (select max(int_col) from functional.alltypesagg
> where identity(tinyint_col) > 2 union all
> (select max(int_col) from functional.alltypesagg
> where identity(smallint_col) > 2 union all
> (select max(int_col) from functional.alltypesagg
> where identity(int_col) > 2 union all
> (select max(int_col) from functional.alltypesagg
> where identity(bigint_col) > 2 union all
> (select max(int_col) from functional.alltypesagg
> where identity(float_col) > 2.0 union all
> (select max(int_col) from functional.alltypesagg
> where identity(double_col) > 2.0 union all
> (select max(int_col) from functional.alltypesagg
> where identity(string_col) > '2'
> )))))))))))))))) v;
> {noformat}
> from the impalad.log:
> {noformat}
> I0316 10:57:33.113581 6342 jni-util.cc:176] java.lang.OutOfMemoryError: PermGen space
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:789)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:274)
> at org.apache.impala.hive.executor.UdfExecutor.init(UdfExecutor.java:584)
> at org.apache.impala.hive.executor.UdfExecutor.<init>(UdfExecutor.java:218)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)