You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Sunil Govindan (JIRA)" <ji...@apache.org> on 2018/08/23 03:27:00 UTC
[jira] [Assigned] (YARN-8691) AMRMClient
unregisterApplicationMaster Api's appMessage should have a maximum size
[ https://issues.apache.org/jira/browse/YARN-8691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sunil Govindan reassigned YARN-8691:
------------------------------------
Assignee: Yicong Cai
> AMRMClient unregisterApplicationMaster Api's appMessage should have a maximum size
> ----------------------------------------------------------------------------------
>
> Key: YARN-8691
> URL: https://issues.apache.org/jira/browse/YARN-8691
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Affects Versions: 2.7.3
> Reporter: Yicong Cai
> Assignee: Yicong Cai
> Priority: Critical
> Fix For: 2.7.7
>
>
> SparkSQL AM Codegen ERROR，then call unregister AM API and send the error message to RM, RM receive the AM state and update to RMStateStore. The Codegen error message maybe is huge, (Our case is about 200MB). If the RMStateStore is ZKRMStateStore, it causes the same exception as YARN-6125, but YARN-6125 doesn't cover the unregisterApplicationMaster's message cut.
>
> SparkSQL Codegen error message show below:
> 18/08/18 08:34:54 ERROR codegen.CodeGenerator: failed to compile: org.codehaus.janino.JaninoRuntimeException: Constant pool has grown past JVM limit of 0xFFFF
> /* 001 */ public java.lang.Object generate(Object[] references)
> { /* 002 */ return new SpecificSafeProjection(references); /* 003 */ }
> /* 004 */
> /* 005 */ class SpecificSafeProjection extends org.apache.spark.sql.catalyst.expressions.codegen.BaseProjection {
> ..........
> about 2 million lines.
> ..........
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org