[jira] [Assigned] (SPARK-26660) Add warning logs for large taskBinary size

["Apache Spark (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-26660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-26660:
------------------------------------

    Assignee: Apache Spark

> Add warning logs for large taskBinary size
> ------------------------------------------
>
>                 Key: SPARK-26660
>                 URL: https://issues.apache.org/jira/browse/SPARK-26660
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.1.0, 2.4.0
>         Environment: Spark2.1
>            Reporter: liupengcheng
>            Assignee: Apache Spark
>            Priority: Minor
>         Attachments: screenshot-1.png
>
>
> In our production environment, we encountered a OOM problem in ML application. After carefully investigation, it was found that some ML library may generate large ml model, thus causing executor may not able to deserialize it and result in OOM failures.
> !screenshot-1.png!
> In order to facilitate the debuging of memory problem caused by large taskBinary broadcast, we may should add same warning logs for it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org