You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "uncleGen (JIRA)" <ji...@apache.org> on 2016/02/04 14:15:39 UTC
[jira] [Created] (SPARK-13192) Some memory leak problem?
uncleGen created SPARK-13192:
--------------------------------
Summary: Some memory leak problem?
Key: SPARK-13192
URL: https://issues.apache.org/jira/browse/SPARK-13192
Project: Spark
Issue Type: Bug
Components: Streaming
Affects Versions: 1.6.0
Reporter: uncleGen
In my Spark Streaming job, the executor container was killed by Yarn. The nodemanager log showed that container memory was constantly increaming, and exceeded the container memory limit. Specifically, only the container which contains receiver occurred this problem. Is there any memory leak issue here? Or is there some known memory leak issue?
My code snippet:
dStream.foreachRDD(new Function<JavaRDD<T>, Void>() {
@Override
public Void call(JavaRDD<T> rdd) throws Exception {
T card = rdd.first();
String time = DateUtil.formatToDay(DateUtil.strLong(card.getTime()));
System.out.println("time:"+time+", count"+rdd.count)
return null;
}
});
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org