You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by RodrigoB <ro...@aspect.com> on 2014/09/23 13:37:08 UTC

Re: broadcast variable get cleaned by ContextCleaner unexpectedly ?

Could you be using by any chance the getOrCreate for the StreamingContext
creation?

I've seen this happen when I tried to first create the Spark context, then
create the broadcast variables, and then recreate the StreamingContext from
the checkpoint directory. So the worker process cannot find the broadcast
variables that were used in the saved StreamingContext.

tnks,
Rod



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/broadcast-variable-get-cleaned-by-ContextCleaner-unexpectedly-tp10347p14884.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org