You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by abi <an...@gmail.com> on 2016/05/13 22:53:50 UTC
broadcast variable not picked up
def kernel(arg):
input = broadcast_var.value + 1
#some processing with input
def foo():
....
....
broadcast_var = sc.broadcast(var)
rdd.foreach(kernel)
def main():
#something
In this code , I get the following error:
NameError: global name 'broadcast_var ' is not defined
Any ideas on how to fix it ?
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/broadcast-variable-not-picked-up-tp26955.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: broadcast variable not picked up
Posted by Davies Liu <da...@databricks.com>.
broadcast_var is only defined in foo(), I think you should have `global` for it.
def foo():
global broadcast_var
broadcast_var = sc.broadcast(var)
On Fri, May 13, 2016 at 3:53 PM, abi <an...@gmail.com> wrote:
> def kernel(arg):
> input = broadcast_var.value + 1
> #some processing with input
>
> def foo():
> ....
> ....
> broadcast_var = sc.broadcast(var)
> rdd.foreach(kernel)
>
>
> def main():
> #something
>
>
> In this code , I get the following error:
> NameError: global name 'broadcast_var ' is not defined
>
>
> Any ideas on how to fix it ?
>
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/broadcast-variable-not-picked-up-tp26955.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org