You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Jonathan Coveney <jc...@gmail.com> on 2015/07/29 01:23:45 UTC

broadcast variable question

i am running in coarse grained mode, let's say with 8 cores per executor.

If I use a broadcast variable, will all of the tasks in that executor share
the same value? Or will each task broadcast its own value ie in this case,
would there be one value in the executor shared by the 8 tasks, or would
there be eight copies, 1 per task?

Re: broadcast variable question

Posted by Jonathan Coveney <jc...@gmail.com>.

That's great! Thanks

El martes, 28 de julio de 2015, Ted Yu <yu...@gmail.com> escribió:

> If I understand correctly, there would be one value in the executor.
>
> Cheers
>
> On Tue, Jul 28, 2015 at 4:23 PM, Jonathan Coveney <jcoveney@gmail.com
> <javascript:_e(%7B%7D,'cvml','jcoveney@gmail.com');>> wrote:
>
>> i am running in coarse grained mode, let's say with 8 cores per executor.
>>
>> If I use a broadcast variable, will all of the tasks in that executor
>> share the same value? Or will each task broadcast its own value ie in this
>> case, would there be one value in the executor shared by the 8 tasks, or
>> would there be eight copies, 1 per task?
>>
>
>

Re: broadcast variable question

Posted by Ted Yu <yu...@gmail.com>.

If I understand correctly, there would be one value in the executor.

Cheers

On Tue, Jul 28, 2015 at 4:23 PM, Jonathan Coveney <jc...@gmail.com>
wrote:

> i am running in coarse grained mode, let's say with 8 cores per executor.
>
> If I use a broadcast variable, will all of the tasks in that executor
> share the same value? Or will each task broadcast its own value ie in this
> case, would there be one value in the executor shared by the 8 tasks, or
> would there be eight copies, 1 per task?
>