Replicating RDD elements

[David Thomas <dt...@gmail.com>]

How can we replicate RDD elements? Say I have 1 element and 100 nodes in
the cluster. I need to replicate this one item on all the nodes i.e.
effectively create an RDD of 100 elements.

Re: Replicating RDD elements

[David Thomas <dt...@gmail.com>]

That helps! Thank you.


On Fri, Mar 28, 2014 at 12:36 AM, Sonal Goyal <so...@gmail.com> wrote:

> Hi David,
>
> I am sorry but your question is not clear to me. Are you talking about
> taking some value and sharing it across your cluster so that it is present
> on all the nodes? You can look at Spark's broadcasting in that case. On the
> other hand, if you want to take one item and create an RDD of 100 or some
> other number of items, you could do a flatMap. Does that help?
>
> Best Regards,
> Sonal
> Nube Technologies <http://www.nubetech.co>
>
>  <http://in.linkedin.com/in/sonalgoyal>
>
>
>
>
> On Fri, Mar 28, 2014 at 9:24 AM, David Thomas <dt...@gmail.com> wrote:
>
>> How can we replicate RDD elements? Say I have 1 element and 100 nodes in
>> the cluster. I need to replicate this one item on all the nodes i.e.
>> effectively create an RDD of 100 elements.
>>
>
>

Re: Replicating RDD elements

[Sonal Goyal <so...@gmail.com>]

Hi David,

I am sorry but your question is not clear to me. Are you talking about
taking some value and sharing it across your cluster so that it is present
on all the nodes? You can look at Spark's broadcasting in that case. On the
other hand, if you want to take one item and create an RDD of 100 or some
other number of items, you could do a flatMap. Does that help?

Best Regards,
Sonal
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>




On Fri, Mar 28, 2014 at 9:24 AM, David Thomas <dt...@gmail.com> wrote:

> How can we replicate RDD elements? Say I have 1 element and 100 nodes in
> the cluster. I need to replicate this one item on all the nodes i.e.
> effectively create an RDD of 100 elements.
>