RDDs and Immutability

[Deep Pradhan <pr...@gmail.com>]

Hi,
We all know that RDDs are immutable.
There are not enough operations that can achieve anything and everything on
RDDs.
Take for example this:
I want an Array of Bytes filled with zeros which during the program should
change. Some elements of that Array should change to 1.
If I make an RDD with all elements as zero, I won't be able to change the
elements. On the other hand, if I declare as Array then so much memory will
be consumed.
Please clarify this to me.

Thank You

Re: RDDs and Immutability

[Nicholas Chammas <ni...@gmail.com>]

Have you tried using RDD.map() to transform some of the RDD elements from 0
to 1? Why doesn’t that work? That’s how you change data in Spark, by
defining a new RDD that’s a transformation of an old one.
​

On Sat, Sep 13, 2014 at 5:39 AM, Deep Pradhan <pr...@gmail.com>
wrote:

> Hi,
> We all know that RDDs are immutable.
> There are not enough operations that can achieve anything and everything
> on RDDs.
> Take for example this:
> I want an Array of Bytes filled with zeros which during the program should
> change. Some elements of that Array should change to 1.
> If I make an RDD with all elements as zero, I won't be able to change the
> elements. On the other hand, if I declare as Array then so much memory will
> be consumed.
> Please clarify this to me.
>
> Thank You
>