Pregel Question

[Saliya Ekanayake <es...@gmail.com>]

Hi,

I've created a graph with vertex data of the form (Int, Array[Int]). I am
using the pregel operator to update values of the array for each vertex.

So my vprog has the following signature. Note the message is a map of
arrays from neighbors

def vprog(vertexId: VertexId, value: (Int, Array[Int]), message:
scala.collection.mutable.HashMap[Int, Array[Int]])

The full program is attached here. The expectation is vprog() to update the
array and then sendMsg() to send the updates to the neighbors.

However, this requires cloning the vertex every time in the vprog()
function. If I don't clone Spark would send the same array that it got
after the initial call.

Is there a way to turn off this caching effect?

Thank you,
Saliya


-- 
Saliya Ekanayake, Ph.D
Applied Computer Scientist
Network Dynamics and Simulation Science Laboratory (NDSSL)
Virginia Tech, Blacksburg

Re: Pregel Question

[Saliya Ekanayake <es...@gmail.com>]

Just realized the attached file has text formatting wrong. The github link
to the file is
https://github.com/esaliya/graphxprimer/blob/master/src/main/scala-2.10/org/saliya/graphxprimer/PregelExample2.scala

On Tue, Nov 22, 2016 at 3:08 PM, Saliya Ekanayake <es...@gmail.com> wrote:

> Hi,
>
> I've created a graph with vertex data of the form (Int, Array[Int]). I am
> using the pregel operator to update values of the array for each vertex.
>
> So my vprog has the following signature. Note the message is a map of
> arrays from neighbors
>
> def vprog(vertexId: VertexId, value: (Int, Array[Int]), message:
> scala.collection.mutable.HashMap[Int, Array[Int]])
>
> The full program is attached here. The expectation is vprog() to update
> the array and then sendMsg() to send the updates to the neighbors.
>
> However, this requires cloning the vertex every time in the vprog()
> function. If I don't clone Spark would send the same array that it got
> after the initial call.
>
> Is there a way to turn off this caching effect?
>
> Thank you,
> Saliya
>
>
> --
> Saliya Ekanayake, Ph.D
> Applied Computer Scientist
> Network Dynamics and Simulation Science Laboratory (NDSSL)
> Virginia Tech, Blacksburg
>
>


-- 
Saliya Ekanayake, Ph.D
Applied Computer Scientist
Network Dynamics and Simulation Science Laboratory (NDSSL)
Virginia Tech, Blacksburg