How to generate seeded random numbers in GraphX Pregel API vertex procedure?

[Roman Pastukhov <me...@gmail.com>]

As far as I understand, best way to generate seeded random numbers in Spark
is to use mapPartititons with a seeded Random instance for each partition.
But graph.pregel in GraphX does not have anything similar to mapPartitions.

Can something like this be done in GraphX Pregel API?

Re: How to generate seeded random numbers in GraphX Pregel API vertex procedure?

[Takeshi Yamamuro <li...@gmail.com>]

Hi,

yea, we have no simple way to do that in GraphX because the GraphX class
has both vertex and edge rdds and
we cannot simply implement mapPartitions there to keep vertex/edge
semantics inside.
Another idea is to generate edge files by using RDD#mapPartitions and write
them into HDFS, and then
you use GraphLoader#edgeListFile to load them.

// maropu

On Thu, Jun 2, 2016 at 11:20 PM, Roman Pastukhov <me...@gmail.com>
wrote:

> As far as I understand, best way to generate seeded random numbers in
> Spark is to use mapPartititons with a seeded Random instance for each
> partition.
> But graph.pregel in GraphX does not have anything similar to mapPartitions.
>
> Can something like this be done in GraphX Pregel API?
>



-- 
---
Takeshi Yamamuro