You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by LINChen <m2...@outlook.com> on 2016/01/16 16:52:00 UTC

How to apply mapPartitionsWithIndex to an emptyRDD?

Hi all,I have some data on the driver side. Then I will broadcast the data to all workers side to ensure each worker has same data. Due to there is no RDD in the memory, I don't know how to make workers to start tasks to do some transformation based on the data. I have try to write code like this: emptyRDD.mapPartitionsWithIndex(...), but it doesn't work. So it seems it is not suitable to use emptyRDD to start tasks on the worker side.It is so appreciated if you can give me some suggestions.Thanks.