You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by sujeet jog <su...@gmail.com> on 2016/09/20 13:12:21 UTC

Convert RDD to JSON Rdd and append more information

Hi,

I have a Rdd of n rows,  i want to transform this to a Json RDD, and also
add some more information , any idea how to accomplish this .


ex : -

i have rdd with n rows with data like below ,  ,

 16.9527493170273,20.1989561393151,15.7065424947394
 17.9527493170273,21.1989561393151,15.7065424947394
 18.9527493170273,22.1989561393151,15.7065424947394


would like to add few rows highlited to the beginning of RDD like below, is
there a way to
do this and transform it to JSON,  the reason being i intend to push this
as input  to some application via pipeRDD for some processing, and want to
enforce a JSON structure on the input.

*{*
*TimeSeriesID : 1234*
*NumOfInputSamples : 1008 *
*Request Type : Fcast*
 16.9527493170273,20.1989561393151,15.7065424947394
 17.9527493170273,21.1989561393151,15.7065424947394
 18.9527493170273,22.1989561393151,15.7065424947394
}


Thanks,
Sujeet

Re: Convert RDD to JSON Rdd and append more information

Posted by Deepak Sharma <de...@gmail.com>.

Enrich the RDDs first with more information and then map it to some case
class , if you are using scala.
You can then use play api's
(play.api.libs.json.Writes/play.api.libs.json.Json) classes to convert the
mapped case class to json.

Thanks
Deepak

On Tue, Sep 20, 2016 at 6:42 PM, sujeet jog <su...@gmail.com> wrote:

> Hi,
>
> I have a Rdd of n rows,  i want to transform this to a Json RDD, and also
> add some more information , any idea how to accomplish this .
>
>
> ex : -
>
> i have rdd with n rows with data like below ,  ,
>
>  16.9527493170273,20.1989561393151,15.7065424947394
>  17.9527493170273,21.1989561393151,15.7065424947394
>  18.9527493170273,22.1989561393151,15.7065424947394
>
>
> would like to add few rows highlited to the beginning of RDD like below,
> is there a way to
> do this and transform it to JSON,  the reason being i intend to push this
> as input  to some application via pipeRDD for some processing, and want to
> enforce a JSON structure on the input.
>
> *{*
> *TimeSeriesID : 1234*
> *NumOfInputSamples : 1008 *
> *Request Type : Fcast*
>  16.9527493170273,20.1989561393151,15.7065424947394
>  17.9527493170273,21.1989561393151,15.7065424947394
>  18.9527493170273,22.1989561393151,15.7065424947394
> }
>
>
> Thanks,
> Sujeet
>



-- 
Thanks
Deepak
www.bigdatabig.com
www.keosha.net