Re: Read text file and convert it to Spark RDD from that create Ignite RDD cache

[vkulichenko <va...@gmail.com>]

Hi Murali,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscribe@ignite.apache.org and follow simple instructions in
the reply.


muralisajja wrote
> I want to know how read the csv file and create spark RDD.
> Once spark RDD create again it should translate to Ignite RDD for the sql
> quiery.
> 
> sql Quiery will be like group by operation and where conditions,sums and
> between.
> 
> Can you please post the code for the above scenario.
> 
> I would be great full for your help.Requesting all of you plz help us.
> 
> This excercide i want to do on 100MN data.

Spark RDD should be created using native Spark capabilities, and then you
can create IgniteRDD based on it and use savePairs method to migrate data to
Ignite (you will have to convert each line to a key-value pair). Refer to
[1] for an example.

Let me know if anything doesn't work for you and/or if you have more
specific questions.

[1]
https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Read-text-file-and-convert-it-to-Spark-RDD-from-that-create-Ignite-RDD-cache-tp9454p9497.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.