how to store share the schema of RDD?

[baozipu <jo...@gmail.com>]

Hello, I am currently using the Shared RDD function.

I am able to write and read data from IgniteCache.

However, besides the data, I also want to store/share the schema of the RDD
to the IgniteCache, so that other spark application can create dataframe
from that, by calling

val keyValueRDD: IgniteRDD[Long, Row] = igniteContext.fromCache[Long,
Row]("trainDF")
val myRDD = keyValueRDD.map(x=>x._2) // just get the Row out of the RDD
val myDF = ss.createDataFrame(myRDD, schema)

I want the "schema" also be shared through IgniteCache, is it possible?

Lei



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/how-to-store-share-the-schema-of-RDD-tp13544.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: how to store share the schema of RDD?

[vkulichenko <va...@gmail.com>]

Hi Lei,

Currently you can create a data frame only based on result set of a query,
and this data frame will be local. We don't have support for data frames
yet. And in case you're not using SQL, then Ignite is actually schema less.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/how-to-store-share-the-schema-of-RDD-tp13544p14516.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.