How to validate that what was retrieved from Ignite cache is not NULL or empty?

[mdolgonos <mi...@citi.com>]

I'm trying to store a Spark RDD in the Ignite cache. My distilled code is
something like this:

val igCfg = new MyIgniteConfiguration().setPeerClassLoadingEnabled(true)
val igContext = new IgniteContext[(Int,Int), List[Double]](sc, ()=>igCfg)
val igCacheCfg = new CacheConfiguration[(Int,Int),
List[Double]]("My_Ignite_Cache"). 
	setAffinity(new RendezvousAffinityFunction(false, 32))	// 32 partitions
instead of 1024
		
val cacheIgRDD = igContext.fromCache(igCacheCfg)
......

Then I create a Spark partRDD and trying to store it in cacheIgRDD:
cacheIgRDD.savePairs(partRDD, true)

I assume, if I run the same program again, that cacheIgRDD should contain my
previously stored RDD. Is this correct? If so, how can I make sure that the
next time I run the line
val cacheIgRDD = igContext.fromCache(igCacheCfg)
the resulted cacheIgRDD  contains what I expect and not 'Nothing' or an
empty RDD? It looks like I'm getting cacheIgRDD=IgniteRDD[0]

Thank you






--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-validate-that-what-was-retrieved-from-Ignite-cache-is-not-NULL-or-empty-tp4668.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: How to validate that what was retrieved from Ignite cache is not NULL or empty?

[vkulichenko <va...@gmail.com>]

Hi,

Your expectation is correct. The second IgniteRDD will connect to the same
Ignite cache that will already have all the data. This was you can share the
data between applications.

Zero in 'IgniteRDD[0]' string is just an ID within SparkContext, so it
doesn't tell anything about the cached data. I would recommend you to go
through the example described in [1], it demonstrates exactly the same use
case that you're trying to implement.

[1]
https://apacheignite-fs.readme.io/docs/testing-integration-with-spark-shell

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/How-to-validate-that-what-was-retrieved-from-Ignite-cache-is-not-NULL-or-empty-tp4668p4678.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.