IgniteContext for multiple IgniteRDD's

[mdolgonos <mi...@citi.com>]

I need to use multiple IgniteRDD's in my project. As far as I found in the
documentation the standard way of using IgniteRDD is by creating
IgniteContext and CacheConfiguration like this:

		val igContext1 = new IgniteContext[Int, List[MyObject1]](sc,
"default-config.xml")
		val igCacheCfg1 = new CacheConfiguration[Int, List[MyObject1]]("mycache1")			
		val cacheIgRDD1 = igContext.fromCache(igCacheCfg1)
		cacheIgRDD1.savePairs(myRDD1, true)
where [Int, List[MyObject1]] is the structure of the first RDD - RDD[(Int,
List[MyObject1])]

However the structure of the second RDD can be different - RDD[(Int,
List[MyObject2])].
How can I reuse the same context and configuration for the second RDD
instead of repeating the same code again like:

		val igContext2 = new IgniteContext[Int, List[MyObject2]](sc,
"default-config.xml")
		val igCacheCfg2 = new CacheConfiguration[Int, List[MyObject2]]("mycache2")			
		val cacheIgRDD2 = igContext.fromCache(igCacheCfg2)
		cacheIgRDD2.savePairs(myRDD2, true)




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteContext-for-multiple-IgniteRDD-s-tp4997.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteContext for multiple IgniteRDD's

[vkulichenko <va...@gmail.com>]

It's expensive only for the first time, when the Ignite client is started.
All other contexts will reuse this client if created with the same config.
So this is just a minor API usability issue, which forces you to create
different instances of IgniteContext just to create RDDs of different types.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteContext-for-multiple-IgniteRDD-s-tp4997p5011.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteContext for multiple IgniteRDD's

[mdolgonos <mi...@citi.com>]

Creating context is expensive. Does that mean that the code will have to
incur this expense for every different RDD ?



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteContext-for-multiple-IgniteRDD-s-tp4997p5002.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: IgniteContext for multiple IgniteRDD's

[vkulichenko <va...@gmail.com>]

Hi,

This is the known issue [1]. However, you can simply create two separate
Ignite contexts, the actual Ignite client will be reused between them, but
you will be able to create RDDs of different types.

Let me know if it helps.

[1] https://issues.apache.org/jira/browse/IGNITE-2929

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IgniteContext-for-multiple-IgniteRDD-s-tp4997p5001.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.