Distributed callables: passing data and getting back results

[kmandalas <Ky...@IRIworldwide.com>]

Dear Igniters,

I have the following question related to Compute Grid and having large
number of callables to be executed in the cluster for fast computation and
results retrieval:

- If I want to pass data to callables, for example Lists of objects, let's
say Collections from 2000 to 6000 objects average, with average list item
size some ~3KBs) what is the best way to do this: to pass them as argument
in callables or put them in the distributed cache for the callables to pick
them up from there? I note here that these data are read-only and with rare
updates, so I guess REPLICATED cache mode is suggested. The callables
inside, read these, perform analytical calculations and then each callable
returns a new list of result objects. So the same question applies to the
result list objects: do I return them at each callable or do I put them in
the distributed cache and I notify e.g. a service to collect them for
further usage? 

Thank you.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-callables-passing-data-and-getting-back-results-tp11224.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Distributed callables: passing data and getting back results

[dkarachentsev <dk...@gridgain.com>]

Hi,

Yes, you're right. You need to start Spring application and Ignite after
that, ignite.sh is unable to start spring context.

Thanks!

-Dmitry.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-callables-passing-data-and-getting-back-results-tp11224p11382.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Distributed callables: passing data and getting back results

[kmandalas <Ky...@IRIworldwide.com>]

Thanks Dmitry.

Based on the scenarios I have to deal with (for the time being at least),
using Ignite distributed cache should ne avoided.

However the issue now is that my callables will have to use Spring beans
(services and DAOs) in order to retrieve on their own the data that they
have to process when run() is executed. In order to do this, I read that I
must define Spring Application Context as *transient* in my Callable class
which extends IgniteCallable. Then, inside run() I can get local references
of the Springs beans I need and use them normally. 

The thing is that if I go this way, I can no longer simply start nodes by
command line script (ignite.sh / ignite.bat). I will either have to start
Spring as web application deployed in e.g. Tomcat at each node using
IgniteSpringBean or to create a class with Main method which will load the
application context and launch it with *java -jar*. Am I correct or is there
a more flexible way? I not here that my main web application which is the
one that triggers the distributed computations is based on Spring 5.

Thank you.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-callables-passing-data-and-getting-back-results-tp11224p11344.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Distributed callables: passing data and getting back results

[dkarachentsev <dk...@gridgain.com>]

Hi,

If your tasks use the same data, then REPLICATED cache will be the best
decision. But if you need for every compute task put new data to cache, it
will slowdown processing, because entries must be distributed, copied and
synchronized.

The same rule is applicable for compute results: if you need to reuse them,
then add to cache.

-Dmitry.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Distributed-callables-passing-data-and-getting-back-results-tp11224p11312.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.