You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Vladimir Ozerov <vo...@gridgain.com> on 2015/10/09 16:22:36 UTC
Which thread is used to run IgniteFuture continuations?
Igniters,
We are missing an ability to specify which thread should run continuation
routine when future is completed.
Currently we either run routine in the callee thread or in completion
thread (usually system pool thread). It means user cannot have any blocking
or cache operations inside the continuation. The code below could surprise
user with either a deadlock or system pool starvation:
final CountDownLatch latch = new CountDownLatch();
Cache cache = ignite.cache().withAsync();
cache.invoke(...);
cache.future().listen(() => {
latch.await();
/** Do something useful. */
});
/** Do something else. */
latch.countDown();
Java 8 and Hazelcast already support custom thread pools for continuations.
E.g.:
Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
Executor executor);
Looks like we should allow users to specify optional thread pool in futures
likewise.
Thoughts?
Vladimir.
Re: Which thread is used to run IgniteFuture continuations?
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Vova,
Are we notifying listeners that a message has been sent? I thought that you
were talking about reply notifications.
D.
On Fri, Oct 16, 2015 at 2:13 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> Dima,
>
> If user call IgniteMessaging.send() in the thread T1, then local message
> listeners on this topic will be notified in the same thread before
> returning from send() method. This our pseudo-code:
>
> IgniteMessaging.send(Object topic, Object msg) {
> ...
> sendToRemoteNodes(topic, msg);
> ...
> for (Listener lsnr : localTopicListeners)
> lsnr.notify(msg);
> }
>
> If user produces a burst of messages in a single thread, he will be very
> surprised that remote node process them concurrently and utilize all CPUs,
> while local node use only 1 core for this.
>
> Vladimir.
>
> On Fri, Oct 16, 2015 at 11:01 AM, Dmitriy Setrakyan <dsetrakyan@apache.org
> >
> wrote:
>
> > On Thu, Oct 15, 2015 at 11:00 PM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > Sorry, not very clear. "same" means the same thread that sent a
> message.
> > > This way if we have a listener and a single thread generating messages
> > > locally, only one CPU core will be utilizied.
> > >
> >
> > Still confused. How can the thread that send a message be notified in a
> > listener about anything? Are you talking about synchronous
> > request-response? In this case it is probably done on purpose.
> >
> > I am still not sure what the problem is.
> >
> >
> > >
> > > On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > > wrote:
> > >
> > > > On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Folks,
> > > > >
> > > > > It looks like the problem is deeper and is not limited to futures.
> It
> > > > > relates to any user code. For example:
> > > > >
> > > > > 1) Register either local or remote message listener:
> > > > > Ignite.message().localListen("myTopic", myLsnr);
> > > > >
> > > > > 2) Send a message to the topic from the same node:
> > > > > Ignite.message().send("myTopic", "Hello world!");
> > > > >
> > > > > As a result, listener will be invoked synchronously from the same
> > > thread.
> > > > > It means we cannot utilize resources efficiently in case of local
> > > > message.
> > > > >
> > > >
> > > > I am a bit confused. Which thread is the "same" thread? Do you mean
> the
> > > > thread that does the listener notification? In that case, I can say
> > that
> > > I
> > > > have performed various benchmarks in the past, and this is the most
> > > > performant way, assuming that the listener logic can execute
> relatively
> > > > fast and does not block the calling thread.
> > > >
> > >
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Dima,
If user call IgniteMessaging.send() in the thread T1, then local message
listeners on this topic will be notified in the same thread before
returning from send() method. This our pseudo-code:
IgniteMessaging.send(Object topic, Object msg) {
...
sendToRemoteNodes(topic, msg);
...
for (Listener lsnr : localTopicListeners)
lsnr.notify(msg);
}
If user produces a burst of messages in a single thread, he will be very
surprised that remote node process them concurrently and utilize all CPUs,
while local node use only 1 core for this.
Vladimir.
On Fri, Oct 16, 2015 at 11:01 AM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> On Thu, Oct 15, 2015 at 11:00 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Sorry, not very clear. "same" means the same thread that sent a message.
> > This way if we have a listener and a single thread generating messages
> > locally, only one CPU core will be utilizied.
> >
>
> Still confused. How can the thread that send a message be notified in a
> listener about anything? Are you talking about synchronous
> request-response? In this case it is probably done on purpose.
>
> I am still not sure what the problem is.
>
>
> >
> > On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <vozerov@gridgain.com
> >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > It looks like the problem is deeper and is not limited to futures. It
> > > > relates to any user code. For example:
> > > >
> > > > 1) Register either local or remote message listener:
> > > > Ignite.message().localListen("myTopic", myLsnr);
> > > >
> > > > 2) Send a message to the topic from the same node:
> > > > Ignite.message().send("myTopic", "Hello world!");
> > > >
> > > > As a result, listener will be invoked synchronously from the same
> > thread.
> > > > It means we cannot utilize resources efficiently in case of local
> > > message.
> > > >
> > >
> > > I am a bit confused. Which thread is the "same" thread? Do you mean the
> > > thread that does the listener notification? In that case, I can say
> that
> > I
> > > have performed various benchmarks in the past, and this is the most
> > > performant way, assuming that the listener logic can execute relatively
> > > fast and does not block the calling thread.
> > >
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Thu, Oct 15, 2015 at 11:00 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> Sorry, not very clear. "same" means the same thread that sent a message.
> This way if we have a listener and a single thread generating messages
> locally, only one CPU core will be utilizied.
>
Still confused. How can the thread that send a message be notified in a
listener about anything? Are you talking about synchronous
request-response? In this case it is probably done on purpose.
I am still not sure what the problem is.
>
> On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > Folks,
> > >
> > > It looks like the problem is deeper and is not limited to futures. It
> > > relates to any user code. For example:
> > >
> > > 1) Register either local or remote message listener:
> > > Ignite.message().localListen("myTopic", myLsnr);
> > >
> > > 2) Send a message to the topic from the same node:
> > > Ignite.message().send("myTopic", "Hello world!");
> > >
> > > As a result, listener will be invoked synchronously from the same
> thread.
> > > It means we cannot utilize resources efficiently in case of local
> > message.
> > >
> >
> > I am a bit confused. Which thread is the "same" thread? Do you mean the
> > thread that does the listener notification? In that case, I can say that
> I
> > have performed various benchmarks in the past, and this is the most
> > performant way, assuming that the listener logic can execute relatively
> > fast and does not block the calling thread.
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Sorry, not very clear. "same" means the same thread that sent a message.
This way if we have a listener and a single thread generating messages
locally, only one CPU core will be utilizied.
On Fri, Oct 16, 2015 at 3:49 AM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Folks,
> >
> > It looks like the problem is deeper and is not limited to futures. It
> > relates to any user code. For example:
> >
> > 1) Register either local or remote message listener:
> > Ignite.message().localListen("myTopic", myLsnr);
> >
> > 2) Send a message to the topic from the same node:
> > Ignite.message().send("myTopic", "Hello world!");
> >
> > As a result, listener will be invoked synchronously from the same thread.
> > It means we cannot utilize resources efficiently in case of local
> message.
> >
>
> I am a bit confused. Which thread is the "same" thread? Do you mean the
> thread that does the listener notification? In that case, I can say that I
> have performed various benchmarks in the past, and this is the most
> performant way, assuming that the listener logic can execute relatively
> fast and does not block the calling thread.
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Thu, Oct 15, 2015 at 9:59 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> Folks,
>
> It looks like the problem is deeper and is not limited to futures. It
> relates to any user code. For example:
>
> 1) Register either local or remote message listener:
> Ignite.message().localListen("myTopic", myLsnr);
>
> 2) Send a message to the topic from the same node:
> Ignite.message().send("myTopic", "Hello world!");
>
> As a result, listener will be invoked synchronously from the same thread.
> It means we cannot utilize resources efficiently in case of local message.
>
I am a bit confused. Which thread is the "same" thread? Do you mean the
thread that does the listener notification? In that case, I can say that I
have performed various benchmarks in the past, and this is the most
performant way, assuming that the listener logic can execute relatively
fast and does not block the calling thread.
Re: Which thread is used to run IgniteFuture continuations?
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Folks,
It looks like the problem is deeper and is not limited to futures. It
relates to any user code. For example:
1) Register either local or remote message listener:
Ignite.message().localListen("myTopic", myLsnr);
2) Send a message to the topic from the same node:
Ignite.message().send("myTopic", "Hello world!");
As a result, listener will be invoked synchronously from the same thread.
It means we cannot utilize resources efficiently in case of local message.
On Mon, Oct 12, 2015 at 7:51 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> On Mon, Oct 12, 2015 at 9:39 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > It would be cool, but CompletableFuture is available since Java 8.
> >
>
> Well, we could add some methods from CompletableFuture to IgniteFuture
> then. I will take a look and suggest a few.
>
>
> >
> > On Mon, Oct 12, 2015 at 7:25 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > Would it be possible for us to support CompletableFuture from Java
> > instead
> > > of adding one-off methods?
> > >
> > > On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <vozerov@gridgain.com
> >
> > > wrote:
> > >
> > > > Yakov, two points there:
> > > >
> > > > 1) This is a matter of convenience. Current mainstream solution for
> > > similar
> > > > tasks is Java's CompletableFuture. Here is what it offers:
> > > >
> > > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
> > > >
> > > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
> > > >
> > > > 2) Probably we should go event further and do not execute
> continuations
> > > in
> > > > user thread or our system threads by default because this is error
> > prone
> > > > and is subject to deadlocks and unpredictable slowdowns in normal
> > > > compute/cache operations. By default we can either delegate to FJP,
> > > provide
> > > > configurable pool for continuations, or use Ignite public pool.
> > > >
> > > > On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <yz...@apache.org>
> > > > wrote:
> > > >
> > > > > Not sure if I get the point. You can do exactly the same inside
> your
> > > > > listener.
> > > > >
> > > > > final Executor executor = ...;
> > > > >
> > > > > // Bla-bla
> > > > >
> > > > > cache.future().listen(() => {
> > > > > executor.execute(new Runnable() {
> > > > > latch.await();
> > > > > /** Do something useful. */
> > > > > }
> > > > > });
> > > > >
> > > > > --Yakov
> > > > >
> > > > > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > We are missing an ability to specify which thread should run
> > > > continuation
> > > > > > routine when future is completed.
> > > > > >
> > > > > > Currently we either run routine in the callee thread or in
> > completion
> > > > > > thread (usually system pool thread). It means user cannot have
> any
> > > > > blocking
> > > > > > or cache operations inside the continuation. The code below could
> > > > > surprise
> > > > > > user with either a deadlock or system pool starvation:
> > > > > >
> > > > > > final CountDownLatch latch = new CountDownLatch();
> > > > > >
> > > > > > Cache cache = ignite.cache().withAsync();
> > > > > > cache.invoke(...);
> > > > > >
> > > > > > cache.future().listen(() => {
> > > > > > latch.await();
> > > > > > /** Do something useful. */
> > > > > > });
> > > > > >
> > > > > > /** Do something else. */
> > > > > > latch.countDown();
> > > > > >
> > > > > > Java 8 and Hazelcast already support custom thread pools for
> > > > > continuations.
> > > > > > E.g.:
> > > > > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V>
> > > callback,
> > > > > > Executor executor);
> > > > > >
> > > > > > Looks like we should allow users to specify optional thread pool
> in
> > > > > futures
> > > > > > likewise.
> > > > > >
> > > > > > Thoughts?
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Mon, Oct 12, 2015 at 9:39 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> It would be cool, but CompletableFuture is available since Java 8.
>
Well, we could add some methods from CompletableFuture to IgniteFuture
then. I will take a look and suggest a few.
>
> On Mon, Oct 12, 2015 at 7:25 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > Would it be possible for us to support CompletableFuture from Java
> instead
> > of adding one-off methods?
> >
> > On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > Yakov, two points there:
> > >
> > > 1) This is a matter of convenience. Current mainstream solution for
> > similar
> > > tasks is Java's CompletableFuture. Here is what it offers:
> > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
> > >
> > >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
> > >
> > > 2) Probably we should go event further and do not execute continuations
> > in
> > > user thread or our system threads by default because this is error
> prone
> > > and is subject to deadlocks and unpredictable slowdowns in normal
> > > compute/cache operations. By default we can either delegate to FJP,
> > provide
> > > configurable pool for continuations, or use Ignite public pool.
> > >
> > > On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <yz...@apache.org>
> > > wrote:
> > >
> > > > Not sure if I get the point. You can do exactly the same inside your
> > > > listener.
> > > >
> > > > final Executor executor = ...;
> > > >
> > > > // Bla-bla
> > > >
> > > > cache.future().listen(() => {
> > > > executor.execute(new Runnable() {
> > > > latch.await();
> > > > /** Do something useful. */
> > > > }
> > > > });
> > > >
> > > > --Yakov
> > > >
> > > > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> > > >
> > > > > Igniters,
> > > > >
> > > > > We are missing an ability to specify which thread should run
> > > continuation
> > > > > routine when future is completed.
> > > > >
> > > > > Currently we either run routine in the callee thread or in
> completion
> > > > > thread (usually system pool thread). It means user cannot have any
> > > > blocking
> > > > > or cache operations inside the continuation. The code below could
> > > > surprise
> > > > > user with either a deadlock or system pool starvation:
> > > > >
> > > > > final CountDownLatch latch = new CountDownLatch();
> > > > >
> > > > > Cache cache = ignite.cache().withAsync();
> > > > > cache.invoke(...);
> > > > >
> > > > > cache.future().listen(() => {
> > > > > latch.await();
> > > > > /** Do something useful. */
> > > > > });
> > > > >
> > > > > /** Do something else. */
> > > > > latch.countDown();
> > > > >
> > > > > Java 8 and Hazelcast already support custom thread pools for
> > > > continuations.
> > > > > E.g.:
> > > > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V>
> > callback,
> > > > > Executor executor);
> > > > >
> > > > > Looks like we should allow users to specify optional thread pool in
> > > > futures
> > > > > likewise.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Vladimir.
> > > > >
> > > >
> > >
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Vladimir Ozerov <vo...@gridgain.com>.
It would be cool, but CompletableFuture is available since Java 8.
On Mon, Oct 12, 2015 at 7:25 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> Would it be possible for us to support CompletableFuture from Java instead
> of adding one-off methods?
>
> On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Yakov, two points there:
> >
> > 1) This is a matter of convenience. Current mainstream solution for
> similar
> > tasks is Java's CompletableFuture. Here is what it offers:
> >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
> >
> >
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
> >
> > 2) Probably we should go event further and do not execute continuations
> in
> > user thread or our system threads by default because this is error prone
> > and is subject to deadlocks and unpredictable slowdowns in normal
> > compute/cache operations. By default we can either delegate to FJP,
> provide
> > configurable pool for continuations, or use Ignite public pool.
> >
> > On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <yz...@apache.org>
> > wrote:
> >
> > > Not sure if I get the point. You can do exactly the same inside your
> > > listener.
> > >
> > > final Executor executor = ...;
> > >
> > > // Bla-bla
> > >
> > > cache.future().listen(() => {
> > > executor.execute(new Runnable() {
> > > latch.await();
> > > /** Do something useful. */
> > > }
> > > });
> > >
> > > --Yakov
> > >
> > > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> > >
> > > > Igniters,
> > > >
> > > > We are missing an ability to specify which thread should run
> > continuation
> > > > routine when future is completed.
> > > >
> > > > Currently we either run routine in the callee thread or in completion
> > > > thread (usually system pool thread). It means user cannot have any
> > > blocking
> > > > or cache operations inside the continuation. The code below could
> > > surprise
> > > > user with either a deadlock or system pool starvation:
> > > >
> > > > final CountDownLatch latch = new CountDownLatch();
> > > >
> > > > Cache cache = ignite.cache().withAsync();
> > > > cache.invoke(...);
> > > >
> > > > cache.future().listen(() => {
> > > > latch.await();
> > > > /** Do something useful. */
> > > > });
> > > >
> > > > /** Do something else. */
> > > > latch.countDown();
> > > >
> > > > Java 8 and Hazelcast already support custom thread pools for
> > > continuations.
> > > > E.g.:
> > > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V>
> callback,
> > > > Executor executor);
> > > >
> > > > Looks like we should allow users to specify optional thread pool in
> > > futures
> > > > likewise.
> > > >
> > > > Thoughts?
> > > >
> > > > Vladimir.
> > > >
> > >
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Would it be possible for us to support CompletableFuture from Java instead
of adding one-off methods?
On Mon, Oct 12, 2015 at 6:53 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> Yakov, two points there:
>
> 1) This is a matter of convenience. Current mainstream solution for similar
> tasks is Java's CompletableFuture. Here is what it offers:
>
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
>
> https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
>
> 2) Probably we should go event further and do not execute continuations in
> user thread or our system threads by default because this is error prone
> and is subject to deadlocks and unpredictable slowdowns in normal
> compute/cache operations. By default we can either delegate to FJP, provide
> configurable pool for continuations, or use Ignite public pool.
>
> On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <yz...@apache.org>
> wrote:
>
> > Not sure if I get the point. You can do exactly the same inside your
> > listener.
> >
> > final Executor executor = ...;
> >
> > // Bla-bla
> >
> > cache.future().listen(() => {
> > executor.execute(new Runnable() {
> > latch.await();
> > /** Do something useful. */
> > }
> > });
> >
> > --Yakov
> >
> > 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> >
> > > Igniters,
> > >
> > > We are missing an ability to specify which thread should run
> continuation
> > > routine when future is completed.
> > >
> > > Currently we either run routine in the callee thread or in completion
> > > thread (usually system pool thread). It means user cannot have any
> > blocking
> > > or cache operations inside the continuation. The code below could
> > surprise
> > > user with either a deadlock or system pool starvation:
> > >
> > > final CountDownLatch latch = new CountDownLatch();
> > >
> > > Cache cache = ignite.cache().withAsync();
> > > cache.invoke(...);
> > >
> > > cache.future().listen(() => {
> > > latch.await();
> > > /** Do something useful. */
> > > });
> > >
> > > /** Do something else. */
> > > latch.countDown();
> > >
> > > Java 8 and Hazelcast already support custom thread pools for
> > continuations.
> > > E.g.:
> > > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
> > > Executor executor);
> > >
> > > Looks like we should allow users to specify optional thread pool in
> > futures
> > > likewise.
> > >
> > > Thoughts?
> > >
> > > Vladimir.
> > >
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Yakov, two points there:
1) This is a matter of convenience. Current mainstream solution for similar
tasks is Java's CompletableFuture. Here is what it offers:
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-
https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html#runAsync-java.lang.Runnable-java.util.concurrent.Executor-
2) Probably we should go event further and do not execute continuations in
user thread or our system threads by default because this is error prone
and is subject to deadlocks and unpredictable slowdowns in normal
compute/cache operations. By default we can either delegate to FJP, provide
configurable pool for continuations, or use Ignite public pool.
On Mon, Oct 12, 2015 at 4:25 PM, Yakov Zhdanov <yz...@apache.org> wrote:
> Not sure if I get the point. You can do exactly the same inside your
> listener.
>
> final Executor executor = ...;
>
> // Bla-bla
>
> cache.future().listen(() => {
> executor.execute(new Runnable() {
> latch.await();
> /** Do something useful. */
> }
> });
>
> --Yakov
>
> 2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
>
> > Igniters,
> >
> > We are missing an ability to specify which thread should run continuation
> > routine when future is completed.
> >
> > Currently we either run routine in the callee thread or in completion
> > thread (usually system pool thread). It means user cannot have any
> blocking
> > or cache operations inside the continuation. The code below could
> surprise
> > user with either a deadlock or system pool starvation:
> >
> > final CountDownLatch latch = new CountDownLatch();
> >
> > Cache cache = ignite.cache().withAsync();
> > cache.invoke(...);
> >
> > cache.future().listen(() => {
> > latch.await();
> > /** Do something useful. */
> > });
> >
> > /** Do something else. */
> > latch.countDown();
> >
> > Java 8 and Hazelcast already support custom thread pools for
> continuations.
> > E.g.:
> > Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
> > Executor executor);
> >
> > Looks like we should allow users to specify optional thread pool in
> futures
> > likewise.
> >
> > Thoughts?
> >
> > Vladimir.
> >
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Yakov Zhdanov <yz...@apache.org>.
Not sure if I get the point. You can do exactly the same inside your
listener.
final Executor executor = ...;
// Bla-bla
cache.future().listen(() => {
executor.execute(new Runnable() {
latch.await();
/** Do something useful. */
}
});
--Yakov
2015-10-09 17:22 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> Igniters,
>
> We are missing an ability to specify which thread should run continuation
> routine when future is completed.
>
> Currently we either run routine in the callee thread or in completion
> thread (usually system pool thread). It means user cannot have any blocking
> or cache operations inside the continuation. The code below could surprise
> user with either a deadlock or system pool starvation:
>
> final CountDownLatch latch = new CountDownLatch();
>
> Cache cache = ignite.cache().withAsync();
> cache.invoke(...);
>
> cache.future().listen(() => {
> latch.await();
> /** Do something useful. */
> });
>
> /** Do something else. */
> latch.countDown();
>
> Java 8 and Hazelcast already support custom thread pools for continuations.
> E.g.:
> Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
> Executor executor);
>
> Looks like we should allow users to specify optional thread pool in futures
> likewise.
>
> Thoughts?
>
> Vladimir.
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Dmitriy Setrakyan <ds...@gridgain.com>.
I will add +1 on this, which together with Andrey's +10, brings the total
to +11 :)
D.
On Fri, Oct 9, 2015 at 7:48 AM, Andrey Kornev <an...@hotmail.com>
wrote:
> Excellent idea! +10
> PS. Got burnt by this a few times already
> _____________________________
> From: Vladimir Ozerov <vo...@gridgain.com>
> Sent: Friday, October 9, 2015 4:22 PM
> Subject: Which thread is used to run IgniteFuture continuations?
> To: <de...@ignite.apache.org>
>
>
> Igniters,
>
> We are missing an ability to specify which thread should run continuation
> routine when future is completed.
>
> Currently we either run routine in the callee thread or in completion
> thread (usually system pool thread). It means user cannot have any
> blocking
> or cache operations inside the continuation. The code below could surprise
> user with either a deadlock or system pool starvation:
>
> final CountDownLatch latch = new CountDownLatch();
>
> Cache cache = ignite.cache().withAsync();
> cache.invoke(...);
>
> cache.future().listen(() => {
> latch.await();
> /** Do something useful. */
> });
>
> /** Do something else. */
> latch.countDown();
>
> Java 8 and Hazelcast already support custom thread pools for
> continuations.
> E.g.:
> Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
> Executor executor);
>
> Looks like we should allow users to specify optional thread pool in
> futures
> likewise.
>
> Thoughts?
>
> Vladimir.
>
Re: Which thread is used to run IgniteFuture continuations?
Posted by Andrey Kornev <an...@hotmail.com>.
Excellent idea! +10
PS. Got burnt by this a few times already
_____________________________
From: Vladimir Ozerov <vo...@gridgain.com>
Sent: Friday, October 9, 2015 4:22 PM
Subject: Which thread is used to run IgniteFuture continuations?
To: <de...@ignite.apache.org>
Igniters,
We are missing an ability to specify which thread should run continuation
routine when future is completed.
Currently we either run routine in the callee thread or in completion
thread (usually system pool thread). It means user cannot have any blocking
or cache operations inside the continuation. The code below could surprise
user with either a deadlock or system pool starvation:
final CountDownLatch latch = new CountDownLatch();
Cache cache = ignite.cache().withAsync();
cache.invoke(...);
cache.future().listen(() => {
latch.await();
/** Do something useful. */
});
/** Do something else. */
latch.countDown();
Java 8 and Hazelcast already support custom thread pools for continuations.
E.g.:
Hazelcast.CompletableFutureTask.andThen(ExecutionCallback<V> callback,
Executor executor);
Looks like we should allow users to specify optional thread pool in futures
likewise.
Thoughts?
Vladimir.