Would Java Fibers (aka Loom Project) be of interest for improving Storm capacities?

[Alexandre Vermeerbergen <av...@gmail.com>]

Hello guys,

I have a weird idea bugging my mind, so let me share it and if that's
improductive/dead end, then I'll forget it :)

I have been reading a few sources about the notion of Fibers in Java
(aka Loom Project) which are generally available with Java 21.

There's an optional "virtual threads executor" in Apache Tomcat since
a while: https://tomcat.apache.org/tomcat-8.5-doc/config/executor.html#Virtual_Thread_Implementation
 ; would it be worth to have something similar in Apache Storm ?

Note: Storm is a scheduler, so normally the main pain of Java threads
which is that they are OS threads is part of Storm's mission to
schedule a potentially larger set of spouts & bolts than there are
physical threads on all supervisor nodes, right?  does it mean that
virtual threads would be useless for Storm ?

Sorry for pushing headaches to some of you with these questions, but
I'd like to challenge the idea :)

Alexandre

Re: Would Java Fibers (aka Loom Project) be of interest for improving Storm capacities?

[Bipin Prasad <bi...@yahoo.com.INVALID>]

Hello Alexandre,
Fibers might be useful. Will need to code for that and do performance analysis. Currently each worker JVM (multiple for a supervisor) runs components in threads.For I/O bound processes/topology components, fibers “should” help. It could be a topology level option (ie whether ot uses threads or fibers). My two cents.

—Bipin

Sent from Yahoo Mail for iPhone


On Saturday, December 2, 2023, 12:48 AM, Alexandre Vermeerbergen <av...@gmail.com> wrote:

Hello guys,

I have a weird idea bugging my mind, so let me share it and if that's
improductive/dead end, then I'll forget it :)

I have been reading a few sources about the notion of Fibers in Java
(aka Loom Project) which are generally available with Java 21.

There's an optional "virtual threads executor" in Apache Tomcat since
a while: https://tomcat.apache.org/tomcat-8.5-doc/config/executor.html#Virtual_Thread_Implementation
 ; would it be worth to have something similar in Apache Storm ?

Note: Storm is a scheduler, so normally the main pain of Java threads
which is that they are OS threads is part of Storm's mission to
schedule a potentially larger set of spouts & bolts than there are
physical threads on all supervisor nodes, right?  does it mean that
virtual threads would be useless for Storm ?

Sorry for pushing headaches to some of you with these questions, but
I'd like to challenge the idea :)

Alexandre