You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Mark Thomas <ma...@apache.org> on 2023/02/27 16:53:32 UTC
Loom research
FYI
https://spring.io/blog/2023/02/27/web-applications-and-project-loom
Written for $dayjob but the data is based purely on Tomcat and a trivial
web application.
I think this could prove useful when we need discuss what to do with the
Loom module code.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Loom research
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi all,
Checked the tccl thing and VirtualThread accept to set their TCCL but
default to the creation thread one so issue is really more on the caller so
guess it is more about tomcat ensuring it is created at the right time and
likely integrated with the standard context loader appropriately so only
the pool to configure IMHO and tomcat could autoswitch to loom when
available :).
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le lun. 27 févr. 2023 à 20:39, Romain Manni-Bucau <rm...@gmail.com> a
écrit :
>
>
> Le lun. 27 févr. 2023 à 20:21, Mark Thomas <ma...@apache.org> a écrit :
>
>> On 27/02/2023 18:02, Romain Manni-Bucau wrote:> Le lun. 27 févr. 2023 à
>> 18:38, Mark Thomas <ma...@apache.org> a écrit :
>> >
>> >> On 27/02/2023 17:09, Romain Manni-Bucau wrote:
>> >>> Hi,
>> >>>
>> >>> AFAIK loom or a standard thread pool are exactly the same except when
>> you
>> >>> use loom you need to ensure you pass the contextual executor to use
>> >>> (scheduler in the API IIRC) otherwise the Thread.startVirtualThread
>> will
>> >>> use the global JVM one (so you use 2 thread pools which is unintended
>> in
>> >>> general and for tomcat which is multi-webapp can be an issue in some
>> >>> contexts).
>> >>> So overall the main benefit is to enable to use the contextual thread
>> >> pool
>> >>> owner to execute any banalized task.
>> >>> This is a great advantage and shouldnt touch other applications (which
>> >>> stays in a thread pool executor) so it sounds hurtless to auto switch
>> to
>> >>> loom when possible.
>> >>>
>> >>> However it means you do not use LoomExecutor but a configured loom
>> >> executor
>> >>> (default works while only using loom friendly and not blocking code
>> but
>> >> as
>> >>> soon as you will break that statement it will not work anymore).
>> >>> Ideally it means using java/lang/ThreadBuilders(Executor) to
>> configure it
>> >>> and pass a standard executor to it.
>> >>>
>> >>> Hope it makes sense.
>> >>
>> >> Not really, no. There is no thread pool with Loom.
>> >>
>> >
>> > There is an implicit default fork join pool, see
>> >
>> https://github.com/openjdk/loom/blob/e9f05191c306b2af9d91397f8a5bfffd948e6d69/src/java.base/share/classes/java/lang/VirtualThread.java#L1081
>> > .
>> > It defaults to the number of proc.
>> > Max is 256 and min number of proc/2.
>> > Similarly there is a scheduled pool of 1 by default for background
>> tasks.
>>
>> Oh. The carrier threads. That clarifies your meaning somewhat. There is
>> a notable difference between that and the standard thread pool which was
>> evident in the tests in the blog post. The blocking queue used for the
>> work queue on the standard thread pool suffers from contention much more
>> than Loom's work stealing queue.
>>
>
> Well theorically it should converge at some point (speaking outside
> continuation part which adds specific contentions and optimization) since
> both are likely the same.
> This is also why the perfs are not that different when the app using a
> thread pool really does not block.
>
>
>>
>> Yes, putting virtual threads into a standard thread pool is pointless
>> (and probably has a negative performance impact too).
>>
>> Tomcat has taken the simple approach for this for now:
>>
>> https://github.com/apache/tomcat/blob/main/modules/loom/src/main/java/org/apache/catalina/core/LoomExecutor.java
>
>
> Means you don't control the tomcat thread pool used by virtual threads
> which can be ok-ish when you deploy a single webapp in a tomcat owning the
> JVM but miss some control when you want to allocate threads to usages
> (executor per webapp is what I have in mind).
>
>
>>
>>
>> Whether we'll need to worry about separate pools of carrier threads - or
>> configuration of that pool / those pools - is TBD.
>>
>
> ack
>
>
>>
>> I'm still not sure what you mean by "default works while only using loom
>> friendly and not blocking code".
>>
>
> Default stays a forkjoin pool, which works as of today without loom so
> switching the default to tomcat is fine but is not LoomExecutor but a
> slightly enhance flavor which controls the underlying scheduler as task
> scheduler (to not rely on system props but tomcat executor props).
>
>
>>
>>
>> > So using loom without caution is using a JVM (with its classloader side
>> > effect) thread pool which is enabled for JVM continuations, nothing
>> more.
>>
>> What class loader effects? The context class loader should be picked up
>> from the virtual thread.
>>
>
> This part highly depends the code and deployment setup.
> Carrying thread has the JVM classloader as contextual one (a bit like
> common pool).
> Ultimately the continuation is what will define the context once submitted
> to the thread pool:
> https://github.com/openjdk/loom/blob/e9f05191c306b2af9d91397f8a5bfffd948e6d69/src/java.base/share/classes/java/lang/VirtualThread.java#L235
> (which enters back in a standard thread pool land ;)).
> Reading the loom spec it is unclear to me if the context classloader can
> be controlled or not (I'm tempted to think not) so this can be an issue if
> the enclosing thread pool does not help and enable it in the context of
> tomcat.
>
>
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: dev-help@tomcat.apache.org
>>
>>
Re: Loom research
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le lun. 27 févr. 2023 à 20:21, Mark Thomas <ma...@apache.org> a écrit :
> On 27/02/2023 18:02, Romain Manni-Bucau wrote:> Le lun. 27 févr. 2023 à
> 18:38, Mark Thomas <ma...@apache.org> a écrit :
> >
> >> On 27/02/2023 17:09, Romain Manni-Bucau wrote:
> >>> Hi,
> >>>
> >>> AFAIK loom or a standard thread pool are exactly the same except when
> you
> >>> use loom you need to ensure you pass the contextual executor to use
> >>> (scheduler in the API IIRC) otherwise the Thread.startVirtualThread
> will
> >>> use the global JVM one (so you use 2 thread pools which is unintended
> in
> >>> general and for tomcat which is multi-webapp can be an issue in some
> >>> contexts).
> >>> So overall the main benefit is to enable to use the contextual thread
> >> pool
> >>> owner to execute any banalized task.
> >>> This is a great advantage and shouldnt touch other applications (which
> >>> stays in a thread pool executor) so it sounds hurtless to auto switch
> to
> >>> loom when possible.
> >>>
> >>> However it means you do not use LoomExecutor but a configured loom
> >> executor
> >>> (default works while only using loom friendly and not blocking code but
> >> as
> >>> soon as you will break that statement it will not work anymore).
> >>> Ideally it means using java/lang/ThreadBuilders(Executor) to configure
> it
> >>> and pass a standard executor to it.
> >>>
> >>> Hope it makes sense.
> >>
> >> Not really, no. There is no thread pool with Loom.
> >>
> >
> > There is an implicit default fork join pool, see
> >
> https://github.com/openjdk/loom/blob/e9f05191c306b2af9d91397f8a5bfffd948e6d69/src/java.base/share/classes/java/lang/VirtualThread.java#L1081
> > .
> > It defaults to the number of proc.
> > Max is 256 and min number of proc/2.
> > Similarly there is a scheduled pool of 1 by default for background tasks.
>
> Oh. The carrier threads. That clarifies your meaning somewhat. There is
> a notable difference between that and the standard thread pool which was
> evident in the tests in the blog post. The blocking queue used for the
> work queue on the standard thread pool suffers from contention much more
> than Loom's work stealing queue.
>
Well theorically it should converge at some point (speaking outside
continuation part which adds specific contentions and optimization) since
both are likely the same.
This is also why the perfs are not that different when the app using a
thread pool really does not block.
>
> Yes, putting virtual threads into a standard thread pool is pointless
> (and probably has a negative performance impact too).
>
> Tomcat has taken the simple approach for this for now:
>
> https://github.com/apache/tomcat/blob/main/modules/loom/src/main/java/org/apache/catalina/core/LoomExecutor.java
Means you don't control the tomcat thread pool used by virtual threads
which can be ok-ish when you deploy a single webapp in a tomcat owning the
JVM but miss some control when you want to allocate threads to usages
(executor per webapp is what I have in mind).
>
>
> Whether we'll need to worry about separate pools of carrier threads - or
> configuration of that pool / those pools - is TBD.
>
ack
>
> I'm still not sure what you mean by "default works while only using loom
> friendly and not blocking code".
>
Default stays a forkjoin pool, which works as of today without loom so
switching the default to tomcat is fine but is not LoomExecutor but a
slightly enhance flavor which controls the underlying scheduler as task
scheduler (to not rely on system props but tomcat executor props).
>
>
> > So using loom without caution is using a JVM (with its classloader side
> > effect) thread pool which is enabled for JVM continuations, nothing more.
>
> What class loader effects? The context class loader should be picked up
> from the virtual thread.
>
This part highly depends the code and deployment setup.
Carrying thread has the JVM classloader as contextual one (a bit like
common pool).
Ultimately the continuation is what will define the context once submitted
to the thread pool:
https://github.com/openjdk/loom/blob/e9f05191c306b2af9d91397f8a5bfffd948e6d69/src/java.base/share/classes/java/lang/VirtualThread.java#L235
(which enters back in a standard thread pool land ;)).
Reading the loom spec it is unclear to me if the context classloader can be
controlled or not (I'm tempted to think not) so this can be an issue if the
enclosing thread pool does not help and enable it in the context of tomcat.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
Re: Loom research
Posted by Mark Thomas <ma...@apache.org>.
On 27/02/2023 18:02, Romain Manni-Bucau wrote:> Le lun. 27 févr. 2023 à
18:38, Mark Thomas <ma...@apache.org> a écrit :
>
>> On 27/02/2023 17:09, Romain Manni-Bucau wrote:
>>> Hi,
>>>
>>> AFAIK loom or a standard thread pool are exactly the same except when you
>>> use loom you need to ensure you pass the contextual executor to use
>>> (scheduler in the API IIRC) otherwise the Thread.startVirtualThread will
>>> use the global JVM one (so you use 2 thread pools which is unintended in
>>> general and for tomcat which is multi-webapp can be an issue in some
>>> contexts).
>>> So overall the main benefit is to enable to use the contextual thread
>> pool
>>> owner to execute any banalized task.
>>> This is a great advantage and shouldnt touch other applications (which
>>> stays in a thread pool executor) so it sounds hurtless to auto switch to
>>> loom when possible.
>>>
>>> However it means you do not use LoomExecutor but a configured loom
>> executor
>>> (default works while only using loom friendly and not blocking code but
>> as
>>> soon as you will break that statement it will not work anymore).
>>> Ideally it means using java/lang/ThreadBuilders(Executor) to configure it
>>> and pass a standard executor to it.
>>>
>>> Hope it makes sense.
>>
>> Not really, no. There is no thread pool with Loom.
>>
>
> There is an implicit default fork join pool, see
> https://github.com/openjdk/loom/blob/e9f05191c306b2af9d91397f8a5bfffd948e6d69/src/java.base/share/classes/java/lang/VirtualThread.java#L1081
> .
> It defaults to the number of proc.
> Max is 256 and min number of proc/2.
> Similarly there is a scheduled pool of 1 by default for background tasks.
Oh. The carrier threads. That clarifies your meaning somewhat. There is
a notable difference between that and the standard thread pool which was
evident in the tests in the blog post. The blocking queue used for the
work queue on the standard thread pool suffers from contention much more
than Loom's work stealing queue.
Yes, putting virtual threads into a standard thread pool is pointless
(and probably has a negative performance impact too).
Tomcat has taken the simple approach for this for now:
https://github.com/apache/tomcat/blob/main/modules/loom/src/main/java/org/apache/catalina/core/LoomExecutor.java
Whether we'll need to worry about separate pools of carrier threads - or
configuration of that pool / those pools - is TBD.
I'm still not sure what you mean by "default works while only using loom
friendly and not blocking code".
> So using loom without caution is using a JVM (with its classloader side
> effect) thread pool which is enabled for JVM continuations, nothing more.
What class loader effects? The context class loader should be picked up
from the virtual thread.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Loom research
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le lun. 27 févr. 2023 à 18:38, Mark Thomas <ma...@apache.org> a écrit :
> On 27/02/2023 17:09, Romain Manni-Bucau wrote:
> > Hi,
> >
> > AFAIK loom or a standard thread pool are exactly the same except when you
> > use loom you need to ensure you pass the contextual executor to use
> > (scheduler in the API IIRC) otherwise the Thread.startVirtualThread will
> > use the global JVM one (so you use 2 thread pools which is unintended in
> > general and for tomcat which is multi-webapp can be an issue in some
> > contexts).
> > So overall the main benefit is to enable to use the contextual thread
> pool
> > owner to execute any banalized task.
> > This is a great advantage and shouldnt touch other applications (which
> > stays in a thread pool executor) so it sounds hurtless to auto switch to
> > loom when possible.
> >
> > However it means you do not use LoomExecutor but a configured loom
> executor
> > (default works while only using loom friendly and not blocking code but
> as
> > soon as you will break that statement it will not work anymore).
> > Ideally it means using java/lang/ThreadBuilders(Executor) to configure it
> > and pass a standard executor to it.
> >
> > Hope it makes sense.
>
> Not really, no. There is no thread pool with Loom.
>
There is an implicit default fork join pool, see
https://github.com/openjdk/loom/blob/e9f05191c306b2af9d91397f8a5bfffd948e6d69/src/java.base/share/classes/java/lang/VirtualThread.java#L1081
.
It defaults to the number of proc.
Max is 256 and min number of proc/2.
Similarly there is a scheduled pool of 1 by default for background tasks.
So using loom without caution is using a JVM (with its classloader side
effect) thread pool which is enabled for JVM continuations, nothing more.
>
> Mark
>
>
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> | Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
> >
> >
> > Le lun. 27 févr. 2023 à 17:53, Mark Thomas <ma...@apache.org> a écrit :
> >
> >> FYI
> >>
> >> https://spring.io/blog/2023/02/27/web-applications-and-project-loom
> >>
> >> Written for $dayjob but the data is based purely on Tomcat and a trivial
> >> web application.
> >>
> >> I think this could prove useful when we need discuss what to do with the
> >> Loom module code.
> >>
> >> Mark
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: dev-help@tomcat.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>
Re: Loom research
Posted by Mark Thomas <ma...@apache.org>.
On 27/02/2023 17:09, Romain Manni-Bucau wrote:
> Hi,
>
> AFAIK loom or a standard thread pool are exactly the same except when you
> use loom you need to ensure you pass the contextual executor to use
> (scheduler in the API IIRC) otherwise the Thread.startVirtualThread will
> use the global JVM one (so you use 2 thread pools which is unintended in
> general and for tomcat which is multi-webapp can be an issue in some
> contexts).
> So overall the main benefit is to enable to use the contextual thread pool
> owner to execute any banalized task.
> This is a great advantage and shouldnt touch other applications (which
> stays in a thread pool executor) so it sounds hurtless to auto switch to
> loom when possible.
>
> However it means you do not use LoomExecutor but a configured loom executor
> (default works while only using loom friendly and not blocking code but as
> soon as you will break that statement it will not work anymore).
> Ideally it means using java/lang/ThreadBuilders(Executor) to configure it
> and pass a standard executor to it.
>
> Hope it makes sense.
Not really, no. There is no thread pool with Loom.
Mark
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le lun. 27 févr. 2023 à 17:53, Mark Thomas <ma...@apache.org> a écrit :
>
>> FYI
>>
>> https://spring.io/blog/2023/02/27/web-applications-and-project-loom
>>
>> Written for $dayjob but the data is based purely on Tomcat and a trivial
>> web application.
>>
>> I think this could prove useful when we need discuss what to do with the
>> Loom module code.
>>
>> Mark
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: dev-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Loom research
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi,
AFAIK loom or a standard thread pool are exactly the same except when you
use loom you need to ensure you pass the contextual executor to use
(scheduler in the API IIRC) otherwise the Thread.startVirtualThread will
use the global JVM one (so you use 2 thread pools which is unintended in
general and for tomcat which is multi-webapp can be an issue in some
contexts).
So overall the main benefit is to enable to use the contextual thread pool
owner to execute any banalized task.
This is a great advantage and shouldnt touch other applications (which
stays in a thread pool executor) so it sounds hurtless to auto switch to
loom when possible.
However it means you do not use LoomExecutor but a configured loom executor
(default works while only using loom friendly and not blocking code but as
soon as you will break that statement it will not work anymore).
Ideally it means using java/lang/ThreadBuilders(Executor) to configure it
and pass a standard executor to it.
Hope it makes sense.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le lun. 27 févr. 2023 à 17:53, Mark Thomas <ma...@apache.org> a écrit :
> FYI
>
> https://spring.io/blog/2023/02/27/web-applications-and-project-loom
>
> Written for $dayjob but the data is based purely on Tomcat and a trivial
> web application.
>
> I think this could prove useful when we need discuss what to do with the
> Loom module code.
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
>