You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2023/09/05 21:14:12 UTC
Re: Virtual Thread Configuration In Tomcat 11
William,
On 8/24/23 09:50, William Crowell wrote:
> I did some performance testing with virtual threads on Apache Tomcat
> 11.0.0-M10 and JDK 21 (21+35-2513). I have a simple REST service
> using Spring 6.0.11 that does an insert into MySQL 8.0.32.
>
> I have 3 separate boxes all running Rocky Linux 9.2 on AWS
> (t3a.xlarge which is 4 vCPUs and 16GiB RAM):
>
> 1) An application server running Tomcat 11.0.0-M10 with JDK 21
>
> 2) MySQL 8.0.32
>
> 3) JMeter 5.6.2
>
> I know that JDK 21 is nowhere near GA, but I got some interesting
> results. I did 10 test runs with useVirtualThreads=“true”, and 10
> test runs without virtual threads (the default). Each run used 1000
> threads in JMeter for about 3 minutes and 20 seconds for a ramp up
> time of 3 seconds.
>
> What I found was using platform threads performed at least twice as
> fast as virtual threads. Maybe I am interpreting the results wrong,
> but this is what I found consistently across each test.
>
> Again, I realize that performance is not the only goal of virtual
> threads. Just my observations.
This is "interesting", for some values of interesting.
I'd be interested in what the application was doing. Virtual Threads
seem to be aimed at things like Web Application Containers where there
is a lot of wasted time in a thread's execution waiting on I/O
(blocking). If you ignore the application's workload (hah!), the
application server is almost entirely occupied with (a) reading the
request and (b) writing the response. There is almost no "computation
time" and so Virtual Threads make a whole lot of sense.
Once inside the application, that may not be true.
From what little reading I have done on Virtual Threads, there are some
poison pills in there depending upon application design. For example, if
your thread blocks while inside a synchronized{} block, then your
Virtual Thread is "pinned" and will NOT be un-mounted from the platform
thread during the blocking operation. So all of that high-speed
context-switching and magic that VT is supposed to provide goes
completely out the window when you used to have 200 threads with maybe 8
of them actually running at once but swapping-out preemptively, but now
you have 200 virtual threads with 8 of them running at once, but of the
8 running at once, several of them are stopped dead unable to make any
progress.
At $work, we have plenty of read-through cache routines like this:
private Object somethingBigLock = new Object();
private SomethingBig big = null;
public Object getSomethingBigFromDatabase() {
synchronized(somethingBigLock) {
if(null == big) {
big = reallyGetSomethingBigFromDb(); // Non-trivial
}
return big;
}
}
Even if reallyGetSomethingBigFromDb() is mostly blocking on I/O -- e.g.
waiting for the db to respond to a query -- it will consume one of your
platform threads and lock it up, leaving fewer platform threads
available to do the other work.
Without a critical analysis of your application and the libraries it
uses, it's hard to tell if what you are observing is that "virtual
threads are not as fast as regular threads" or "your application is an
anti-pattern for Virtual Threads". My guess is that your application
and/or the libraries you are using need to be re-written so they no
longer use "classic" synchronization and instead use things like
ReentrantLock for a similar purpose to play-nice with VT. Or you can
wait for the (likely) inevitable improvement of the JVM which allows for
threads holding monitors to be unmounted from platform threads. My guess
is that is in the future for the JVM.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Virtual Thread Configuration In Tomcat 11
Posted by Christopher Schultz <ch...@christopherschultz.net>.
William,
On 9/7/23 08:04, William Crowell wrote:
> When I set -Djdk.tracePinnedThreads=short, then I see this:
>
> …
> Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads]
> com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1
> Thread[#39,ForkJoinPool-1-worker-2,5,CarrierThreads]
> com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1
> Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads]
> com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1
> Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
> com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:893) <== monitors:1
> com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1061) <== monitors:1
> com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1009) <== monitors:1
> Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
> com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:791) <== monitors:1
> Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
> com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1
> …
>
> I started digging into the MySQL client code: https://github.com/mysql/mysql-connector-j/blob/8.0.33/src/main/user-impl/java/com/mysql/cj/jdbc/ConnectionImpl.java
>
> This class is littered with synchronized blocks, so now this makes sense to me why virtual threads would take longer than a regular OS thread.
This is precisely the kind of thing I was expecting to see.
If your "application code" (which includes everything your application
does, including calling into libraries) contains lots of "synchronized"
blocks, then I think this is the situation you will often find yourself in.
I would expect that if you were to increase the size of the virtual
thread pool things would start to get better, but in order to truly use
virtual threads effectively, you have to track-down and eliminate many
uses of synchronized blocks and methods in both your application and the
libraries you use.
I wouldn't expect virtual threads to become a very widely-deployed
performance-optimization technique in the immediate future. I expect
that, eventually, lots of code will be re-written to be more
virtual-thread-friendly.
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Virtual Thread Configuration In Tomcat 11
Posted by William Crowell <WC...@perforce.com.INVALID>.
When I set -Djdk.tracePinnedThreads=short, then I see this:
…
Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads]
com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1
Thread[#39,ForkJoinPool-1-worker-2,5,CarrierThreads]
com.mysql.cj.jdbc.ConnectionImpl.isValid(ConnectionImpl.java:2516) <== monitors:1
Thread[#41,ForkJoinPool-1-worker-4,5,CarrierThreads]
com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1
Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:893) <== monitors:1
com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1061) <== monitors:1
com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1009) <== monitors:1
Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:791) <== monitors:1
Thread[#43,ForkJoinPool-1-worker-5,5,CarrierThreads]
com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1
…
I started digging into the MySQL client code: https://github.com/mysql/mysql-connector-j/blob/8.0.33/src/main/user-impl/java/com/mysql/cj/jdbc/ConnectionImpl.java
This class is littered with synchronized blocks, so now this makes sense to me why virtual threads would take longer than a regular OS thread.
Regards,
William Crowell
This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
Re: Virtual Thread Configuration In Tomcat 11
Posted by William Crowell <WC...@perforce.com.INVALID>.
Chris,
I did set -Djdk.tracePinnedThreads=full and found there were a few pinned threads:
Thread[#43,ForkJoinPool-1-worker-4,5,CarrierThreads]
java.base/java.lang.VirtualThread$VThreadContinuation.onPinned(VirtualThread.java:185)
java.base/jdk.internal.vm.Continuation.onPinned0(Continuation.java:393)
java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:631)
java.base/java.lang.System$2.parkVirtualThread(System.java:2648)
java.base/jdk.internal.misc.VirtualThreads.park(VirtualThreads.java:67)
java.base/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:408)
…
com.mysql.cj.protocol.FullReadInputStream.readFully(FullReadInputStream.java:64)
com.mysql.cj.protocol.a.SimplePacketReader.readHeaderLocal(SimplePacketReader.java:81)
…
com.mysql.cj.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:2005) <== monitors:1
com.zaxxer.hikari.pool.ProxyConnection.setAutoCommit(ProxyConnection.java:401)
com.zaxxer.hikari.pool.HikariProxyConnection.setAutoCommit(HikariProxyConnection.java)
…
Over the 12 test runs I have performed, I found a 53% decrease in throughput when using virtual threads: http://ec2-18-188-185-212.us-east-2.compute.amazonaws.com:8080/web-report/
My example code is located here: https://github.com/wcrowell/rest-mysql-app/
The JMX test I used is located here: https://github.com/wcrowell/rest-mysql-app/blob/main/virtual-thread-test.jmx
This test makes a POST request to save a record to MySQL: http://ec2-18-117-196-158.us-east-2.compute.amazonaws.com:8080/vt/save
Regards,
William Crowell
This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
Re: Virtual Thread Configuration In Tomcat 11
Posted by Christopher Schultz <ch...@christopherschultz.net>.
William,
On 9/5/23 17:41, William Crowell wrote:
> Great post earlier today! This is a super interesting topic to me.
>
> You can find the performance testing results located here: http://ec2-18-188-185-212.us-east-2.compute.amazonaws.com:8080/web-report/
>
> I did 10 runs with 1000 threads with a ramp up time of 3 seconds for a duration of 200 seconds. Ignore the 11th run. This is a really simple REST application with Spring. It just makes an insert into a MySQL table. When I get a moment I will put my code into GitHub, but again, there is not much to it. You can find a very similar Spring Boot example with embedded Tomcat here:
>
> https://medium.com/@anil.java.story/embracing-virtual-threads-in-spring-boot-4140d3b8a5a
>
> But he uses JDK 20 with --enable-preview turned on.
>
> The mistake I think I made was that the inserts are not really blocking because they happen so quickly. Maybe because I did not put enough load in the test? I have no synchronized blocks in my code. So, it appeared to me that the inserts were taking twice the amount of time with virtual threads. I think that extra time is just overhead of pinning virtual threads to a platform thread and managing the ForkJoinPool.
>
> What is going to happen is that people are going to “flip the switch”…useVirtualThreads=”true”. They will find that performance will not be as good as using an operating system thread and abandon virtual threads because their code does not block.
>
> So, I need to come up with an example where my code blocks and redo the perf test and prove this out. I don’t know.
Try setting this system property before running your tests and see if it
produces any output on the server while the load test is running:
-Djdk.tracePinnedThreads=full
-chris
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Virtual Thread Configuration In Tomcat 11
Posted by William Crowell <WC...@perforce.com.INVALID>.
Chris,
Great post earlier today! This is a super interesting topic to me.
You can find the performance testing results located here: http://ec2-18-188-185-212.us-east-2.compute.amazonaws.com:8080/web-report/
I did 10 runs with 1000 threads with a ramp up time of 3 seconds for a duration of 200 seconds. Ignore the 11th run. This is a really simple REST application with Spring. It just makes an insert into a MySQL table. When I get a moment I will put my code into GitHub, but again, there is not much to it. You can find a very similar Spring Boot example with embedded Tomcat here:
https://medium.com/@anil.java.story/embracing-virtual-threads-in-spring-boot-4140d3b8a5a
But he uses JDK 20 with --enable-preview turned on.
The mistake I think I made was that the inserts are not really blocking because they happen so quickly. Maybe because I did not put enough load in the test? I have no synchronized blocks in my code. So, it appeared to me that the inserts were taking twice the amount of time with virtual threads. I think that extra time is just overhead of pinning virtual threads to a platform thread and managing the ForkJoinPool.
What is going to happen is that people are going to “flip the switch”…useVirtualThreads=”true”. They will find that performance will not be as good as using an operating system thread and abandon virtual threads because their code does not block.
So, I need to come up with an example where my code blocks and redo the perf test and prove this out. I don’t know.
Regards,
William Crowell
This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.