You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Roy Zhang <ro...@gmail.com> on 2019/11/01 12:23:57 UTC

Question about thread's "keep alive time" value in Embed Tomcat 8.5 in Spring Boot 1.5

Dear Tomcat experts,

I am confused about "keep alive time" value in Embed Tomcat. I am using
Spring Boot 1.5, embed tomcat version is 8.5.

According to
https://github.com/spring-projects/spring-boot/issues/16450#issuecomment-480236238,
we can't configured thread's "keep alive time" via "maxIdleTime"
parameter, "keep alive time" value is hard coded as 60s via
https://github.com/apache/tomcat/blob/de1f55e672106031821948d5cac80f40adc09513/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L884,
I found 60s "keep alive time" sometimes take effect, some times it doesn't
take effect. Could u please kindly help to review my following two cases
and kindly provide explanation? Really appreciate ur help in advance!

*Case 1 (60s "keep alive time" DOESN'T take effect):*
API latency: *100ms*
Test step: run API with 800 concurrent threads for 1 minutes, then run API
with 10 concurrent threads for about 1 hour. As we can see from following
graph, after BusyThreads reduced from ~800 to 10, CurrentThreadsCount
remains at 800 for following 1 hour (60s "keep alive time" DOESN'T take
effect). After test stopped, CurrentThreadsCount reduced from 800 to
100(MinSpareThreads).
[image: image.png]

*Case 2 (60s "keep alive time" DOES take effect):*
API latency: *10s*
Test step: run API with 800 concurrent threads for 1 minutes, then run API
with 10 concurrent threads for about 1 hour. As we can see from following
graph, after BusyThreads reduced from ~800 to 10, CurrentThreadsCount
reduced from 800 to 100(MinSpareThreads). 60s "keep alive time" DOES take
effect)

[image: image.png]

Thanks,
Roy

Re: Question about thread's "keep alive time" value in Embed Tomcat 8.5 in Spring Boot 1.5

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Roy,

On 11/4/19 18:22, Roy Zhang wrote:
> Dear Christopher,
> 
> Really appreciate ur quick response! Sorry for late reply due to
> personal affairs...
> 
>>>> I haven't read the code in detail, but I'm guessing that the
>>>> thread pool
> uses a queue and not a stack of threads, so each thread gets used, 
> repeatedly, in order of availability.
>>>> Note that Tomcat's thread pool is just a wrapper around
>>>> Java'sThreadPoolExecutor
> -- adding methods to handle Tomcat's lifecycle events, etc.
>>>> So if the pool isn't behaving as you'd like it, you'll need
>>>> to file a
> but against the Java API and not Tomcat. [Roy] Based on ur comment,
> my understanding is tomcat is using Java'sThreadPoolExecutor, 
> Java's ThreadPoolExecutor pick up new threads from thread pool
> using FIFO approach. Regarding "So if the pool isn't behaving as
> you'd like it, you'll need to file a but against the Java API and
> not Tomcat.", do we have any mature solution (e.g, any open source
> project) which change the tomcat thread pool's behavior (e.g, FIFO
> approach)?

So you'd like to use a LIFO thread pool? If you can find a LIFO
executor in Java, then we can probably configure it in Tomcat. But
writing a LIFO executor isn't exactly on our list of things to add to
Tomcat itself.

What's the goal, here? Your process must be able to tolerate a
completely full thread-pool, so you have to plan for maximum capacity.
Reducing process threads buys absolutely you nothing except bragging
rights for "using the fewest possible resources".

If you want fewer threads, make your thread pool smaller.

If you want to auto-scale, don't look at live-threads. Look instead at
recent-peak-concurrency.

- -chris

> On Fri, Nov 1, 2019 at 9:56 PM Christopher Schultz < 
> chris@christopherschultz.net> wrote:
> 
> Roy,
> 
> On 11/1/19 08:23, Roy Zhang wrote:
>>>> I am confused about "keep alive time" value in Embed Tomcat.
>>>> I am using Spring Boot 1.5, embed tomcat version is 8.5.
>>>> 
>>>> According to 
>>>> https://github.com/spring-projects/spring-boot/issues/16450#issueco
mme
>
>>>> 
nt-480236238
> <https://github.com/spring-projects/spring-boot/issues/16450#issuecomm
ent-480236238>
>
> 
,
>>>> 
>>>> 
> we can't configured thread's "keep alive time" via "maxIdleTime"
>>>> parameter, "keep alive time" value is hard coded as 60s via 
>>>> https://github.com/apache/tomcat/blob/de1f55e672106031821948d5cac80
f40
>
>>>> 
adc09513/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L884
> <https://github.com/apache/tomcat/blob/de1f55e672106031821948d5cac80f4
0adc09513/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L884>
>
> 
,
> 
> It
>>>> 
> is only hard-coded if you use the internal, default Executor. You 
> can instantiate your own Executor with whatever settings you wish.
> 
> It's a bad name for that setting, but this is how long "excess" 
> threads will live when they are not being used.
> 
>>>> I found 60s "keep alive time" sometimes take effect, some
>>>> times it doesn't take effect. Could u please kindly help to
>>>> review my following two cases and kindly provide explanation?
>>>> Really appreciate ur help in advance!
>>>> 
>>>> *Case 1 (60s "keep alive time" DOESN'T take effect):* API
>>>> latency: _100ms_ Test step: run API with 800 concurrent
>>>> threads for 1 minutes, then run API with 10 concurrent
>>>> threads for about 1 hour. As we can see from following graph,
>>>> after BusyThreads reduced from ~800 to 10,
>>>> CurrentThreadsCount remains at 800 for following 1 hour (60s
>>>> "keep alive time" DOESN'T take effect). After test stopped, 
>>>> CurrentThreadsCount reduced from 800 to
>>>> 100(MinSpareThreads). image.png
> 
> Your images have been stripped from the mailing list. Find another
> way to describe your problem.
> 
> When you say "X concurrent threads", how often are requests being 
> made? Constantly? So you are sending requests as fast as possible
> with c=10?
> 
> I haven't read the code in detail, but I'm guessing that the
> thread pool uses a queue and not a stack of threads, so each thread
> gets used, repeatedly, in order of availability. That means that if
> you keep making requests, you'll keep hitting *all* the threads in
> the queue and not just the same 10 over and over again. So they
> never time out.
> 
>>>> *Case 2 (60s "keep alive time" DOES take effect):* API
>>>> latency: _10s_ Test step: run API with 800 concurrent threads
>>>> for 1 minutes, then run API with 10 concurrent threads for
>>>> about 1 hour. As we can see from following graph, after
>>>> BusyThreads reduced from ~800 to 10, CurrentThreadsCount
>>>> reduced from 800 to 100(MinSpareThreads). 60s "keep alive
>>>> time" DOES take effect)
> 
> Note that Tomcat's thread pool is just a wrapper around Java's 
> ThreadPoolExecutor -- adding methods to handle Tomcat's lifecycle 
> events, etc.
> 
> So if the pool isn't behaving as you'd like it, you'll need to file
> a but against the Java API and not Tomcat.
> 
> -chris
>> 
>> ---------------------------------------------------------------------
>>
>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>> 
>> 
> 
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl3CBRQACgkQHPApP6U8
pFhXzg/+ONG+GEmyafoiLAsnTVE5urfrVtsn4th6NQC/2kEyIzXvL7unG0tW0U09
nzXdyuFWyG/U6d+dHgyeKa37XpSAEWSIUF4dHGCURaJnv1KhYO7L9FvNbFDgQFj7
4XwxrKB/Od33g7pkyIP+U+gT2KQCS22dy/8/Pev43TgvobQhuWR7jYGdWHTkWqHD
Qg4n7WUvXDvueoumnKnGNHmIeaJT3LL6dFu0uUjGU4NWa1EpAUdxy6keEZtsLuJ5
C+tEQPoaj1nsWGK0jb4izqARpdHSBwsAysTdNdU8EwnpY/iW2mQWA06n3WgfmgcZ
aFTggSA0n4HXyQ3VDlrUtOh/T/liOgrDj08UnoFtP7NBowr4k8Mllz78UdMT0r+T
wkAzvZ1nDGdavOKgBLlBPZXmU7q3zQLfui/yOkvoFI/FeNYxIcXqkwV6dNJR/7YK
rvQuKmDdCqpvyhacQ2ummWMomo0QnK+M6FHPh8y/nUOinAF3gU/2yFGu/FM1j6AK
6/siYKeno6goISlWiQWtnyOKNTNyO1Rprs3HYmqA7NDbVUIvOf4oPc+ZEcWY6com
WpD6P3c0TiDlDSwK8c09tVC7EdOREtxhasYvJs8smBtEZ+Ey0LVjj2/Bu4vJRxCn
zxf/ro0oo3W1vBiZEkZN1EVnyV9Yk2QE2WCJpz1mnEIDy3I4W3A=
=teeo
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Question about thread's "keep alive time" value in Embed Tomcat 8.5 in Spring Boot 1.5

Posted by Roy Zhang <ro...@gmail.com>.
Dear Christopher,

Really appreciate ur quick response! Sorry for late reply due to personal
affairs...

>>>I haven't read the code in detail, but I'm guessing that the thread pool
uses a queue and not a stack of threads, so each thread gets used,
repeatedly, in order of availability.
>>>Note that Tomcat's thread pool is just a wrapper around Java'sThreadPoolExecutor
-- adding methods to handle Tomcat's lifecycle events, etc.
>>>So if the pool isn't behaving as you'd like it, you'll need to file a
but against the Java API and not Tomcat.
[Roy] Based on ur comment, my understanding is tomcat is using
Java'sThreadPoolExecutor,
Java's ThreadPoolExecutor pick up new threads from thread pool using FIFO
approach. Regarding "So if the pool isn't behaving as you'd like it, you'll
need to file a but against the Java API and not Tomcat.", do we have any
mature solution (e.g, any open source project) which change the tomcat
thread pool's behavior (e.g, FIFO approach)?

Thanks in advance!

Thanks,
Roy

On Fri, Nov 1, 2019 at 9:56 PM Christopher Schultz <
chris@christopherschultz.net> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Roy,
>
> On 11/1/19 08:23, Roy Zhang wrote:
> > I am confused about "keep alive time" value in Embed Tomcat. I am
> > using Spring Boot 1.5, embed tomcat version is 8.5.
> >
> > According to
> > https://github.com/spring-projects/spring-boot/issues/16450#issuecomme
> nt-480236238
> <https://github.com/spring-projects/spring-boot/issues/16450#issuecomment-480236238>
> ,
> >
> >
> we can't configured thread's "keep alive time" via "maxIdleTime"
> > parameter, "keep alive time" value is hard coded as 60s via
> > https://github.com/apache/tomcat/blob/de1f55e672106031821948d5cac80f40
> adc09513/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L884
> <https://github.com/apache/tomcat/blob/de1f55e672106031821948d5cac80f40adc09513/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L884>
> ,
>
> It
> >
> is only hard-coded if you use the internal, default Executor. You
> can instantiate your own Executor with whatever settings you wish.
>
> It's a bad name for that setting, but this is how long "excess"
> threads will live when they are not being used.
>
> > I found 60s "keep alive time" sometimes take effect, some times it
> > doesn't take effect. Could u please kindly help to review my
> > following two cases and kindly provide explanation? Really
> > appreciate ur help in advance!
> >
> > *Case 1 (60s "keep alive time" DOESN'T take effect):* API latency:
> > _100ms_ Test step: run API with 800 concurrent threads for 1
> > minutes, then run API with 10 concurrent threads for about 1 hour.
> > As we can see from following graph, after BusyThreads reduced from
> > ~800 to 10, CurrentThreadsCount remains at 800 for following 1 hour
> > (60s "keep alive time" DOESN'T take effect). After test stopped,
> > CurrentThreadsCount reduced from 800 to 100(MinSpareThreads).
> > image.png
>
> Your images have been stripped from the mailing list. Find another way
> to describe your problem.
>
> When you say "X concurrent threads", how often are requests being
> made? Constantly? So you are sending requests as fast as possible with
> c=10?
>
> I haven't read the code in detail, but I'm guessing that the thread
> pool uses a queue and not a stack of threads, so each thread gets
> used, repeatedly, in order of availability. That means that if you
> keep making requests, you'll keep hitting *all* the threads in the
> queue and not just the same 10 over and over again. So they never time
> out.
>
> > *Case 2 (60s "keep alive time" DOES take effect):* API latency:
> > _10s_ Test step: run API with 800 concurrent threads for 1 minutes,
> > then run API with 10 concurrent threads for about 1 hour. As we can
> > see from following graph, after BusyThreads reduced from ~800 to
> > 10, CurrentThreadsCount reduced from 800 to 100(MinSpareThreads).
> > 60s "keep alive time" DOES take effect)
>
> Note that Tomcat's thread pool is just a wrapper around Java's
> ThreadPoolExecutor -- adding methods to handle Tomcat's lifecycle
> events, etc.
>
> So if the pool isn't behaving as you'd like it, you'll need to file a
> but against the Java API and not Tomcat.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl28NzYACgkQHPApP6U8
> pFjcRxAApIe9WPrO43rIn+8TZ/KBncZbw2ZQr/nEew1pS7xdRFl2LVn/gMpuYaIe
> 78rF/3M9S0aYcG93vaqzn1+BZXW5EI9EINo+UG3E17ZYeD162Cpl2Q7aqaB/Te0j
> QR/JHIv31TAUpH39VLEpzfk/d1uct7oVZDq4P3kUjjAFwAo2G0useiHjokXtdIEm
> 3wXL1e8ChpvdP8RfrPdeATdGtEvGh0zNDW2Qjce+7R5LoC55UpHqJfB+p8BB6nV/
> W6Jdp8CN8pVp+wROMMyK3HFvubFqSPcrKGURCmwKMH/QgRfRSTNpIfXYpubfOOqm
> EF0XxvRc2s/x6ZqRrfnbkunqqmb0h35SXPaWl1RJWpe/cAAaHi0DGORorbDZfnff
> agzNvl7Wl0/6HL09CXr2PHU6ZQGytSYbItVAjEHqLKhbYW4pLKviLYB84ZRiEc5N
> 2ZA2ZE9VKWQRgu1kS25U4c5R7Zgy5oVAAIPLzNqEdcqu2dXSsZZeW4sJutwjioOL
> pia5qqRcotdDPxATqv0Eo75cqvvgE5WeWZ/tMb2sVia85UPRL/cJCgrc/+HmYPmb
> JRzhobGhQ7WA33yqRfvBdcJ5HIGe2aU4o06VoAv6tHXPldjD4jQ/YHyyicP+bj6E
> wswEM7oIAIU8+R7JFp1Lhv6ZMuLlKohDmOu2bubhAgweSrAQ8jY=
> =901h
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: Question about thread's "keep alive time" value in Embed Tomcat 8.5 in Spring Boot 1.5

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Roy,

On 11/1/19 08:23, Roy Zhang wrote:
> I am confused about "keep alive time" value in Embed Tomcat. I am
> using Spring Boot 1.5, embed tomcat version is 8.5.
> 
> According to
> https://github.com/spring-projects/spring-boot/issues/16450#issuecomme
nt-480236238,
>
> 
we can't configured thread's "keep alive time" via "maxIdleTime"
> parameter, "keep alive time" value is hard coded as 60s via
> https://github.com/apache/tomcat/blob/de1f55e672106031821948d5cac80f40
adc09513/java/org/apache/tomcat/util/net/AbstractEndpoint.java#L884,

It
> 
is only hard-coded if you use the internal, default Executor. You
can instantiate your own Executor with whatever settings you wish.

It's a bad name for that setting, but this is how long "excess"
threads will live when they are not being used.

> I found 60s "keep alive time" sometimes take effect, some times it 
> doesn't take effect. Could u please kindly help to review my
> following two cases and kindly provide explanation? Really
> appreciate ur help in advance!
> 
> *Case 1 (60s "keep alive time" DOESN'T take effect):* API latency:
> _100ms_ Test step: run API with 800 concurrent threads for 1
> minutes, then run API with 10 concurrent threads for about 1 hour.
> As we can see from following graph, after BusyThreads reduced from
> ~800 to 10, CurrentThreadsCount remains at 800 for following 1 hour
> (60s "keep alive time" DOESN'T take effect). After test stopped,
> CurrentThreadsCount reduced from 800 to 100(MinSpareThreads). 
> image.png

Your images have been stripped from the mailing list. Find another way
to describe your problem.

When you say "X concurrent threads", how often are requests being
made? Constantly? So you are sending requests as fast as possible with
c=10?

I haven't read the code in detail, but I'm guessing that the thread
pool uses a queue and not a stack of threads, so each thread gets
used, repeatedly, in order of availability. That means that if you
keep making requests, you'll keep hitting *all* the threads in the
queue and not just the same 10 over and over again. So they never time
out.

> *Case 2 (60s "keep alive time" DOES take effect):* API latency:
> _10s_ Test step: run API with 800 concurrent threads for 1 minutes,
> then run API with 10 concurrent threads for about 1 hour. As we can
> see from following graph, after BusyThreads reduced from ~800 to 
> 10, CurrentThreadsCount reduced from 800 to 100(MinSpareThreads).
> 60s "keep alive time" DOES take effect)

Note that Tomcat's thread pool is just a wrapper around Java's
ThreadPoolExecutor -- adding methods to handle Tomcat's lifecycle
events, etc.

So if the pool isn't behaving as you'd like it, you'll need to file a
but against the Java API and not Tomcat.

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl28NzYACgkQHPApP6U8
pFjcRxAApIe9WPrO43rIn+8TZ/KBncZbw2ZQr/nEew1pS7xdRFl2LVn/gMpuYaIe
78rF/3M9S0aYcG93vaqzn1+BZXW5EI9EINo+UG3E17ZYeD162Cpl2Q7aqaB/Te0j
QR/JHIv31TAUpH39VLEpzfk/d1uct7oVZDq4P3kUjjAFwAo2G0useiHjokXtdIEm
3wXL1e8ChpvdP8RfrPdeATdGtEvGh0zNDW2Qjce+7R5LoC55UpHqJfB+p8BB6nV/
W6Jdp8CN8pVp+wROMMyK3HFvubFqSPcrKGURCmwKMH/QgRfRSTNpIfXYpubfOOqm
EF0XxvRc2s/x6ZqRrfnbkunqqmb0h35SXPaWl1RJWpe/cAAaHi0DGORorbDZfnff
agzNvl7Wl0/6HL09CXr2PHU6ZQGytSYbItVAjEHqLKhbYW4pLKviLYB84ZRiEc5N
2ZA2ZE9VKWQRgu1kS25U4c5R7Zgy5oVAAIPLzNqEdcqu2dXSsZZeW4sJutwjioOL
pia5qqRcotdDPxATqv0Eo75cqvvgE5WeWZ/tMb2sVia85UPRL/cJCgrc/+HmYPmb
JRzhobGhQ7WA33yqRfvBdcJ5HIGe2aU4o06VoAv6tHXPldjD4jQ/YHyyicP+bj6E
wswEM7oIAIU8+R7JFp1Lhv6ZMuLlKohDmOu2bubhAgweSrAQ8jY=
=901h
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org