You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jcs-users@jakarta.apache.org by Travis Savo <ts...@IFILM.com> on 2004/05/21 04:24:18 UTC
RE: Thread Control for Element Event Queue Processor - Number ofT
hreads too high
All right, if Aaron thinks we need it, once he has the patches in I can
start looking into adapting this throttle.
The one caveat I didn't mention is the new design would essentially need an
entire thread to mediate the throttling. Otherwise, things entering the
queue will block until there's a thread available to process them, which is
not really desirable. If I make a thread to mediate the throttle, events can
enter the queue without blocking even though a thread to process them won't
spawn. I may be able to wrap this up into the queue processing thread
itself... I guess wait and see.
-Travis Savo
-----Original Message-----
From: Aaron Smuts [mailto:aasmuts@wisc.edu]
Sent: Thursday, May 20, 2004 6:25 PM
To: 'Turbine JCS Users List'
Subject: RE: Thread Control for Element Event Queue Processor - Number
ofThreads too high
I think we may need something like a throttle.
I'm working the getting the patches in right now.
Aaron
> -----Original Message-----
> From: Travis Savo [mailto:tsavo@IFILM.com]
> Sent: Thursday, May 20, 2004 7:56 PM
> To: 'Turbine JCS Users List'
> Cc: 'ankur.kumar@axa.com.au'
> Subject: RE: Thread Control for Element Event Queue Processor - Number
> ofThreads too high
>
> I've submitted a patch which makes it only spawn threads when the
queues
> are
> in use, and keeps those threads around for a set amount of time to
avoid
> having to respawn them if the queue is active.
>
> I believe the updated code is already in 'experimental', and I hope it
> moves
> into prime-time soon because it's an absolute requirement for users
with a
> large number of regions.
>
> If it's a requirement to only have X amount of threads active at any
given
> time, the solution would be to introduce a Thread throttle, like this
one:
>
> import java.util.ArrayList;
> import java.util.List;
>
> import org.apache.log4j.Logger;
>
> /**
> * @author tsavo
> *
> */
> public class ThreadThrottle {
>
> private static final Logger logger =
> Logger.getLogger(ThreadThrottle.class);
> private static final int MAX_THREADS = 10;
> private static ThreadThrottle counter = null;
> private int threadCount;
> private List list;
>
> private ThreadThrottle() {
> threadCount = 0;
> list = new ArrayList();
> }
>
> public static synchronized ThreadThrottle getInstance() {
> if (counter == null) {
> counter = new ThreadThrottle();
> }
> return counter;
> }
>
> public void getThread() {
> Object waiter = null;
> synchronized (this) {
> ++threadCount;
> if (logger.isDebugEnabled()) {
> logger.debug("Current Thread Count: " + threadCount);
> }
> if (threadCount >= MAX_THREADS) {
> waiter = new Object();
> list.add(waiter);
> }
> }
> if (waiter != null) {
> if (logger.isDebugEnabled()) {
> logger.debug("Causing " + Thread.currentThread().getName() + "
to
> wait.");
> }
> synchronized (waiter) {
> try {
> waiter.wait();
> } catch (InterruptedException e) {
> return;
> }
> }
> if (logger.isDebugEnabled()) {
> logger.debug(Thread.currentThread().getName() + " got woken
up.");
> }
> } else {
> if (logger.isDebugEnabled()) {
> logger.debug(Thread.currentThread().getName() + " is running
> without
> waiting.");
> }
> }
> }
>
> public void returnThread() {
> Object waiter = null;
> synchronized (this) {
> if (logger.isDebugEnabled()) {
> logger.debug(Thread.currentThread().getName() + " finished.");
> }
> --threadCount;
> if (list.size() > 0) {
> waiter = list.remove(0);
> }
> }
> if (waiter != null) {
> if (logger.isDebugEnabled()) {
> logger.debug("Waking next thread in line...");
> }
> synchronized (waiter) {
> waiter.notify();
> }
> }
> }
> }
>
>
> Then you can call 'ThreadThrottle.getInstance().getThread()' right
before
> you spawn a new thread, and
ThreadThrottle.getInstance().returnThread()'
> right before it dies. getThread() will block until you're under your
> MAX_THREAD count.
>
> Making this Singleton a Multiton or simply non-static for the purposes
of
> having multiple ThreadThrottles, and implementing this in the
> EventQueueProcessor is an excercise left up to the user, but I have to
> immagine it's pretty straight forward work.
>
> -Travis Savo
>
>
>
> -----Original Message-----
> From: ankur.kumar@axa.com.au [mailto:ankur.kumar@axa.com.au]
> Sent: Thursday, May 20, 2004 6:11 PM
> To: Turbine JCS Users List
> Subject: Thread Control for Element Event Queue Processor - Number of
> Threads too high
>
>
> org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor
threads
> are nearly 16. I want to control that and limit it to 5-6 as when
> JCS will go to Linux (production) 16 threads are very costly for that
> environment. How can I restrict that to some limited number of
threads?
>
> Here is thread dump: (All seems to be waiting)
>
> "CacheEventQueue.QProcessor-1" daemon prio=5 tid=0x28b74d80 nid=0xb9c
> waiting on monito
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at org.apache.jcs.engine.CacheEventQueue.take(Unknown Source)
> at org.apache.jcs.engine.CacheEventQueue.access$500(Unknown
> Source)
> at
org.apache.jcs.engine.CacheEventQueue$QProcessor.run(Unknown
> Source)
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28c8ee18
nid=0xa10
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x290ee810
nid=0xc78
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x29578b98
nid=0x15fc
> waiting on mon
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28c8ece0
nid=0x854
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28aba450
nid=0xd74
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x290c3870
nid=0x1534
> waiting on mon
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x2895ca78
nid=0x3cc
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "CacheEventQueue.QProcessor-1" daemon prio=5 tid=0x2930ea50 nid=0x994
> waiting on monito
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at org.apache.jcs.engine.CacheEventQueue.take(Unknown Source)
> at org.apache.jcs.engine.CacheEventQueue.access$500(Unknown
> Source)
> at
org.apache.jcs.engine.CacheEventQueue$QProcessor.run(Unknown
> Source)
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x290d1780
nid=0x5fc
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x29019308
nid=0x8bc
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28871440
nid=0xfb8
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28b09640
nid=0xd30
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28992e00
nid=0xbbc
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28989ad0
nid=0x9d8
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "cache-PrepareThread" daemon prio=5 tid=0x28b01e80 nid=0x168 runnable
> [0x26d4f000..0x26
> at java.lang.Object.wait(Native Method)
> at
>
weblogic.persist.TxIndexedFileImpl$PrepareThread.run(TxIndexedFileImpl.j
av
> a:
>
> "cache-CommitThread" daemon prio=5 tid=0x2882ad88 nid=0xedc runnable
> [0x26cdf000..0x26c
> at java.lang.Object.wait(Native Method)
> at
>
weblogic.persist.TxIndexedFileImpl$CommitThread.run(TxIndexedFileImpl.ja
va
> :6
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x287dccd0
nid=0xc24
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x28928710
nid=0xac0
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
> "SSLListenThread.Default" prio=5 tid=0x288b2250 nid=0x92c runnable
> [0x269df000..0x269df
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:463)
> at java.net.ServerSocket.implAccept(ServerSocket.java:238)
> at javax.net.ssl.impl.SSLServerSocketImpl.accept(Unknown
Source)
> at weblogic.t3.srvr.ListenThread.run(ListenThread.java:263)
>
> "ListenThread.Default" prio=5 tid=0x288cbd70 nid=0x644 runnable
> [0x265df000..0x265dfdbc
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:463)
> at java.net.ServerSocket.implAccept(ServerSocket.java:238)
> at
>
com.wily.introscope.agent.probe.net.ManagedServerSocket.accept(ManagedSe
rv
> er
> at
>
weblogic.socket.WeblogicServerSocket.accept(WeblogicServerSocket.java:26
)
> at weblogic.t3.srvr.ListenThread.run(ListenThread.java:263)
>
> "ElementEventQueue.QProcessor-1" daemon prio=5 tid=0x288d28e0
nid=0x898
> waiting on moni
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:415)
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.take(ElementEventQ
ue
> ue
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue.access$300(Element
Ev
> en
> at
>
org.apache.jcs.engine.control.event.ElementEventQueue$QProcessor.run(Ele
me
> nt
>
>
>
>
************************************************************************
**
> **
> *****
> Important Note
> This email (including any attachments) contains information which is
> confidential and may be subject to legal privilege. If you are not
> the intended recipient you must not use, distribute or copy this
> email. If you have received this email in error please notify the
> sender immediately and delete this email. Any views expressed in this
> email are not necessarily the views of AXA. Thank you.
>
************************************************************************
**
> **
> *****
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
turbine-jcs-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail:
turbine-jcs-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: turbine-jcs-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: turbine-jcs-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: turbine-jcs-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: turbine-jcs-user-help@jakarta.apache.org