You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jmeter-dev@jakarta.apache.org by Jamie Davidson <Ja...@bridgewatersystems.com> on 2001/05/24 22:46:23 UTC
ThreadGroup::SampleQueue woes
I think that I have figured out the 'Out Of Memory Errors' that I have been recently enjoying with JMeter 1.6alpha on Solaris 2.7, with Java 1.3.
Apparently the LinkedList class has a flaw in it (or perhaps my knowledge of the LinkedList is lacking - as I said, I am a relative newbie in this development area).
Seems that when the sample is done, the size of the queue is big. I would have expected it to be zero (or close to zero).
Some further investigation turned up the following:
It appears that the LinkedList can get into a state where it throw a NoSuchElementException when removeFirst() is called, even when the list is not empty.
To test this, I changed the SampleQueue class a bit:
private class SampleQueue extends Thread { private LinkedList occurredQ = new LinkedList();
public SampleQueue() { Debug.println("SampleQueue created..."); }
public synchronized void sampleOccurred(SampleEvent e) { occurredQ.addLast(e); this.notify(); }
public long size() { return occurredQ.size(); }
public void run() { Debug.println("Event Queue has started running...", Debug.MIN_PRIORITY); SampleEvent event = null; long sample = 1; long iteration = 1; long exception = 1; while(true) { if (++iteration%1000==0) Debug.println("Processing iteration #"+iteration+" of the Event Queue", Debug.MIN_PRIORITY); //get the first available item in the queue... try { event = (SampleEvent)occurredQ.removeFirst(); if (++sample%1000==0) Debug.println("Samples processed: "+sample, Debug.MIN_PRIORITY); } catch (NoSuchElementException nseex) { if (!occurredQ.isEmpty()) { Debug.println("NoSuchElementException thrown when queue not empty (Queue size: " + occurredQ.size() + ")", Debug.MIN_PRIORITY); } if (++exception%1000==0) { Debug.println("Exception #"+exception+" has occurred", Debug.MIN_PRIORITY); nseex.printStackTrace(); } waitForSamples(); continue; } catch (Exception ex) { ex.printStackTrace(); exception++; continue; }
try { if(event != null) { Iterator iter = listeners.iterator(); while (iter.hasNext()) { ((SampleListener)iter.next()).sampleOccurred(event); } iter = remoteListeners.iterator();
while (iter.hasNext()) { try { ((RemoteSampleListener)iter.next()).sampleOccurred(event); } catch (Exception ex) { ex.printStackTrace(); } } } else waitForSamples(); } catch (Throwable ex) { ex.printStackTrace(); } this.yield(); } }
private synchronized void waitForSamples() { try { this.wait(); } catch (Exception ex) { ex.printStackTrace(); } } }
(Note that the
I found that a few samples would be taken, and then I would start to get a stream of
NoSuchElementException thrown when queue is not empty (Queue size: 516)
Once this occurs, the items in the queue are never retrieved, and the queue continues to grow. Even with a 1 GB heap, it does not take long before we run out of system resources (at 900+ sampleEvents per second).
I am doing the majority of my work on a Sun Enterprise 420R, with 4 450MHz CPUs and 2 GB RAM. Perhaps these issues are not as visible on machines with smaller horsepower.
Can anyone please either a) clear up any misconceptions that I may have regarding the use of the java LinkedList or b) please suggest an alternate solution to the use of the LinkedList in this implementation?
Thanks in Advance.
Jamie
--------------------------------------------------------------------------------
Jamie Davidson, Product Verification Specialist
Bridgewater Systems Corporation
555 Legget Drive
Suite 800, Tower A
Kanata Ontario K2K 2X3
Phone: (613) 591-6655
Fax: (613) 591-6656
RE: ThreadGroup::SampleQueue woes
Posted by Jamie Davidson <Ja...@bridgewatersystems.com>.
Okay - I figured it out - LinkedList is not serialized, and with multiple threads writing to the queue concurrently, it takes next to no time for the internal Linked List to become unstable.
I resolved this by replacing it with a Vector, which is serialized. This may cause some internal bottlenecks, but these can easily be circumvented by adding multiple thread groups with few threads.
I 'll follow up if I see any significant problems with this modification.
-----Original Message-----
From: Jamie Davidson [mailto:Jamie.Davidson@bridgewatersystems.com]
Sent: May 24, 2001 4:46 PM
To: jmeter-dev@jakarta.apache.org
Subject: ThreadGroup::SampleQueue woes
I think that I have figured out the 'Out Of Memory Errors' that I have been recently enjoying with JMeter 1.6alpha on Solaris 2.7, with Java 1.3.
Apparently the LinkedList class has a flaw in it (or perhaps my knowledge of the LinkedList is lacking - as I said, I am a relative newbie in this development area).
Seems that when the sample is done, the size of the queue is big. I would have expected it to be zero (or close to zero).
Some further investigation turned up the following:
It appears that the LinkedList can get into a state where it throw a NoSuchElementException when removeFirst() is called, even when the list is not empty.
To test this, I changed the SampleQueue class a bit:
private class SampleQueue extends Thread { private LinkedList occurredQ = new LinkedList();
public SampleQueue() { Debug.println("SampleQueue created..."); }
public synchronized void sampleOccurred(SampleEvent e) { occurredQ.addLast(e); this.notify(); }
public long size() { return occurredQ.size(); }
public void run() { Debug.println("Event Queue has started running...", Debug.MIN_PRIORITY); SampleEvent event = null; long sample = 1; long iteration = 1; long exception = 1; while(true) { if (++iteration%1000==0) Debug.println("Processing iteration #"+iteration+" of the Event Queue", Debug.MIN_PRIORITY); //get the first available item in the queue... try { event = (SampleEvent)occurredQ.removeFirst(); if (++sample%1000==0) Debug.println("Samples processed: "+sample, Debug.MIN_PRIORITY); } catch (NoSuchElementException nseex) { if (!occurredQ.isEmpty()) { Debug.println("NoSuchElementException thrown when queue not empty (Queue size: " + occurredQ.size() + ")", Debug.MIN_PRIORITY); } if (++exception%1000==0) { Debug.println("Exception #"+exception+" has occurred", Debug.MIN_PRIORITY); nseex.printStackTrace(); } waitForSamples(); continue; } catch (Exception ex) { ex.printStackTrace(); exception++; continue; }
try { if(event != null) { Iterator iter = listeners.iterator(); while (iter.hasNext()) { ((SampleListener)iter.next()).sampleOccurred(event); } iter = remoteListeners.iterator();
while (iter.hasNext()) { try { ((RemoteSampleListener)iter.next()).sampleOccurred(event); } catch (Exception ex) { ex.printStackTrace(); } } } else waitForSamples(); } catch (Throwable ex) { ex.printStackTrace(); } this.yield(); } }
private synchronized void waitForSamples() { try { this.wait(); } catch (Exception ex) { ex.printStackTrace(); } } }
(Note that the
I found that a few samples would be taken, and then I would start to get a stream of
NoSuchElementException thrown when queue is not empty (Queue size: 516)
Once this occurs, the items in the queue are never retrieved, and the queue continues to grow. Even with a 1 GB heap, it does not take long before we run out of system resources (at 900+ sampleEvents per second).
I am doing the majority of my work on a Sun Enterprise 420R, with 4 450MHz CPUs and 2 GB RAM. Perhaps these issues are not as visible on machines with smaller horsepower.
Can anyone please either a) clear up any misconceptions that I may have regarding the use of the java LinkedList or b) please suggest an alternate solution to the use of the LinkedList in this implementation?
Thanks in Advance.
Jamie
--------------------------------------------------------------------------------
Jamie Davidson, Product Verification Specialist
Bridgewater Systems Corporation
555 Legget Drive
Suite 800, Tower A
Kanata Ontario K2K 2X3
Phone: (613) 591-6655
Fax: (613) 591-6656