Heading [OT] Re: Tomcat memory leak error launching web app in NetBeans 6.9.1

[Pid <pi...@pidster.com>]

On 19/10/2010 01:07, Mark Eggers wrote:
> Once again, I apologize for the wall of text. However, most of it is
> pretty quick and dirty code, so it should be easy to skim.
> 
> I'm guessing the end result is harmless?

Well, if the ClassLoader is still extant after it's supposed to have
been cleared up, it isn't so harmless.

> It does seem like a race condition.
>
> Everything seems to work fine
> until shutdown. When DEBUG is set in logging, you get the following in
> catalina.out:

<snip>

> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:595)
> 
> Here's the relevant portions of the class in question (both 
> from org.quartz.simpl.SimpleThreadPool):
> 
>     try {
>         getLog().debug("WorkerThread is shut down.");
>     } catch(Exception e) {
>         // ignore to help with a tomcat glitch
>     }

A Tomcat glitch?  Maybe we should find out what that was, given the edit
was 5 years ago...


> And part of the shutdown code - lots of hard-coded wait times:
> 
> public void shutdown(boolean waitForJobsToComplete) {
>     synchronized (nextRunnableLock) {
>         isShutdown = true;
>         if(workers == null) // case where the pool wasn't even initialize()ed
>             return;
>         // signal each worker thread to shut down
>         Iterator workerThreads = workers.iterator();
>         while(workerThreads.hasNext()) {
>             WorkerThread wt = (WorkerThread) workerThreads.next();
>             wt.shutdown();

This is where the thread is told to shutdown.

>             availWorkers.remove(wt);
>         }

>         // Give waiting (wait(1000)) worker threads a chance to shut down.

It's 1000 in the comment here ^^

>         // Active worker threads will shut down after finishing their
>         // current job.
>         nextRunnableLock.notifyAll();
>         if (waitForJobsToComplete == true) {
>             // wait for hand-off in runInThread to complete...
>             while(handoffPending) {
>                 try { nextRunnableLock.wait(100); } catch(Throwable t) {}
>             }

It's 100 here ^^

>             // Wait until all worker threads are shut down
>             while (busyWorkers.size() > 0) {
>                 WorkerThread wt = (WorkerThread) busyWorkers.getFirst();
>                 try {
>                     getLog().debug(
>                             "Waiting for thread " + wt.getName()
>                                     + " to shut down");
>                     // note: with waiting infinite time the
>                     // application may appear to 'hang'.
>                     nextRunnableLock.wait(2000);

It's 2000 here ^^

>                 } catch (InterruptedException ex) {
>                 }
>             }
>             getLog().debug("shutdown complete");
>         }
>     }
> }
> 
> Looking at the above code, there's a wait of 2.1 seconds per thread?
> If that's the case, then the 3 threads will take up to 6.3 seconds
> while waiting.

6.1s, the 100ms wait only happens once doesn't it?


> What's odd is that I don't see the entry: "Waiting for thread " name "
> to shut down" in any of the logs.

You'd need to set 'debug' on Quartz & the package the class is in, I guess.


> Here's some admittedly ugly code I put together:

<snip>


Tangent: I wonder if removing each Job from the Scheduler prior to
shutdown has a bearing on Quartz ability to find & stop them.

Tomcat's log message reports that the threads aren't actually stopped,
so does Quartz actually wait for them to stop, or does it just wait for
2secs and then move on to the next thread?

Quartz doesn't seem to be aware that the job thread is still running.


p