intermittent XALAN performance issue

["Holk, David A" <da...@groton.pfizer.com>]

Hi everyone, I've got a fun one here I hope one of you enlightened souls can
help me with.

I am experiencing an intermittent performance problem with the
javax.xml.transform.Transformer.transform(src, res) call. I am using xalan
2.4.1.

 - When I run a specific transformation job (exact files data every time) in
my development environment (Windows2000, Weblogic 7.1) it always takes
approximately 15 seconds.

 - When I run this same job in our test and production environments
(Solaris, Weblogic 7.1) it normally takes 15 seconds but sometimes takes
between 15 and 27 minutes to run.

 -  Sometimes if I restart the Weblogic server instance I am deploying to,
the job will go back to taking 15 seconds.

 - When the job appears to be hung, there are plenty of threads and plenty
of RAM available according to the Weblogic console.

I included a thread dump below from one of the Solaris boxes that was taken
while the job was hanging in hopes one of you can make more sense of it than
I can.  

Any insight or suggestions would be appreciated.

David Holk


"ExecuteThread: '4' for queue: 'default'" daemon prio=5 tid=0x11dee0
nid=0x11 waiting on monitor [0xcfa81000..0xcfa819d8]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:415)
        at
weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:105)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:129)

"ExecuteThread: '3' for queue: 'default'" daemon prio=5 tid=0x11d5a0
nid=0x10 waiting on monitor [0xcfb81000..0xcfb819d8]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:415)
        at
weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:105)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:129)

"ExecuteThread: '2' for queue: 'default'" daemon prio=5 tid=0x238510 nid=0xf
runnable [0xcfc81000..0xcfc819d8]
        at
java.io.ObjectOutputStream.writeStreamHeader(ObjectOutputStream.java:700)
        at java.io.ObjectOutputStream.<init>(ObjectOutputStream.java:154)
        at
weblogic.rjvm.OutboundMsgAbbrev.writeObject(OutboundMsgAbbrev.java:80)
        at
weblogic.rjvm.OutboundMsgAbbrev.writeAbbrevs(OutboundMsgAbbrev.java:60)
        at weblogic.rjvm.OutboundMsgAbbrev.write(OutboundMsgAbbrev.java:43)
        at
weblogic.rjvm.MsgAbbrevJVMConnection.writeMsgAbbrevs(MsgAbbrevJVMConnection.
java:186)
        at
weblogic.rjvm.MsgAbbrevJVMConnection.sendMsg(MsgAbbrevJVMConnection.java:154
)
        at
weblogic.rjvm.ConnectionManager.sendMsg(ConnectionManager.java:537)
        at weblogic.rjvm.RJVMImpl.send(RJVMImpl.java:541)
        at
weblogic.rjvm.MsgAbbrevOutputStream.flushAndSendRaw(MsgAbbrevOutputStream.ja
va:250)
        at
weblogic.rjvm.MsgAbbrevOutputStream.flushAndSend(MsgAbbrevOutputStream.java:
258)
        at
weblogic.rjvm.MsgAbbrevOutputStream.send(MsgAbbrevOutputStream.java:319)
        at
weblogic.t3.srvr.ClientRequest.tryToSendObject(ClientContext.java:646)
        at weblogic.t3.srvr.ClientRequest.execute(ClientContext.java:696)
        at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:153)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:134)

"ExecuteThread: '1' for queue: 'default'" daemon prio=5 tid=0x2383d0 nid=0xe
waiting on monitor [0xcfd81000..0xcfd819d8]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:415)
        at
weblogic.kernel.ExecuteThread.waitForRequest(ExecuteThread.java:105)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:129)


LEGAL NOTICE
Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately.

Re: intermittent XALAN performance issue

[Joseph Kesselman <ke...@us.ibm.com>]

Hm. The closest I've seen to that set of symptoms was when I had set the
JVM's maximum heap size much too large, with the result that the beast was
swapping itself to death rather than running GC and bringing itself back
down to a reasonable working set... or when we were reading a file from the
network and there were bandwidth problems on the wire.

I don't suppose you could run this under a performance analyser and
determine where the time is going...?

______________________________________
Joe Kesselman, IBM Next-Generation Web Technologies: XML, XSL and more.
"The world changed profoundly and unpredictably the day Tim Berners Lee
got bitten by a radioactive spider." -- Rafe Culpin, in r.m.filk