Weird 'No deserializer for ... string' errors occur after hours o f successful run

["Shetty, Ashish (Consultant)" <As...@GTECH.COM>]

We have a simple web service -- on Axis 1.1 -- that interfaces with EJB
business components to process 3rd party transactions. The product is
required to be up 24/7. While running a load test on a WebSphere 5.0.2
cluster of 3 nodes, each on a separate Linux system, I have observed strange
errors occurring after a few hours. It is difficult to pinpoint exactly when
this occurs. I have seen it occur a mere 30 minutes after start of test and
most times only after 8 hours or so. In our most recent test it happened
after about 30 hours. Within minutes, the symptoms showed up on other
cluster nodes. Once this problem occurs, Axis is rendered unusable. No
further transactions are entertained and only a server restart remedies the
situation. I have monitored memory and other system resources but none seem
stretched. 

Here is the exception stack trace:

AxisFault
 faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
 faultSubcode: 
 faultString: org.xml.sax.SAXException: Deserializing parameter
'arg0':  could not find deserializer for type
{http://www.w3.org/2001/XMLSchema}string
 faultActor: 
 faultNode: 
 faultDetail: 
        {http://xml.apache.org/axis/}stackTrace: AxisFault
 faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
 faultSubcode: 
 faultString: org.xml.sax.SAXException: Deserializing parameter
'arg0':  could not find deserializer for type
{http://www.w3.org/2001/XMLSchema}string
 faultActor: 
 faultNode: 
 faultDetail: 
 
org.xml.sax.SAXException: Deserializing parameter 'arg0':  could not find
deserializer for type {http://www.w3.org/2001/XMLSchema}string
        at
org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:2
60)
        at
org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:16
9)
        at
org.apache.axis.encoding.DeserializationContextImpl.endElement(Deserializati
onContextImpl.java(Compiled Code))
        at
org.apache.xerces.parsers.AbstractSAXParser.endElement(AbstractSAXParser.jav
a(Compiled Code))
        at
org.apache.xerces.impl.XMLNamespaceBinder.handleEndElement(XMLNamespaceBinde
r.java(Compiled Code))
        at
org.apache.xerces.impl.XMLNamespaceBinder.endElement(XMLNamespaceBinder.java
(Compiled Code))
        at
org.apache.xerces.impl.dtd.XMLDTDValidator.handleEndElement(XMLDTDValidator.
java(Compiled Code))
        at
org.apache.xerces.impl.dtd.XMLDTDValidator.endElement(XMLDTDValidator.java(C
ompiled Code))
        at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.handleEndElement(XMLDo
cumentFragmentScannerImpl.java(Compiled Code))
        at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocu
mentFragmentScannerImpl.java(Compiled Code))
        at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatc
her.dispatch(XMLDocumentFragmentScannerImpl.java(Compiled Code))
        at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocume
ntFragmentScannerImpl.java(Compiled Code))
        at
org.apache.xerces.parsers.StandardParserConfiguration.parse(StandardParserCo
nfiguration.java(Compiled Code))
        at
org.apache.xerces.parsers.StandardParserConfiguration.parse(StandardParserCo
nfiguration.java(Compiled Code))
        at
org.apache.xerces.parsers.AbstractSAXParser.parse(AbstractSAXParser.java(Com
piled Code))
        at javax.xml.parsers.SAXParser.parse(Unknown Source)(Compiled Code)
        at
org.apache.axis.encoding.DeserializationContextImpl.parse(DeserializationCon
textImpl.java(Compiled Code))
        at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java(Compiled
Code))
        at org.apache.axis.Message.getSOAPEnvelope(Message.java(Compiled
Code))
        at org.apache.axis.client.Call.invokeEngine(Call.java:2583)
        at org.apache.axis.client.Call.invoke(Call.java:2553)
        at org.apache.axis.client.Call.invoke(Call.java:1753)
        at
org.apache.jsp._webserviceSimulatorResult.exec(_webserviceSimulatorResult.ja
va:91)
        at
org.apache.jsp._webserviceSimulatorResult._jspService(_webserviceSimulatorRe
sult.java:245)
[truncated]

It is my understanding that the default type mapping accounts for
serializers/deserializers for xsd:string types so the nature of the error is
all the more befuddling. The problem has occurred when transaction rates
were anywhere between 10 t.p.s and 50 t.p.s. I haven't got data for lower or
higher rates so you might not want to take those numbers too seriously. My
team mates and I are *not* web services / Axis newbies and we have exhausted
most avenues before posting this message.

Any ideas or suggestions will be greatly appreciated.

Best regards,

Ashish S Shetty

-----------------------------------------
This email may contain confidential and privileged material for the sole
use of the intended recipient(s). Any review, use, retention, distribution
or disclosure by others is strictly prohibited. If you are not the intended
recipient (or authorized to receive for the recipient), please contact the
sender by reply email and delete all copies of this message. Also, email is
susceptible to data corruption, interception, tampering, unauthorized
amendment and viruses. We only send and receive emails on the basis that we
are not liable for any such corruption, interception, tampering, amendment
or viruses or any consequence thereof.