Intermittent gump failures

[Sam Ruby <ru...@apache.org>]

We seem to have a truly intermittent failure.  To the point where I 
literally ran the following, and 38 minutes later had a success:

while grep "BUILD FAILED" all
   do
   build xml-axis all | tee all
   done

The failures allways occur at a consistent point.  I inserted trace 
statements into org.apache.axis.encoding.TypeMappingImpl to log events 
whenever {urn:AddrNoImplSEI}Address was accessed.  At the point of the 
failure, there were two calls to register:

>      [java]     at org.apache.axis.encoding.TypeMappingImpl.register(TypeMappingImpl.java:272)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.deployTypeMapping(WSDDService.java:476)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.initTMR(WSDDService.java:583)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.validateDescriptors(WSDDService.java:221)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.<init>(WSDDService.java:206)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDDeployment.<init>(WSDDDeployment.java:242)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDDocument.<init>(WSDDDocument.java:107)
>      [java]     at org.apache.axis.configuration.FileProvider.configureEngine(FileProvider.java:210)
>      [java]     at org.apache.axis.AxisEngine.init(AxisEngine.java:182)
>      [java]     at org.apache.axis.AxisEngine.<init>(AxisEngine.java:167)
>      [java]     at org.apache.axis.server.AxisServer.<init>(AxisServer.java:125)
>      [java]     at org.apache.axis.server.AxisServer.<init>(AxisServer.java:120)
>      [java]     at org.apache.axis.transport.http.SimpleAxisServer.getAxisServer(SimpleAxisServer.java:130)
>      [java]     at org.apache.axis.transport.http.SimpleAxisWorker.run(SimpleAxisWorker.java:187)
>      [java]     at java.lang.Thread.run(Thread.java:479)

and

>      [java]     at org.apache.axis.encoding.TypeMappingImpl.register(TypeMappingImpl.java:272)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.deployTypeMapping(WSDDService.java:476)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.initTMR(WSDDService.java:583)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.validateDescriptors(WSDDService.java:221)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDService.<init>(WSDDService.java:206)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDDeployment.<init>(WSDDDeployment.java:242)
>      [java]     at org.apache.axis.deployment.wsdd.WSDDDocument.<init>(WSDDDocument.java:121)
>      [java]     at org.apache.axis.utils.Admin.processWSDD(Admin.java:155)
>      [java]     at org.apache.axis.utils.Admin.process(Admin.java:228)
>      [java]     at org.apache.axis.utils.Admin.AdminService(Admin.java:109)
>      [java]     at java.lang.reflect.Method.invoke(Native Method)
>      [java]     at org.apache.axis.providers.java.MsgProvider.processMessage(MsgProvider.java:143)
>      [java]     at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:296)
>      [java]     at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:71)
>      [java]     at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:150)
>      [java]     at org.apache.axis.SimpleChain.invoke(SimpleChain.java:120)
>      [java]     at org.apache.axis.server.AxisServer.invoke(AxisServer.java:314)
>      [java]     at org.apache.axis.transport.http.SimpleAxisWorker.run(SimpleAxisWorker.java:379)
>      [java]     at java.lang.Thread.run(Thread.java:479)

Then the first attempt (it is always the first attempt) to access this 
information SOMETIMES fails.  Here is a stack traceback from a 
successful attempt.

>      [java]     at org.apache.axis.encoding.TypeMappingImpl.getClassForQName(TypeMappingImpl.java:490)
>      [java]     at org.apache.axis.description.ServiceDesc.syncOperationToClass(ServiceDesc.java:583)
>      [java]     at org.apache.axis.description.ServiceDesc.getSyncedOperationsForName(ServiceDesc.java:777)
>      [java]     at org.apache.axis.description.ServiceDesc.loadServiceDescByIntrospectionRecursive(ServiceDesc.java:666)
>      [java]     at org.apache.axis.description.ServiceDesc.loadServiceDescByIntrospection(ServiceDesc.java:649)
>      [java]     at org.apache.axis.description.ServiceDesc.loadServiceDescByIntrospection(ServiceDesc.java:633)
>      [java]     at org.apache.axis.description.ServiceDesc.initQNameMap(ServiceDesc.java:442)
>      [java]     at org.apache.axis.description.ServiceDesc.getOperationsByQName(ServiceDesc.java:420)
>      [java]     at org.apache.axis.MessageContext.getPossibleOperationsByQName(MessageContext.java:237)
>      [java]     at org.apache.axis.message.BodyBuilder.onStartChild(BodyBuilder.java:145)
>      [java]     at org.apache.axis.encoding.DeserializationContextImpl.startElement(DeserializationContextImpl.java:861)
>      [java]     at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
>      [java]     at org.apache.xerces.impl.XMLNamespaceBinder.handleStartElement(Unknown Source)
>      [java]     at org.apache.xerces.impl.XMLNamespaceBinder.startElement(Unknown Source)
>      [java]     at org.apache.xerces.impl.dtd.XMLDTDValidator.startElement(Unknown Source)
>      [java]     at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
>      [java]     at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
>      [java]     at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
>      [java]     at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
>      [java]     at org.apache.xerces.parsers.DTDConfiguration.parse(Unknown Source)
>      [java]     at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
>      [java]     at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
>      [java]     at javax.xml.parsers.SAXParser.parse(Unknown Source)
>      [java]     at org.apache.axis.encoding.DeserializationContextImpl.parse(DeserializationContextImpl.java:213)
>      [java]     at org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:459)
>      [java]     at org.apache.axis.Message.getSOAPEnvelope(Message.java:365)
>      [java]     at org.apache.axis.handlers.soap.SOAPService$SOAPRequestHandler.invoke(SOAPService.java:179)
>      [java]     at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:71)
>      [java]     at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:150)
>      [java]     at org.apache.axis.SimpleChain.invoke(SimpleChain.java:120)
>      [java]     at org.apache.axis.server.AxisServer.invoke(AxisServer.java:314)
>      [java]     at org.apache.axis.transport.http.SimpleAxisWorker.run(SimpleAxisWorker.java:379)

The only other piece of relevant information I have gathered so far is 
that these three events occurred against three different TypeMappingImpl 
instances.  This makes no sense to me as the log call was made in the 
"else if (pair != null)" path of the getClassForQName (i.e., the xmlType 
was found in the instance private HashMap.  Following the delegate chain 
one hop in the failure case leads to yet another TypeMappingImpl 
instance - distinct from the other three.

Frustrating.

- Sam Ruby