You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xalan.apache.org by "Rony Zoghby (JIRA)" <xa...@xml.apache.org> on 2005/04/14 18:40:21 UTC

[jira] Created: (XALANJ-2101) DTM Usage in a multithreaded environment

DTM Usage in a multithreaded environment
----------------------------------------

         Key: XALANJ-2101
         URL: http://issues.apache.org/jira/browse/XALANJ-2101
     Project: XalanJ2
        Type: Bug
  Components: DTM  
    Versions: 2.5    
 Environment: All platforms, JDK1.4.2_05, xalan with xerces 2.5
    Reporter: Rony Zoghby
    Priority: Critical


Hi,

We have a multi-threaded memory intensive application with extensive use of xsl processing.
Performance is critical so we're using a caching mechanism of xalan DTM structures 
so not to build the DTM everytime.

Based on the code below, we would like to know if our usage is thread safe, as the DTM
structure is an internal xalan structure.      

The code we're using (a structure of the code rather than the actual code) is the following:

/* 
get the xsl stylesheet templates object from our templates cache using the templatesKey
The templates key is for us the content of the stylesheet 
*/

Templates templates=null;
synchronized(templatesCache){
    templates=cache.getTemplates(templatesKey);
    if(templates==null){
        synchronized(transformerFactory){
            templates=transformerFactory.newTemplates(...);
        }
        cache.addTemplates(templatesKey, templates);
    }
}
        
/*
Each xsl stylesheet templates object has a pool of xsl transformers, so not to create 
a transformer each time.
*/

TransformerImpl transformerImpl=templates.getTransformerPool().getTransfomer();
try{
    transformerImpl.setURIResolver(...);
            
    /*
      Get the DTM structure from our dtmCache, if not in cache create it and put it in cache
      We have a DTM cache so that we don't create the DTM each time and we didn't want to use a DOM
      object because the xalan library transforms it internally to DTM. The dtmKey is the content
      of the XML document that we want to execute the XSL formula on. 
    */  
    DTM dtm=null;
    synchronized(dtmCache){
       dtm=(org.apache.xml.dtm.DTM)dtmCache.getDTM(dtmKey);
       if(dtm==null){
           /*
             We also have a DOMParser cache so that we don't create one each time.
             Here we're building the DTM for the first time and adding it to our DTM cache
           */
           DOMParser domParser=(DOMParser)domParserPool.getDomParser();
           try{
              domParser.setEntityResolver(...);
              InputSource inputSource=new InputSource(new StringReader(...));
              domParser.parse(inputSource);
              Node node=domParser.getDocument();
              DOMSource domSource=new DOMSource(node);
              dtm=transformerImpl.getXPathContext().getDTM(domSource, false, transformerImpl, true, true);
              dtmCache.addDTM(dtmKey, dtm);
           }finally{
              domParserPool.release(domParser);
           }
       }else{
           /* 
             We found the DTM in our DTM cache, so we're setting it in the manager to be able to use that
             DTM for our execution
           */ 
              
   DTMManagerDefault dtmManagerDefault=(DTMManagerDefault)transformerImpl.getXPathContext().getDTMManager();
           dtmManagerDefault.addDTM(dtm, dtmManagerDefault.getFirstFreeDTMID());
       }
    }
    transformerImpl.setQuietConflictWarnings(true);
    /*
       We set the parameters for xsl:param
    */
    transformerImpl.setParameter(strParameterKey, strParameterValue);
    /*
       We execute the XSL stylesheet using the DTM from our DTM cache 
    */
    transformerImpl.transformNode(dtm.getDocument(), new SAXResult(xslQueryHandler));
}finally{
    /* We reset the objects for next execution*/
    transformerImpl.clearParameters();
    transformerImpl.reset();
    templates.getTransformerPool().release(transformerImpl);
}

Using this code, we faced a problem where two same XMLs executed a foreach xsl command
at the same time on the same DTM and we had the xalan library hanging (kill -QUIT
on the java process) in:

at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$IndexedDTMAxisTraverser.getNextIndexed(Unknown Source)
	at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$DescendantTraverser.first(Unknown Source)
	at org.apache.xpath.axes.DescendantIterator.nextNode(Unknown Source)
	at org.apache.xpath.axes.LocPathIterator.asNode(Unknown Source)
	at org.apache.xpath.axes.DescendantIterator.asNode(Unknown Source)
	at org.apache.xpath.axes.LocPathIterator.bool(Unknown Source)
	at org.apache.xpath.XPath.bool(Unknown Source)
	at org.apache.xalan.templates.ElemChoose.execute(Unknown Source)
	at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Unknown Source)
	at org.apache.xalan.templates.ElemIf.execute(Unknown Source)
	at org.apache.xalan.templates.ElemForEach.transformSelectedNodes(Unknown Source)
	at org.apache.xalan.templates.ElemForEach.execute(Unknown Source)

So we guessed that the DTM cannot be shared by two threads, and we're thinking of adding
a synchronized(dtm) just before doing the transformNode. Is this safe?

- Can we also use this code to sequentially execute different XSL stylesheets on the same DTM?
- Is there another safe way of using XML documents caching, DomParser pooling, etc ...?

Thank you.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
If you want more information on JIRA, or have a bug to report see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org


[jira] Assigned: (XALANJ-2101) DTM Usage in a multithreaded environment

Posted by "Brian Minchau (JIRA)" <xa...@xml.apache.org>.
     [ http://issues.apache.org/jira/browse/XALANJ-2101?page=all ]

Brian Minchau reassigned XALANJ-2101:
-------------------------------------

    Assign To: Henry Zongaro

Assigning to Henry Z., per the Xalan-J issue meeting on May 3, 2005.

> DTM Usage in a multithreaded environment
> ----------------------------------------
>
>          Key: XALANJ-2101
>          URL: http://issues.apache.org/jira/browse/XALANJ-2101
>      Project: XalanJ2
>         Type: Bug
>   Components: DTM
>     Versions: 2.5
>  Environment: All platforms, JDK1.4.2_05, xalan with xerces 2.5
>     Reporter: Rony Zoghby
>     Assignee: Henry Zongaro
>     Priority: Critical

>
> Hi,
> We have a multi-threaded memory intensive application with extensive use of xsl processing.
> Performance is critical so we're using a caching mechanism of xalan DTM structures 
> so not to build the DTM everytime.
> Based on the code below, we would like to know if our usage is thread safe, as the DTM
> structure is an internal xalan structure.      
> The code we're using (a structure of the code rather than the actual code) is the following:
> /* 
> get the xsl stylesheet templates object from our templates cache using the templatesKey
> The templates key is for us the content of the stylesheet 
> */
> Templates templates=null;
> synchronized(templatesCache){
>     templates=cache.getTemplates(templatesKey);
>     if(templates==null){
>         synchronized(transformerFactory){
>             templates=transformerFactory.newTemplates(...);
>         }
>         cache.addTemplates(templatesKey, templates);
>     }
> }
>         
> /*
> Each xsl stylesheet templates object has a pool of xsl transformers, so not to create 
> a transformer each time.
> */
> TransformerImpl transformerImpl=templates.getTransformerPool().getTransfomer();
> try{
>     transformerImpl.setURIResolver(...);
>             
>     /*
>       Get the DTM structure from our dtmCache, if not in cache create it and put it in cache
>       We have a DTM cache so that we don't create the DTM each time and we didn't want to use a DOM
>       object because the xalan library transforms it internally to DTM. The dtmKey is the content
>       of the XML document that we want to execute the XSL formula on. 
>     */  
>     DTM dtm=null;
>     synchronized(dtmCache){
>        dtm=(org.apache.xml.dtm.DTM)dtmCache.getDTM(dtmKey);
>        if(dtm==null){
>            /*
>              We also have a DOMParser cache so that we don't create one each time.
>              Here we're building the DTM for the first time and adding it to our DTM cache
>            */
>            DOMParser domParser=(DOMParser)domParserPool.getDomParser();
>            try{
>               domParser.setEntityResolver(...);
>               InputSource inputSource=new InputSource(new StringReader(...));
>               domParser.parse(inputSource);
>               Node node=domParser.getDocument();
>               DOMSource domSource=new DOMSource(node);
>               dtm=transformerImpl.getXPathContext().getDTM(domSource, false, transformerImpl, true, true);
>               dtmCache.addDTM(dtmKey, dtm);
>            }finally{
>               domParserPool.release(domParser);
>            }
>        }else{
>            /* 
>              We found the DTM in our DTM cache, so we're setting it in the manager to be able to use that
>              DTM for our execution
>            */ 
>               
>    DTMManagerDefault dtmManagerDefault=(DTMManagerDefault)transformerImpl.getXPathContext().getDTMManager();
>            dtmManagerDefault.addDTM(dtm, dtmManagerDefault.getFirstFreeDTMID());
>        }
>     }
>     transformerImpl.setQuietConflictWarnings(true);
>     /*
>        We set the parameters for xsl:param
>     */
>     transformerImpl.setParameter(strParameterKey, strParameterValue);
>     /*
>        We execute the XSL stylesheet using the DTM from our DTM cache 
>     */
>     transformerImpl.transformNode(dtm.getDocument(), new SAXResult(xslQueryHandler));
> }finally{
>     /* We reset the objects for next execution*/
>     transformerImpl.clearParameters();
>     transformerImpl.reset();
>     templates.getTransformerPool().release(transformerImpl);
> }
> Using this code, we faced a problem where two same XMLs executed a foreach xsl command
> at the same time on the same DTM and we had the xalan library hanging (kill -QUIT
> on the java process) in:
> at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$IndexedDTMAxisTraverser.getNextIndexed(Unknown Source)
> 	at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$DescendantTraverser.first(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.nextNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.bool(Unknown Source)
> 	at org.apache.xpath.XPath.bool(Unknown Source)
> 	at org.apache.xalan.templates.ElemChoose.execute(Unknown Source)
> 	at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Unknown Source)
> 	at org.apache.xalan.templates.ElemIf.execute(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.transformSelectedNodes(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.execute(Unknown Source)
> So we guessed that the DTM cannot be shared by two threads, and we're thinking of adding
> a synchronized(dtm) just before doing the transformNode. Is this safe?
> - Can we also use this code to sequentially execute different XSL stylesheets on the same DTM?
> - Is there another safe way of using XML documents caching, DomParser pooling, etc ...?
> Thank you.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org


[jira] Resolved: (XALANJ-2101) DTM Usage in a multithreaded environment

Posted by "Henry Zongaro (JIRA)" <xa...@xml.apache.org>.
     [ http://issues.apache.org/jira/browse/XALANJ-2101?page=all ]
     
Henry Zongaro resolved XALANJ-2101:
-----------------------------------

    Resolution: Invalid

So far as I can tell, this does not describe a bug in Xalan-J.  Please reopen this problem report if you feel that there's a bug here.

> DTM Usage in a multithreaded environment
> ----------------------------------------
>
>          Key: XALANJ-2101
>          URL: http://issues.apache.org/jira/browse/XALANJ-2101
>      Project: XalanJ2
>         Type: Bug
>   Components: DTM
>     Versions: 2.5
>  Environment: All platforms, JDK1.4.2_05, xalan with xerces 2.5
>     Reporter: Rony Zoghby
>     Assignee: Henry Zongaro
>     Priority: Critical

>
> Hi,
> We have a multi-threaded memory intensive application with extensive use of xsl processing.
> Performance is critical so we're using a caching mechanism of xalan DTM structures 
> so not to build the DTM everytime.
> Based on the code below, we would like to know if our usage is thread safe, as the DTM
> structure is an internal xalan structure.      
> The code we're using (a structure of the code rather than the actual code) is the following:
> /* 
> get the xsl stylesheet templates object from our templates cache using the templatesKey
> The templates key is for us the content of the stylesheet 
> */
> Templates templates=null;
> synchronized(templatesCache){
>     templates=cache.getTemplates(templatesKey);
>     if(templates==null){
>         synchronized(transformerFactory){
>             templates=transformerFactory.newTemplates(...);
>         }
>         cache.addTemplates(templatesKey, templates);
>     }
> }
>         
> /*
> Each xsl stylesheet templates object has a pool of xsl transformers, so not to create 
> a transformer each time.
> */
> TransformerImpl transformerImpl=templates.getTransformerPool().getTransfomer();
> try{
>     transformerImpl.setURIResolver(...);
>             
>     /*
>       Get the DTM structure from our dtmCache, if not in cache create it and put it in cache
>       We have a DTM cache so that we don't create the DTM each time and we didn't want to use a DOM
>       object because the xalan library transforms it internally to DTM. The dtmKey is the content
>       of the XML document that we want to execute the XSL formula on. 
>     */  
>     DTM dtm=null;
>     synchronized(dtmCache){
>        dtm=(org.apache.xml.dtm.DTM)dtmCache.getDTM(dtmKey);
>        if(dtm==null){
>            /*
>              We also have a DOMParser cache so that we don't create one each time.
>              Here we're building the DTM for the first time and adding it to our DTM cache
>            */
>            DOMParser domParser=(DOMParser)domParserPool.getDomParser();
>            try{
>               domParser.setEntityResolver(...);
>               InputSource inputSource=new InputSource(new StringReader(...));
>               domParser.parse(inputSource);
>               Node node=domParser.getDocument();
>               DOMSource domSource=new DOMSource(node);
>               dtm=transformerImpl.getXPathContext().getDTM(domSource, false, transformerImpl, true, true);
>               dtmCache.addDTM(dtmKey, dtm);
>            }finally{
>               domParserPool.release(domParser);
>            }
>        }else{
>            /* 
>              We found the DTM in our DTM cache, so we're setting it in the manager to be able to use that
>              DTM for our execution
>            */ 
>               
>    DTMManagerDefault dtmManagerDefault=(DTMManagerDefault)transformerImpl.getXPathContext().getDTMManager();
>            dtmManagerDefault.addDTM(dtm, dtmManagerDefault.getFirstFreeDTMID());
>        }
>     }
>     transformerImpl.setQuietConflictWarnings(true);
>     /*
>        We set the parameters for xsl:param
>     */
>     transformerImpl.setParameter(strParameterKey, strParameterValue);
>     /*
>        We execute the XSL stylesheet using the DTM from our DTM cache 
>     */
>     transformerImpl.transformNode(dtm.getDocument(), new SAXResult(xslQueryHandler));
> }finally{
>     /* We reset the objects for next execution*/
>     transformerImpl.clearParameters();
>     transformerImpl.reset();
>     templates.getTransformerPool().release(transformerImpl);
> }
> Using this code, we faced a problem where two same XMLs executed a foreach xsl command
> at the same time on the same DTM and we had the xalan library hanging (kill -QUIT
> on the java process) in:
> at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$IndexedDTMAxisTraverser.getNextIndexed(Unknown Source)
> 	at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$DescendantTraverser.first(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.nextNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.bool(Unknown Source)
> 	at org.apache.xpath.XPath.bool(Unknown Source)
> 	at org.apache.xalan.templates.ElemChoose.execute(Unknown Source)
> 	at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Unknown Source)
> 	at org.apache.xalan.templates.ElemIf.execute(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.transformSelectedNodes(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.execute(Unknown Source)
> So we guessed that the DTM cannot be shared by two threads, and we're thinking of adding
> a synchronized(dtm) just before doing the transformNode. Is this safe?
> - Can we also use this code to sequentially execute different XSL stylesheets on the same DTM?
> - Is there another safe way of using XML documents caching, DomParser pooling, etc ...?
> Thank you.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org


[jira] Commented: (XALANJ-2101) DTM Usage in a multithreaded environment

Posted by "Henry Zongaro (JIRA)" <xa...@xml.apache.org>.
     [ http://issues.apache.org/jira/browse/XALANJ-2101?page=comments#action_65502 ]
     
Henry Zongaro commented on XALANJ-2101:
---------------------------------------

The DOM2DTM implementation always builds the DTM lazily from the DOM tree in a depth-first fashion - as a transformation progresses through the DOM, it builds more of the DTM structures corresponding to those parts of the DOM that it needs to traverse.  If two or more transformations are accessing the same partially built DTM on different threads in an unsynchronized fashion, the DTM will attempt to continue building itself on both of those threads, and is likely to mess up the links it creates in the process.

On top of that, DOM does not guarantee that two threads can access the same DOM tree without synchronization.

You might have more success adding synchronization around the transformNode method invocation, though I can't say for certain whether that will always work.

> DTM Usage in a multithreaded environment
> ----------------------------------------
>
>          Key: XALANJ-2101
>          URL: http://issues.apache.org/jira/browse/XALANJ-2101
>      Project: XalanJ2
>         Type: Bug
>   Components: DTM
>     Versions: 2.5
>  Environment: All platforms, JDK1.4.2_05, xalan with xerces 2.5
>     Reporter: Rony Zoghby
>     Assignee: Henry Zongaro
>     Priority: Critical

>
> Hi,
> We have a multi-threaded memory intensive application with extensive use of xsl processing.
> Performance is critical so we're using a caching mechanism of xalan DTM structures 
> so not to build the DTM everytime.
> Based on the code below, we would like to know if our usage is thread safe, as the DTM
> structure is an internal xalan structure.      
> The code we're using (a structure of the code rather than the actual code) is the following:
> /* 
> get the xsl stylesheet templates object from our templates cache using the templatesKey
> The templates key is for us the content of the stylesheet 
> */
> Templates templates=null;
> synchronized(templatesCache){
>     templates=cache.getTemplates(templatesKey);
>     if(templates==null){
>         synchronized(transformerFactory){
>             templates=transformerFactory.newTemplates(...);
>         }
>         cache.addTemplates(templatesKey, templates);
>     }
> }
>         
> /*
> Each xsl stylesheet templates object has a pool of xsl transformers, so not to create 
> a transformer each time.
> */
> TransformerImpl transformerImpl=templates.getTransformerPool().getTransfomer();
> try{
>     transformerImpl.setURIResolver(...);
>             
>     /*
>       Get the DTM structure from our dtmCache, if not in cache create it and put it in cache
>       We have a DTM cache so that we don't create the DTM each time and we didn't want to use a DOM
>       object because the xalan library transforms it internally to DTM. The dtmKey is the content
>       of the XML document that we want to execute the XSL formula on. 
>     */  
>     DTM dtm=null;
>     synchronized(dtmCache){
>        dtm=(org.apache.xml.dtm.DTM)dtmCache.getDTM(dtmKey);
>        if(dtm==null){
>            /*
>              We also have a DOMParser cache so that we don't create one each time.
>              Here we're building the DTM for the first time and adding it to our DTM cache
>            */
>            DOMParser domParser=(DOMParser)domParserPool.getDomParser();
>            try{
>               domParser.setEntityResolver(...);
>               InputSource inputSource=new InputSource(new StringReader(...));
>               domParser.parse(inputSource);
>               Node node=domParser.getDocument();
>               DOMSource domSource=new DOMSource(node);
>               dtm=transformerImpl.getXPathContext().getDTM(domSource, false, transformerImpl, true, true);
>               dtmCache.addDTM(dtmKey, dtm);
>            }finally{
>               domParserPool.release(domParser);
>            }
>        }else{
>            /* 
>              We found the DTM in our DTM cache, so we're setting it in the manager to be able to use that
>              DTM for our execution
>            */ 
>               
>    DTMManagerDefault dtmManagerDefault=(DTMManagerDefault)transformerImpl.getXPathContext().getDTMManager();
>            dtmManagerDefault.addDTM(dtm, dtmManagerDefault.getFirstFreeDTMID());
>        }
>     }
>     transformerImpl.setQuietConflictWarnings(true);
>     /*
>        We set the parameters for xsl:param
>     */
>     transformerImpl.setParameter(strParameterKey, strParameterValue);
>     /*
>        We execute the XSL stylesheet using the DTM from our DTM cache 
>     */
>     transformerImpl.transformNode(dtm.getDocument(), new SAXResult(xslQueryHandler));
> }finally{
>     /* We reset the objects for next execution*/
>     transformerImpl.clearParameters();
>     transformerImpl.reset();
>     templates.getTransformerPool().release(transformerImpl);
> }
> Using this code, we faced a problem where two same XMLs executed a foreach xsl command
> at the same time on the same DTM and we had the xalan library hanging (kill -QUIT
> on the java process) in:
> at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$IndexedDTMAxisTraverser.getNextIndexed(Unknown Source)
> 	at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$DescendantTraverser.first(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.nextNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.bool(Unknown Source)
> 	at org.apache.xpath.XPath.bool(Unknown Source)
> 	at org.apache.xalan.templates.ElemChoose.execute(Unknown Source)
> 	at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Unknown Source)
> 	at org.apache.xalan.templates.ElemIf.execute(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.transformSelectedNodes(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.execute(Unknown Source)
> So we guessed that the DTM cannot be shared by two threads, and we're thinking of adding
> a synchronized(dtm) just before doing the transformNode. Is this safe?
> - Can we also use this code to sequentially execute different XSL stylesheets on the same DTM?
> - Is there another safe way of using XML documents caching, DomParser pooling, etc ...?
> Thank you.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org


[jira] Updated: (XALANJ-2101) DTM Usage in a multithreaded environment

Posted by "Brian Minchau (JIRA)" <xa...@xml.apache.org>.
     [ http://issues.apache.org/jira/browse/XALANJ-2101?page=all ]

Brian Minchau updated XALANJ-2101:
----------------------------------

    Fix Version: 2.7

> DTM Usage in a multithreaded environment
> ----------------------------------------
>
>          Key: XALANJ-2101
>          URL: http://issues.apache.org/jira/browse/XALANJ-2101
>      Project: XalanJ2
>         Type: Bug
>   Components: DTM
>     Versions: 2.5
>  Environment: All platforms, JDK1.4.2_05, xalan with xerces 2.5
>     Reporter: Rony Zoghby
>     Assignee: Henry Zongaro
>     Priority: Critical
>      Fix For: 2.7

>
> Hi,
> We have a multi-threaded memory intensive application with extensive use of xsl processing.
> Performance is critical so we're using a caching mechanism of xalan DTM structures 
> so not to build the DTM everytime.
> Based on the code below, we would like to know if our usage is thread safe, as the DTM
> structure is an internal xalan structure.      
> The code we're using (a structure of the code rather than the actual code) is the following:
> /* 
> get the xsl stylesheet templates object from our templates cache using the templatesKey
> The templates key is for us the content of the stylesheet 
> */
> Templates templates=null;
> synchronized(templatesCache){
>     templates=cache.getTemplates(templatesKey);
>     if(templates==null){
>         synchronized(transformerFactory){
>             templates=transformerFactory.newTemplates(...);
>         }
>         cache.addTemplates(templatesKey, templates);
>     }
> }
>         
> /*
> Each xsl stylesheet templates object has a pool of xsl transformers, so not to create 
> a transformer each time.
> */
> TransformerImpl transformerImpl=templates.getTransformerPool().getTransfomer();
> try{
>     transformerImpl.setURIResolver(...);
>             
>     /*
>       Get the DTM structure from our dtmCache, if not in cache create it and put it in cache
>       We have a DTM cache so that we don't create the DTM each time and we didn't want to use a DOM
>       object because the xalan library transforms it internally to DTM. The dtmKey is the content
>       of the XML document that we want to execute the XSL formula on. 
>     */  
>     DTM dtm=null;
>     synchronized(dtmCache){
>        dtm=(org.apache.xml.dtm.DTM)dtmCache.getDTM(dtmKey);
>        if(dtm==null){
>            /*
>              We also have a DOMParser cache so that we don't create one each time.
>              Here we're building the DTM for the first time and adding it to our DTM cache
>            */
>            DOMParser domParser=(DOMParser)domParserPool.getDomParser();
>            try{
>               domParser.setEntityResolver(...);
>               InputSource inputSource=new InputSource(new StringReader(...));
>               domParser.parse(inputSource);
>               Node node=domParser.getDocument();
>               DOMSource domSource=new DOMSource(node);
>               dtm=transformerImpl.getXPathContext().getDTM(domSource, false, transformerImpl, true, true);
>               dtmCache.addDTM(dtmKey, dtm);
>            }finally{
>               domParserPool.release(domParser);
>            }
>        }else{
>            /* 
>              We found the DTM in our DTM cache, so we're setting it in the manager to be able to use that
>              DTM for our execution
>            */ 
>               
>    DTMManagerDefault dtmManagerDefault=(DTMManagerDefault)transformerImpl.getXPathContext().getDTMManager();
>            dtmManagerDefault.addDTM(dtm, dtmManagerDefault.getFirstFreeDTMID());
>        }
>     }
>     transformerImpl.setQuietConflictWarnings(true);
>     /*
>        We set the parameters for xsl:param
>     */
>     transformerImpl.setParameter(strParameterKey, strParameterValue);
>     /*
>        We execute the XSL stylesheet using the DTM from our DTM cache 
>     */
>     transformerImpl.transformNode(dtm.getDocument(), new SAXResult(xslQueryHandler));
> }finally{
>     /* We reset the objects for next execution*/
>     transformerImpl.clearParameters();
>     transformerImpl.reset();
>     templates.getTransformerPool().release(transformerImpl);
> }
> Using this code, we faced a problem where two same XMLs executed a foreach xsl command
> at the same time on the same DTM and we had the xalan library hanging (kill -QUIT
> on the java process) in:
> at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$IndexedDTMAxisTraverser.getNextIndexed(Unknown Source)
> 	at org.apache.xml.dtm.ref.DTMDefaultBaseTraversers$DescendantTraverser.first(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.nextNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.DescendantIterator.asNode(Unknown Source)
> 	at org.apache.xpath.axes.LocPathIterator.bool(Unknown Source)
> 	at org.apache.xpath.XPath.bool(Unknown Source)
> 	at org.apache.xalan.templates.ElemChoose.execute(Unknown Source)
> 	at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(Unknown Source)
> 	at org.apache.xalan.templates.ElemIf.execute(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.transformSelectedNodes(Unknown Source)
> 	at org.apache.xalan.templates.ElemForEach.execute(Unknown Source)
> So we guessed that the DTM cannot be shared by two threads, and we're thinking of adding
> a synchronized(dtm) just before doing the transformNode. Is this safe?
> - Can we also use this code to sequentially execute different XSL stylesheets on the same DTM?
> - Is there another safe way of using XML documents caching, DomParser pooling, etc ...?
> Thank you.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: xalan-dev-unsubscribe@xml.apache.org
For additional commands, e-mail: xalan-dev-help@xml.apache.org