processPipelineTo or output modules never return after XML parsing error

["Hunsberger, Peter" <Pe...@STJUDE.ORG>]

This is long, but I'd appreciate any help on a critical problem that is
mucking up our production system.

We use flow and processPipelineTo to run Schematron validation on form
responses.  The flow code is essentially the following:

function _validate( collection ) {
    var sourceURI = "run/_validate/"+collection;
    var destinationURI = "xmodule:request-attr:validate";
    var resolver = null;
    var destination = null;
    var output = null;
    try {
        resolver = cocoon.getComponent( 
	
Packages.org.apache.cocoon.environment.SourceResolver.ROLE );
        destination = resolver.resolveURI( destinationURI );
        output = destination.getOutputStream();
        cocoon.processPipelineTo( sourceURI, {}, output );
        output.close();
        if ( cocoon.request.getAttribute( "validationErrorLevel" ) == 1
) {
            return false;
        }
    }
    catch ( error ) {
        return false;
    }
    finally {
        if (destination != null){
            resolver.release( destination );
        }
        cocoon.releaseComponent( resolver );
    }
    return true;
}         

As you can see we use processPipelineTo to run a Cocoon pipeline which
is passed a particular "collection" name.  This collection name is used
to look up a Schematron template and the pipeline runs this against the
request parameters.  Our analysts use an internally developed editor to
write the Schematron templates.  On rare occasions they manage to
somehow produce invalid XML (a bug we're tracking down).  However, when
this happens the Cocoon pipeline necessarily blows up trying to run the
validation process.  The problem is, that from then on the
processPipelineTo call does not work.   The weird thing is that the
problem now seems to show up for every call from every user, we've got
to shut down Cocoon and restart it to clear up the problem. 

It appears that once the error occurs the last data that was ever
written to the output stream is always returned from then on. I'm
guessing that somehow the output stream isn't cleaned up properly.  I've
tried guessing at some clean up code to add into the finally clause
above but so far no luck; eg. adding an output.close() in the finally
just throws a new exception.

A work around might be to wrap the appropriate Cocoon components in some
kinds of wrapper that contains the exception caused by the invalid XML,
basically acting as though the validation failed. I'm guessing I'd need
to wrap o.a.c.transformation.TraxTransformer but since the validation
template is aggregated with the request data I may need to wrap
o.a.c.sitemap.ContentAggregator? 

Even if I find a local work around I believe that there might be an
underlying bug in Cocoon with processPipelineTo and finalialization
cleanup that needs to be fixed?  Has anyone seen this kind of problem? I
know some people are using similar flow patterns to handle Web services
and I'd imagine that some one must have a Web services pipeline blow up
on occasion?

We're running Cocoon 2.1.4 if that makes a difference: the _validate
pipeline used is an internal pipeline and perhaps the sitemap issue with
error handling on internal pipelines has something to do with this?  I'm
going to try running on Cocoon 2.1.5.1 to see if that makes a difference
but that will take a day or so to get set up, and in the mean time we're
having to restart production once or twice a day, though now that we
know the cause we can slowly eliminate the invalid validation templates,
(hopefully there aren't many...).

Peter Hunsberger

Re: processPipelineTo or output modules never return after XML parsing error

[Sylvain Wallez <sy...@apache.org>]

Sylvain Wallez wrote:

<snip/>

> The implementation of processPipelineTo has been drastically 
> simplified after 2.1.4 and is actually not a simple call to the 
> PipelineUtil class.


Sorry : "... is actually *now* a simple call ..."

Sylvain

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }

Re: processPipelineTo or output modules never return after XML parsing error

[Sylvain Wallez <sy...@apache.org>]

Hunsberger, Peter wrote:

>This is long, but I'd appreciate any help on a critical problem that is
>mucking up our production system.
>
>We use flow and processPipelineTo to run Schematron validation on form
>responses.  The flow code is essentially the following:
>  
>

<snip/>

>We're running Cocoon 2.1.4 if that makes a difference: the _validate
>pipeline used is an internal pipeline and perhaps the sitemap issue with
>error handling on internal pipelines has something to do with this?  I'm
>going to try running on Cocoon 2.1.5.1 to see if that makes a difference
>but that will take a day or so to get set up, and in the mean time we're
>having to restart production once or twice a day, though now that we
>know the cause we can slowly eliminate the invalid validation templates,
>(hopefully there aren't many...).
>  
>

The implementation of processPipelineTo has been drastically simplified 
after 2.1.4 and is actually not a simple call to the PipelineUtil class.

So you may try to change :
    cocoon.processPipelineTo(uri, stream);
to
    
cocoon.createObject(org.apache.cocoon.components.flow.util.PipelineUtil).processToStream(uri, 
stream);

Hope this helps,
Sylvain

-- 
Sylvain Wallez                                  Anyware Technologies
http://www.apache.org/~sylvain           http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }